ArrakisResearchCo

Sorry to be correcting you MrFlibble but the domain is www.dune2.dk - without the "k" :) For now I have just moved the entire site to the new domain but I will be updating the pages again soon.

Tadaah ... :D Great - thanks for the info proyvind - I'll make sure to note those findings along with the rest of the FORM format on my site :)

Well .. first 4 bytes are easy - they define the size which is 4. The next 4 bytes are the real mystery ... I went through my notes but couldnt really find anything about these 4 data bytes. My description of the [sINF][Data] is wrong yes - should be [sINF][Data] - it's this 4 byte data block : 02 02 03 04 None of the values I calculate from them seem to refer to anything else in the file. Third byte could refer to the number of following tags :- ??? ... I'll have to re-write my FORM analyser before I can check the actual values against other sizes / headers / tags within the file. Sorry, at this point I'm clueless about these 4 bytes [Data] :( EDIT: I just checked my TilesClass.cls file in the compression method : Result = Result & "SINF" Result = Result & LongToString(4) ' Data block is 4 bytes Result = Result & Chr(2) & Chr(2) & Chr(3) & Chr(4) ' As of yet unknown data - identical regardless of version I've noted in the sourcecode that these 4 bytes [Data] are unknowns ...

Well ... most of it was dropped from memory off-hand ... sorry for any missing / wrong info - I've been away from this for some 10 months or so - mistakes are bound to be made :P I'm in the process of building my new site and I'll re-check all the research and information I've gathered when I upload everything to dune2.dk along with the new site.

I'll sure give it at try :) Ok - first off - the bytes are ordered like this: first byte of the 4 is most-significant - last byte of the 4 is least-significant. All the multi-byte values are ordered this way. That said ... some FORM formated files may have size definitions which point one byte too "early" within the FORM structure. The FixFORMSize() function "repairs" this error. This function takes the original pointer, the size already read from the data stream and a copy of the buffer. It then copies 4 characters from the buffer at the pointer position plus the size that was passed. It then checks the contents of these 4 characters against the known FORM tags. If one of the known tags turns up the size is correct and then passed back as a result. If none of the tags turn up one is added to the size and the process is repeated until the size has been corrected. For the most part the loop is run only once but for those size values that are incorrect the loop runs twice. (Thanks goes to Olaf for pointing out the FORM structure to me in the first place) In the following: "Tag" = This is a 4 character text that make up the header of a section. "Pointer" / "Size" = A 4 byte value creating a longint value (as described above) "Data" = A block of data the length of the size given along with the tag. When reading from the data stream remember to move your pointer +4 from the current position after reading the 4 byte value you need ! The FORM format is made up of a tag that defines the section followed by a size value. The ICON.ICN file has the following structure : [FORM] : the first entry in the FORM structure - the header - followed by the FORM structure size. Should be the size of the file or filesize -4. Differs. [iCON] : defines that this FORM structure contains ICONs or tiles. This tag does NOT have a size ! [sINF][Data] : StructureINFormation - describes that this block contains a data structure. The first defines the size of the following structure - this is usually 4 - i.e. the next info is the second . The second defines the total number of binary bytes the [Data] block contains. Note that sometimes this second may be faulty in size - this is where the FixFORMSize() function described above is needed to correct for this error. When you've read the second remember to move your pointer 4 bytes so you end up at the beginning of the [Data] block. I never needed to figure out what the [Data] part in this section is actually for so I can't be sure - sorry. [sSET] : defines the StructureSET data block. This block contains TileCount * 128 bytes of "compressed" tile data. [RPAL] : defines the ReferencePALette data block. This block contains the palette cross reference sorted color tables. [RTBL] : defines the ReferenceTable data block. This block contains TileCount bytes of cross references to the RPAL block. I've described each of the last 3 data-blocks in more detail here : http://forum.dune2k.com/index.php?topic=20441.msg339067#msg339067 I can't explain why you get that large number - your pointer could be placed wrong, you are encoding the longint value in reverse order, the file may be corrupt. Hope this helps you a bit :)

The 8 bytes right after the "SINF" tag make up two longint values (4 bytes each). The first of these two denotes the "SINF" header size - usually it's 4 indicating the next longint size. The next longint size denotes the total size of the data block in the "SINF" tag. If you hit my site and download the sourcecode for my Tile Editor you'll find a file called "TilesClass.cls" in the ClassesTiles folder. There's a Sub called "DecodeICON" where all the bytes and tags are handled and also commented ... Part of the FORM decoder in my class : Case "SINF" ' ICON.ICN ' Get the size of the SINF block SINFSize = GetLong(Posi, Buffer) ' Move the pointer past the size block Posi = Posi + 4 ' Fix up the Size SINFSize = FixFORMSize(Posi, SINFSize, Buffer) ' Get the data block of the SINF block SINFData = Mid(Buffer, Posi, SINFSize) ' Move the pointer past the data block to the next header Posi = Posi + SINFSize As you can see the first 4 bytes are fetched as a longint value. It usually has a value of 4 indicating that you should move the pointer an additional 4 bytes for the next value however I discovered that some versions of the file has a header-offset error which is why I implemented the FixFORMSize() function to handle this error and find the correct position in case the header-offset is faulty. The next 4 byte block simply holds the size of the following data block. It should be fairly easy reading VB code - the TilesClass.cls file contains the entire code for handling the ICON.ICN file both ways: decompression and compression including color count check for each tile. http://www.dune2/d2te/ [edit: may 4th '10] Changed the URL to the correct new site

Well - I've already released the sourcecode at my temporary site : http://www.dune2.dk/d2te/ You can download the Tile Editor executable and the VB6 sourcecode from the above page :) Oh - the Tiles class contains all the code for decompressing / compressing the ICON.ICN file. The TileMap class handles the ICON.MAP file. EDIT: Uhm - 8 bytes ? are you referring to the 8 bytes at the beginning of the data block inside ICON.ICN or do you mean in any of the SHP/WSA files ? [edit: may 4th '10] Changed the URL to the correct new site

I've been out of country on a job most of this year but I'm back again. I wrote a Tile Editor that can edit ICON.ICN and ICON.MAP ... it's written in VB6 - should be fairly easy to read the code. Let me know if anyone wants a copy of the two classes that handle those two files. When I get the time to move my site I'll release all the VB6 tools I originally wrote along with the VB6 sourcecodes for each of the tools.

After I fixed my (de)compressor my House of IX didnt have any visible remapping with wrong colors. Maybe there's a structure bug or something or maybe some of the tiles have been encoded wrongly / maybe it has been patched already. I'm not sure why the wrong colors persit. Anyhow - I hope my editor has been of some use and just let you guys know: I'm still working on a few more tweaks to my editor. I'm working on adding the option to edit foundation size for the structures in the editor and to keep the same zoom view regardless of foundation size. Things are a bit slow on the project at the moment tho because of other things :)

Thanks for adding my editor MrFlibble :) I'm still working on the changes we've discussed in the Format80 thread and I'll inform you when the changes are done and the final release is out.

Nyerguds: I'll keep your request about the zoom view in mind :) dvalin: Thanks for the hint. I've already looked at your fine library but it took me a while to figure out which repository tool was needed to download the library. Bazar does the trick.

The grid will "auto-show" regardless of foundation dimensions but will fit the foundation dimensions of the specific TileMap reference (one of the 26). So if one of the TileMap references - say the Repair Facility - is changed from 3x2 to 3x3 the grid will auto-resize to the new dimensions. I do have to copy/shift the previous setup into the new dimensions and then fit the new "empty" tiles with new tile references. My plan is to drop concrete slabs into these "empty" tiles. I have to re-write the TileMap reference setup first though: to read the dimensions from the EXE. This will present a problem: if your editor has been used to change the foundation dimensions the ICON.MAP file will not match and will result in under-/overflow and seriously messed up references. Alternatively I have to add code to patch the EXE from the Tile Editor when changing the foundation size in the editor. I'm still looking into these things :)

Yup :) As I said: I've tried making the editor as simple to use as possible. The latest beta release has the graphics changed for the third tool.

After I cleaned up the original code and rewrote my own (de)compressor I left out the parts that replace colors in the tiles. The result should be that all tiles should be correctly indexed in regards to the colors used. There are minute differences beteween the versions but primarily different selections of the cyan colors used in the IX Research Center. I've tried loading ICON.ICN from all four versions and all seem to be displayed without the remapping errors we have seen on other screenshots. Changing the house colors in the palette will definately screw things up in the game. At least if I look at the changes I've tried making with my editor - like when I changed the concrete slab to use some of the Harkonnen colors instead. Result was that all concrete slabs were colored like the house they belonged to and sometimes when a unit died on one of my slabs it changed color to that enemy units house. Looked pretty cool :) If you want to draw something on a tile that has to be the house color you should use the harkonnen colors to draw it. Notice that my editor shows all house specific parts in the harkonnen colors. The game replaces these pixels with the respective house colors when needed.

I've setup a page describing the editor a bit ... http://www.dune2.dk/d2te/ You can get the beta of the editor here : http://www.dune2.dk/D2TE(Beta).exe Grab a copy of your DUNE.PAK file and load the copy into the editor using File->Open. The first link hopefully describes the tools enough - if not please let me know :) [edit: may 4th '10] Changed the URL to the correct new site