Instant Seed Map Revealer

[MrFlibble]

Using Nyerguds' Dune II Editor, I have been able to improve the process of revealing and extracting Dune II maps. With this new version of the seed revealer, you'll be able to get new seed maps in no time (more info in the readme file)

Note, however, that the terrain on the maps is represented 100% accurately save for the spice dunes (thick spice), which are represented by regular spice instead. This is, however, irrelevant for programs like Stefan's Dunedit or TCH's Dune 2 Scenario Editor.

I have contacted Olaf van der Spek, who wrote the original tool that extracts the map from the Dune 2 savegames, and he told me he can't find the source for it anymore (it's ten years since he wrote it ), but presumably, the information about all kinds of terrain should be present in the savegame. Maybe if someone disassembled Olaf's tool or could figure out the way how terrain is encoded in the saves, a new, improved map extractor could be created.

MAPREV.zip

[Nyerguds]

So, have you found any limitations on how much sight you can give units? ;D

[edit]

Sixteen? wtf? You put it on 16?

Uh, MrFlibble... Sight is a 2-byte value, meaning it can go up to 32.000. I tested it... just put the sight to 300 and you got an instant map reveal with a SINGLE rocket launcher. It seems the value has no limitation whatsoever :O

Huh, this is odd... the Seed generator doesn't generate a full 64x64 map. It cuts off the first row and column from the map file...

Well I guess these are the map borders anyway so it doesn't really matter.

Anyway, here's the updated exe + ini file. One Launcher, works perfectly :)

[MrFlibble]

Sixteen? wtf? You put it on 16?

In short, I didn't want to edit the INI file. Sixteen is enough to reveal the whole map with the launcher grid I used for the previous version of the revealer. Of course, a single unit can do that as well, but what's the difference? The units present on the map do not interfere with the extraction process - it is only buildings that add extra rock tiles if they're placed over non-rock terrain.

[Nyerguds]

Heh, true but still :P

[ArrakisResearchCo]

Hi guys - I'm a new poster but I've been following this board from the side-line for a while.

All maps created by the seed generator are 64 by 64 however Dune 2 only shows 62 by 62 maps at MapScale=0 because the remaining tiles are used for hidden units and as "spawn-points". My guess is that our friends at Westwood had to include all units including those that spawn at home-/enemy-bases later on in the game in order to keep track of the unit count restrictions for each scenario. Back then the data segment wasnt all that big (64Kb). The last tiles on the maps are usually outside the map range and should be treated as full rock tiles (the full filled brown rock type). There is an exception to this because it is possible to build concrete slabs and wall tiles on these outside tiles aswell as place units there. The last scenario Harkonnen vs. Ordos has a few Atreides units placed outside the map but they usually get stuck there for some reason.

MrFlibble: if you make a backup copy of SCENARIO.PAK you can mess with the INI files as much as you like. All you need is a PAK decompressor / compressor (the download section has one if memory serves - otherwise I can send you one for DOS or make one in a VB6 app for Win). PAK files appear to be identical regardless of which version of Dune 2 is used. I have a few different versions: the one from Abandonia.com - the one from nahoo.com and the one from my original HitSquad CD which is 1.07 containing english, french and german files - i.e. the complete package.

Scenario files are fairly easy to modify. Just always remember that maps are always 64 by 64 when you calculate positions - for example: X = 15 (on the visible map) and Y = 2 (on the visible map) - just add one to each coordinate for the hidden tiles - makes: X = 16 and Y = 3. M = (Y * 64) + X ~ (3 * 64) + 16 = 208.

Units are always prefixed with ID### where ### is a unique number for each unit added. Try to keep these ID numbers unique or the game will crash. You should always keep ID numbers sequential allthough Westwood doesnt always.

ID###=House,UnitType,256,Position,64,Action

I havnt figured out what the 256 and the 64 are but I'm guessing the 256 is for unit health and 64 is some kind of flag setting combined with the Action setting. Structures are identical to units except 64 and Action are left out. Use the structure name instead of the unit name.

Concrete slabs and walls are setup a bit differently. They are prefixed with "GEN" followed by the map location. Use the formula I showed above to calculate the position.

GEN???=House,Type

"Type" is always either "Concrete" or "Wall" and can be nothing else. Replace the "???" with the map coordinate to place the slab or wall.

I can't remember who posted it but someone posted a link to a Windows based visual scenario editor - that anti BillGates dude made it. It's a fairly usable scenario editor I'd recomend using if you don't want to mess around manually.

I've made some experimenting with map-revealing based on scenario editing. I havnt edited the dune2.exe file to change the sight range of units yet. My experiments are based on editing the first scenario for both Harkonnen and Ordos. I use the Ordos scenario to patch in a bunch of trikes, an outpos, windtrap and const yard. Run each unit across the map to reveal it then screenshot dump the entire map using DOSBox' capture feature. Then I convert the png files into 256 colored bmp files and import them into my map analyser. The map analyser reads the map pixel for pixel and compares the play map aswell as the minimap to the tiles in the game and constructs a full map of each scenario. Since the visible map is only 62 by 62 I've built my map analyser to "fix" these missing tiles on the full 64 by 64 map. The result from each analysis is three new bitmaps: 1 full-map bitmap (1024x1024 pixels) where the border is fixed to fit the rest of the map, 1 minimap (64x64 pixels) which is a copy of the real minimap ingame and 1 tilemap (64x64 pixels) where each pixel represents a tile index number.

Sample maps - seed 5 is used here:

Full map (1Mb !) http://www.dune2.dk/FullMap_5.bmp

Minimap http://www.dune2.dk/Minimap_5.bmp

Tilemap http://www.dune2.dk/Tilemap_5.bmp

Since I've patched the scenario to remove the opposing units nothing in the map is changed except the position of the units. I'm considering patching the sight range of certain units and then use my screenshot map analyser to reveal the maps and generate new seed maps.

The only problem this method has is that tile ordering has to be fixed and accepted. My first attempts used a tilemap bitmap I've downloaded somewhere (sorry - can't remember where I got it from) but since then I've successfully decoded the contents of the ICON.ICN file using the C sample shown elsewhere on this board (converted to a VB6 app). I'm considering rebuilding the screenshot analyser but changing the tile order. All landscape feature tiles (sand, dunes, spice, stone, rock) are made up of 16 "identical" tiles for each type of base feature and there are 5 master base features. A fresh tile-map image can be constructed for clones / remakes setting up these base feature tiles in the order sand, dunes, spice, stone, rock followed by the clean sand tile and the two spice bloom tiles. If following this order the Tilemap_##.bmp would be fairly easy to use combined with the new tilemap bitmap.

It takes a lot of time to reveal the maps this way but I've seen a number of posts regarding the seed maps. Some have tried making new seed generators while others are now comparing their different seed-maps finding differences. I have no idea how the savegame files are constructed (yet) but my guess is that they contain some sort of reference to the spice value aswell as the map contours. Since there are 399 tiles in the game the savegame files might contain 2 byte entries for each map tile aswell as some sort of overlay for spice value and structure/unit placement on the map. Using the savegames as a reference to the actual map contents might be a viable source but I'm pretty sure that making visual captures of each seed-map is the right way to go to ensure precise maps.

Given the huge differences between DOS programming and Windows programming it would be nearly impossible to reproduce the Westwood seed generator in a modern environment. The Borland C compiler they used back the handles this very differently from a DOS Pascal compiler and any other compiler based on the Windows environment - i.e. the random generator in each language is different. I've also experimented with a seed generator in VB6 and it works based on parts of the "life simulator" and a bunch of fixed random placements however it's not identical to Westwoods original and never will be.

Hope some of this is usefull  :)

[edit: may 4th '10] Changed the URL to the correct new site

[MrFlibble]

ArrakisResearchCo, in my first post, there's a link to a very easy-to-use Windows-based Dune II scenario editor with a friendly GUI. No need to "calculate" anything by hand anymore :)

Your way of map revealing seems very intricate. I'm sure it would be easier to find a way to extract full tileset information from savegames, since most of it had already been done. However, I'm not skilled enough to write an extractor program myself.

[ArrakisResearchCo]

After I posted my "novel" I found your reference to that particular editor and I went: do'h :P

Anyhow - the new tilemap base I'm going to use as tile-index-reference-thingamajing is based on this image :

http://www.dune2.dk/TileMap.bmp

Bitmap contains 83 landscape feature tiles copied from the original ICON.ICN file merged with IBM.PAL (upped for 256 value RGB palette). The remaining 13 tiles have been painted magenta to allow them to be empty / transparent. The TrueVision3D engine allows for easy-to-setup tilemap usage and setting magenta as transparent. Color index used for magentatiles is color 239 (RGB: 255, 0, 255).

If there's interest I could finish my revealer/analyser and start releasing seed-maps based on this tilemap. I can supply the original tile index numbers aswell if anyone is interested and I can also supply all the tiles with or without magenta areas for transparency. Propably only interesting for cloners more than modders :)

[edit: may 4th '10] Changed the URL to the correct new site

[Nyerguds]

isn't there a better one for that third last tile on the first row? The inwards thing seems kinda out of the ordinary... a straight piece should look better.

[ArrakisResearchCo]

Nyerguds: you mean on the full map ? the stone piece ?

My plan is to rewrite the map revealer to include a smoother routine that will change the border tiles from their current "flat" ones to those that make up the smooth borders everywhere else on the map. With a map-smoother in place those flat tiles on the border would be replaced and make it look more right :)

[Nyerguds]

I mean the third last tile on the first row of the tilemap image. It should be a completely filled rock tile with an edge at the right side, but it looks more like a fjord.

[ArrakisResearchCo]

All the tiles on TileMap.bmp have been copied as they are from Westwoods original ICON.ICN file. If you enlarge my bitmap you'll see that it's actually a "U" like shape that's tilted 90 degrees clockwise. It would propably make sense to re-paint that particular tile to make it look more like the others but I'm guessing those brown rock tiles where drawn the way they are for a reason. Most of the other tiles in the first row are all somewhat "off" compared to all the other tiles with the same configuration. Propably to make the brown rock look more "natural".

[MrFlibble]

So does your revealing method still need to assemble the entire map from screenshots by hand (if I understood your first post on that matter right, that is)?

[ArrakisResearchCo]

My analyser needs screenshots of the entire map so yes there's some manual assembling required. I'll put some more detailed info about the analyser and the process on my site for reference :)

[MrFlibble]

I'm sure I'd generally prefer an easier way around...

[ArrakisResearchCo]

Couldnt agree more. The "easy" way of doing this correctly would be to spend a month taking dune2.exe apart to find their original seed generator and have it setup the maps and export the data to files but that would require someone with highlevel assembler knowledge and a copy of the compiler used by Westwood to create the game.

I havnt figured out how the savegame files are structured yet and I havnt been able to find any documentation anywhere on the structure of these files. If someone has any kind of documentation or knowledge on the savegame files I'd be more than happy to try to create a map converter by using the savegame files instead. Someone made a savegame converter for DOS but the result is ascii characters and there are errors in them. I've seen notes that the existing savegame converter also has problems with structures on the map.

Any help on the savegames would be greatly appreciated - would save me a lot of wasted time doing screenshots :P

[MrFlibble]

You may want to contact Olaf van der Spek. I'll PM you his e-mail now. Tell him I gave it to you if he asks.

[Nyerguds]

All the tiles on TileMap.bmp have been copied as they are from Westwoods original ICON.ICN file.

Ohh, sorry.. I thought the game had multiple tiles for each of those to have some more randomization on the map. I thought there were better alternatived inside Dune II itself. I knew they were taken from the game itself, but I didn't know it were the only ones.

[ArrakisResearchCo]

If you go hunting for Stefans Graphics Pack for "Dune 2 - The Maker" you'll find a bitmap called "bigone.bmp" inside the graphics pack. It contains all the original tiles from the game although a few of them have a some pixel errors. The first tile in Stefans tile image has 4-5 pixles in different colors but they are supposed to be gray and black. The folder containing all the tiles as single bitmaps is missing one tile. There are 399 tiles in the game but the folder only contains 398 bitmaps.

[Nyerguds]

I wish XCC supported those ICN files -_-

[ArrakisResearchCo]

I've managed to write some code that can parse the ICN file and decompress the correct tiles but I havnt quite figured out how the compression actually works. I just needed to get the correct tiles but if I get the time I might be able to figure out how it works and then maybe write a tile editor. The FNT files were easy enough to figure out and I think that the ICN file uses similar tricks to compress the tiles.

[Nyerguds]

Awesome. A tool to edit terrain and buildings would be really awesome.

[ArrakisResearchCo]

I'm working on a font editor - currently it supports both Dune 2 and C&C FNT files - but it doesnt encode the resulting FNT files correctly yet. My FNT encoder seems to have trouble selecting a default character for characters that have not been defined resulting in larger files.

When I load the INTRO.FNT from Dune 2 it uses some 128 characters from the character set however a lot of the characters point to offset 194h which is defined as an exclamation point. My guess is that our friends at Westwood manually defined these "unused" characters to point to the exclamation point character. Problem is: when I decompress each character into my editor I create a seperate workspace for each character - that means I get copies of the same character. Since the original fonts only use references to the first defined character I need some sort of automation that can define those "unused" characters instead of just copying the character they point to. I still have some way to go on the font editor project.

The tiles in Dune 2 are located in the compressed ICON.ICN file. I have yet to figure out how the compression works before I can write a compression tool. If I can figure out how the compression is designed I might be able to write a tool to compress the tiles. Decompressing them works at the moment and grabbing the decompressed pixels and throwing them into a pixel editor shouldnt be a problem. The problem is to compress them back into the correct format when storing the changed tiles.

Hmm ... maybe I should take a different aproach to this: start with a pixel editor for images and then simply decompress the images into raw pixel data for the pixel editor. It should then be a matter of figuring out the different compression formats and writing compression/decompression functions for the different image formats. Decompress the image into the pixel editor and compress the raw pixel data from the editor back into the correct format for the file - or export the raw pixel data to a bitmap image. I'll look into this.

[Nyerguds]

Heh, I dunno. The general way of doing stuff like that is just to make a tool to convert them to/from paletted PCX or PNG so people can edit em with whatever program they want.

[XTF]

I've managed to write some code that can parse the ICN file and decompress the correct tiles but I havnt quite figured out how the compression actually works. I just needed to get the correct tiles but if I get the time I might be able to figure out how it works and then maybe write a tile editor. The FNT files were easy enough to figure out and I think that the ICN file uses similar tricks to compress the tiles.

Isn't it just raw 16 x 16 pixel data?

Hmm, no. Have a look at http://xccu.svn.sourceforge.net/viewvc/xccu/trunk/xd2/misc/icon_form_decoder.cpp?revision=300&view=markup

http://xhp.xwis.net/temp/dune2_icons.png ;)

[ArrakisResearchCo]

The tiles seem to be in some form of nibble compression with a compression rate of about 50%.

16 * 16 = 256 ... 256 * 399 = 102144 bytes - and the file is only 52893 bytes.

Olaf found a different version of the tile decompression C code - there's one more on this board somewhere which I used as a source of inspiration for my code. I havnt figured out how the 16 entry shift-table was constructed in the ICN file but I'm hoping to be able to decipher it in time.

I've dumped a copy of my ICON.MAP reference at my site for future reference and I'll setup a description of the GRAYRMAP.TBL file today.