EXE editing & programming issues

[MrFlibble]

Now that my work on the Super Dune II Second Edition project has slowed down significantly (I'm generally quite busy with other things ATM), I thought that maybe I'd use some help from people who have more expertise in the field of programming.

It is now obvious that to fix some of the issues in Dune 2 that prevent the new sides from working as the original ones, certain modifications to the game executable need to be made. Unfortunately, I generally lack any considerable skills in programming, decompiling, debugging etc., so I thought that maybe someone with enough enthusiasm, capabilities and free time would agree to help me out on this endeavor. Below is a near-complete list of bugs and other problematic issues that need remedy.

The number one issue is the so-called concrete bug. It's a nasty thing that prevents the Mercenaries and Sardaukar from placing concrete foundations properly (well it's not exactly a bug, since these sides were originally not supposed to be playable). Normally, you can place concrete slabs one next to another so far there's buildable rock, and then place the buildings wherever you wish near/on that concrete. The Mercenaries and the Sardaukar can't do this, since they can place slabs and buildings only next to other non-slab and non-wall structures. This is very annoying, as you can't spread out building foundations normally, having to build on incomplete foundations and waste money on repairs. Also, it is almost impossible to leave spaces between buildings for the units to move around, which is also inconvenient when you have a large base.

ab

a. Normal concrete placement (Fremen)

b. Concrete placement error (Sardaukar)

The vanishing Houses bug: Fremen and Mercenary territories appear normal when first drawn on the global map, but "vanish" when the mission is complete, or when the player gets to the global map screen after failing a mission.

EDIT: Both issues have been successfully remedied thanks to segra.

Dune2 Concrete bug fix

SuperDune2: ConquestMap Vanishing Houses Fix

There are other, less annoying problems, most of which could be solved if the data sections in the EXE file that define the specific House parameters (see below) were found. These include:

• the Atreides ending is played for all three new factions (fixed)
• the announcer still says "Atreides", "Harkonnen" and "Ordos" in the side selection screen when choosing the Mercenaries, Sardaukar or Fremen respectively
• frigates arrive at the Starport immediately after the order was sent, instead of a 10 tick delay (fixed)
• all three new sides have the same tech trees, which include all the regular units plus the Atreides Sonic Tank and Harkonnen Devastator (fixed using Nyerguds' editor, however, some tech-tree specs like Ordos building Raiders instead of Trikes and Harkonnen skipping the Barracks are hardcoded)

[nemafakei]

Ok, do the Fremen, Mercenaries and Sardaukar replace AT, OR, HK completely?

If so, some of your problems can be solved with the PAK files; many of the rest shouldn't happen and it may be that you've edited some fields you don't need to.

Apart from the Tech Trees. Those will be tricky.

[stefanhendriks]

In case anybody has the skills to disasemble (reverse engineer) the exe file of dune2. I would be certainly interested in the seed-generator  ::)

I surely hope someone can help you out mrFlibble!

[MrFlibble]

Ok, do the Fremen, Mercenaries and Sardaukar replace AT, OR, HK completely?

In fact Super Dune 2 uses quite a tricky technique to make the Fremen, Mercenaries and Sardaukar playable. Instead of replacing one House with another (that would also require editing things that are hard-coded in the executable), you start the game as a regular House, yet in each starting mission the human player is changed to a new faction. E.g. when you select the leftmost icon on the House selection screen, the first Atreides map, SCENA001.INI, is loaded. However, this file is edited to contain the following lines:

; Scenario 1 control for house Mercenary.

[bASIC]

<...>

[Mercenary]

Quota=1500

Credits=1000

Brain=Human

MaxUnit=25

And tada, when the mission starts, you play as the Mercenaries. If you reload that mission, SCENM001.INI will load, and after completion, SCENM002.INI, SCENM004.INI etc. will be loaded.

I guess doing it the other way (i.e. replacing the original playable Houses with the new ones, changing teamcolors etc. etc.) would be far more difficult than changing the few bugs that the originally unplayable sides have yet still require reverse-engineering/debugging the EXE (since there's no such thing as Rules.ini in Dune II :))

[nemafakei]

OK - you're still changing more than necessary.

Why is there any need for the game to think of the Mercenaries as Mercenaries and not Ordos? All you need to do is change the strings (i.e. not the ini headers, but the language files) and the art.

[MrFlibble]

May I quote myself:

I guess doing it the other way (i.e. replacing the original playable Houses with the new ones, changing teamcolors etc. etc.) would be far more difficult

I can indeed rename Mercenaries into Ordos etc., and also I can change the sound files so that the Mentat will say, "Mercenary unit approaching" instead of "Ordos unit...", etc. etc. This is indeed more or less easily doable (of course it will take some time to edit a dozen or so strings in Dune.eng, and some other stuff as well). However, the only way to change team colors (so that the original Ordos are now yellow and the Mercenaries are green) is by editing the executable, i.e. finding that part of the code which corresponds to House team colors and changing it.

Also, when you select a unit in the game, there's a message in the info box saying, for example, "Atreides Soldier". Well, the House name there is taken directly from the executable rather than from the Dune.eng string file, so changing House names would be problematic since their length is different.

I'm not even talking about the Mentat animations which will be cumbersome to change as well, because the sprites corresponding to the eyes, mouth and so on have fixed positions on the screen (whereas the new sides' Mentats don't, which made my job doing the facial animations easier), so even flipping Cyril with Ammon would already be a problem, to say nothing about adding completely new animations.

[nemafakei]

Can you not do it by editing the pallette file?

[MrFlibble]

Hmm, that's an interesting idea I haven't considered... But won't it screw up other things? Like Mentat images etc. etc.? ???

[nemafakei]

Hm. There are four palettes that Dune II uses:

bene.pal

ibm.pal

intro.pal

westwood.pal

The units are drawn either with ibm or bene. The Bene Gesserit is drawn with bene. Mentats are drawn with either ibm or intro.

It could yet work without too much hassle.

The alternative would be to switch round all the instances of all the remapping colours in all the graphics files. It would work... but I think it would be a little too far from ideal.

Is team.emc editable? That might also have the remap ranges.

[MrFlibble]

AFAIK, the palettes are used as follows:

bene.pal - BG/Mercenary Mentat originally used in the House description screen;

intro.pal - used exclusively for intro and, possibly, cutscenes and outros;

westwood.pal - used for the animated Westwood Studios logo;

ibm.pal - used for everything else

EMC files contain various scripts, and I believe TEAM.EMC is used to define team behavior (meaning teams like Normal, Kamikaze, etc., found in the [TEAMS] section of the scenario INIs). However, EMC files have not been excessively studied (with only one notable exception of BUILD.EMC decompiled by Minniatian), and I'm afraid no one knows for sure what data TEAM.EMC contains.

BTW, there are internal file lists within DUNE2.EXE, and a file HOUSE.INI is mentioned. However, no such file is found among the game files...

[MrFlibble]

OK - you're still changing more than necessary.

Why is there any need for the game to think of the Mercenaries as Mercenaries and not Ordos? All you need to do is change the strings (i.e. not the ini headers, but the language files) and the art.

Can you not do it by editing the pallette file?

Recently I have actually tried out this idea, with some mixed results. First, the different length of side names is not a problem - there's some extra space, and the game recognizes the end of a string by the zero byte at the end, and not by fixed length. However, editing the palette screws up a lot of colors, which is especially noticeable with the Mercenaries, since some of their colors are widely used in the yellowish pattern design of various menus. This means I'd have to edit all the graphics to accommodate such a palette change, and the current SHP editing tools allow for a very limited capability of editing SHPs with remapable color areas (like the "Mentat", "Options" or "Build This" menu buttons).

However, there's another idea. The House/side names, as I mentioned above, are 0-terminated strings in the EXE file (Nyerguds knows what I'm talking about ;)). This is exactly the same as with units, projectiles and structures, meaning that somewhere there is a section of code which is responsible for the House team color, probably its superweapon and, more importantly, the solution to the concrete bug, and this section references such a "House name" string. If this reference is somehow identified (and that's where I need help from a competent programmer), the code section pertaining to a particular House would be detected with a certain degree of accuracy (since the locations of the strings are known).

[CCHyper]

One day Mr Flibble, i will look into some of this for you, i promise ;)

[Nyerguds]

the concrete bug just sems like some loop or counter that only checks 4 sides instead of all 6 -_-

Still I'm WAY too busy with school and TDX to look into Dune II now.

[MrFlibble]

Thanks for support guys :) Some more info about swapping sides (I was a bit sketchy in my post above). The data chunks I'm looking for should contain, from what I was able to test, the following information:

House colors - don't know how they're coded. Maybe each color separately, maybe there's one single reference to a section of the palette.

Coordinates for Mentat facial animation sprites. The coordinates are the X,Y position of the upper left corner pixel of the sprite on a 320x200 screen (the attack arrows on the "Select your next conquest" screen are positioned in the same way ). Hopefully they're just plain HEX numbers. Separate coordinates are for the eyes, mouth, shoulder section (if it covers the image screen) and props like Cyril's book or Ammon's gem in the ring. All coordinates for the new sides' Mentats are 0,0.

All tech tree specifics that are not included in the basic section. The solution to the concrete bug is probably there as well.

Information on what cutscenes should be played for each side. This defines both the intermission after level 5 and the final scene. All three new sides have no intermission cutscenes and the ending for House Atreides. same here, don't know how this info is coded. Perhaps there are separate references to the text lines (in DUNE.ENG and INTRO.ENG) and to the WSA animations/animation frames.

Definition of the herald used on the build list, starport and map selection screens.

Maybe there's other data I'm not aware of as well. By the way, it is interesting to note that when I swapped Harkonnen and Sardaukar, I still had to press H to select Sardaukar on the "Choose your House" screen, but all the other media regarding the selected House (text, mentat image, missions and such) was for the Sardaukar (having an S in the corresponding file name).

[Nyerguds]

I know how house colours work in C&C1... I doubt Dune II will be much different.

Fun fact - remap palettes (in C&C1 anyway) rearrange the ENTIRE palette for the remapped unit/structure - not just the 'team-colour' part ;D

[MrFlibble]

Really? How does that work?

[Nyerguds]

The entire palette for that SHP file is replaced by a re-ordered version. If the palette is kept the same, it just goes from 00, 01, 02, 03, 04 etc up to FF. But in C&C you see that the range of B0 to BF (the team-specific part) is replaced by indexes of the colour they use for the actual team colouring.

For example, the Nod red palette looks like this:

00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F

20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F

30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F

40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F

50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F

60 61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F

70 71 72 73 74 75 76 77 78 79 7A 7B 7C 7D 7E 7F

80 81 82 83 84 85 86 87 88 89 8A 8B 8C 8D 8E 8F

90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F

A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF

7F 7E 7D 7C 7A 2E 78 2F 7D 7C 7B 7A 2A 79 78 78

C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 CA CB CC CD CE CF

D0 D1 D2 D3 D4 D5 D6 D7 D8 D9 DA DB DC DD DE DF

E0 E1 E2 E3 E4 E5 E6 E7 E8 E9 EA EB EC ED EE EF

F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 FA FB FC FD FE FF

As you see, the B0-BF range is replaced by other indexes... these indexes are red colours on the palette. This remapped palette is applied to the SHP file, and put on the playing field as red unit or structure.

[MrFlibble]

Believe it or not, but I managed to locate the corresponding House chunks myself :) However, I used Nyerguds' editor to get the right reference codes to 0-terminated strings. Here's the entire section:

[pre]34 3C F0 32 C8 00 55 00 02 00 90 00 58 02 0A 00

48 00 01 00 06 00 03 00 18 00 3E 3C F0 32 48 3C

F0 32 4D 00 00 00 00 00 A0 00 2C 01 0A 00 41 00

02 00 07 00 04 00 19 00 51 3C F0 32 5B 3C F0 32

80 00 0A 00 01 00 B0 00 2C 01 0A 00 4F 00 03 00

05 00 02 00 1A 00 61 3C F0 32 6B 3C F0 32 0A 00

00 00 00 00 C0 00 2C 01 00 00 4F 00 02 00 05 00

02 00 FF FF 72 3C F0 32 7E 3C F0 32 0A 00 00 00

00 00 D0 00 58 02 00 00 48 00 01 00 06 00 03 00

FF FF 88 3C F0 32 92 3C F0 32 00 00 00 00 00 00

E0 00 2C 01 00 00 4D 00 03 00 07 00 04 00 FF FF

9C 3C F0 32 [/pre]

Which is segmented as follows:

[pre]Harkonnen

34 3C F0 32 C8 00 55 00 02 00 90 00 58 02 0A 00

48 00 01 00 06 00 03 00 18 00 3E 3C F0 32

Atreides

48 3C F0 32 4D 00 00 00 00 00 A0 00 2C 01 0A 00

41 00 02 00 07 00 04 00 19 00 51 3C F0 32

Ordos

5B 3C F0 32 80 00 0A 00 01 00 B0 00 2C 01 0A 00

4F 00 03 00 05 00 02 00 1A 00 61 3C F0 32

Fremen

6B 3C F0 32 0A 00 00 00 00 00 C0 00 2C 01 00 00

4F 00 02 00 05 00 02 00 FF FF 72 3C F0 32

Sardaukar

7E 3C F0 32 0A 00 00 00 00 00 D0 00 58 02 00 00

48 00 01 00 06 00 03 00 FF FF 88 3C F0 32

Mercenary

92 3C F0 32 00 00 00 00 00 00 E0 00 2C 01 00 00

4D 00 03 00 07 00 04 00 FF FF 9C 3C F0 32 [/pre]

So far I have been able to identify some values:

[pre]34 3C F0 32 C8 00 55 00 02 00 90 00 58 02 0A 00

                                          |

                                  Frigate delay time

48 00 01 00 06 00 03 00 18 00 3E 3C F0 32

|    |

| Superweapon type

Mentat voice [/pre]

The first four bytes (in bold) are the reference code to the House name string (Harkonnen, Atreides, etc. - which is used in the scenario INI files), and the last four refer to the VOC file name (probably obsolete) which goes right after the House name in the EXE:[pre]Harkonnen nhark.voc Atreides nattr.voc Ordos nordo.voc Fremen afremen.voc Sardaukar asard.voc Mercenary amerc.voc[/pre] The Mentat voices used are A/H/O/M, perhaps S and F also usable if the corresponding file sets are present.

However, not all of the House parameters are present in the section. When I copied all Ordos parameters (except the superweapon, which I mistakenly copied from the Atreides section, thus learning which byte was responsible for this feature) to the Harkonnen section, I got this:

So basically it didn't even change team color except for the minimap image. Meaning that the graphical stuff like the team colors, the herald on the build screens and Mentat face animation sprites' offsets are located somewhere else.

On the bright side, this study helped me solve the instant frigate arrival issue. As you can see above, each House can have its own value for the number of "ticks" before the frigate arrival. All the three unplayable sides had 0 ticks, which resulted in "Frigate arrival in T minus 0" in v1.0, and in v1.07 they seemingly built in an override command of sorts for the T minus 0 bug, hence we got instant arrival. Also now there shouldn't be any more need in the separate sets of voice files for the Mercenaries and the Fremen (I wanted both of them to use the Atreides female voice, now they can read it directly from ATRE.PAK).

So, any ideas about where else the House data might be located? So far, I thought the House data goes in two directions from its name string: line one goes to the data section above, and line two uses the first letter of the name string to identify the associated media like TEXT#.ENG, MENTAT#.ENG, MENTAT#.CPS, MENSHP#.SHP, as well as the voice files like #ATRE.VOC etc. (where # is the first letter of the House name).

[Nyerguds]

Hm odd... I had expected there to be another reference, to the remap palette...

The C&C1 house section has

-scenario reading side letter

-structure radar colour (as palette index)

-unit radar colour (I think only one colour is used for both in Dune2)

-reference to the remap palette

-reference to the full name

-reference to the short name (not sure what that is ever used for. I think multiplayer network packets)

-the side ID number

The scenario letter seems to be the same thing used for the mentat voice here.

I'll look into this when I got some more time ;)

[edit]

A quick search into the exe reveals that there are no data structures like the C&C1 remap palettes in it (a nice list from 00 to FF is pretty hard to miss). It's possible they only stored the actual row that gets remapped, to save space.

[Nyerguds]

This byte 90 corresponds with the palette location of the team colour.

Harkonnen

34 3C F0 32 C8 00 55 00 02 00 90 00 58 02 0A 00

48 00 01 00 06 00 03 00 18 00 3E 3C F0 32

It's odd that that isn't used as reference for the complete remap palette though... seeing as that colour index is always the first of the 7 colours used for the remap of that side.

[MrFlibble]

Interesting. However, I'm more excited at the possible prospect of finding the data corresponding to the new sides' characteristics, and changing them so that they all play normally. My idea was to fish some more info from the EXE by finding the sprite offsets for Mentat facial animations, because when I swapped Harkonnen and Sardaukar (by changing the string names), the game, when playing as the former Harkonnen, would use the MENTATS.CPS and MENSHPS.SHP, but with the offsets for MENSHPH.SHP (as you may know, all Mentat animation offsets for the Sardaukar and the Fremen are 0,0). I tried locating the offsets (which are, as far as I understand, x,y coordinates for the upper left corner of each sprite) in the EXE (converted them to hex and looked for those values in a consecutive order), but some values just seem not to be there at all. Maybe I'm doing something wrong. The values for the Atreides and Harkonnen Mentats are as follows (the top left corner of the 320x200 screen is 0,0):

[pre]Atreides

Eyes: 40,80

Mouth: 40,96

Shoulder: 128,128

Book: 48,98

Harkonnen

Eyes: 32,88

Mouth: 32,104

Shoulder: 128,104

(all values are in decimal)[/pre]

Perhaps there's also the other way round.

Another thing is the House announcements at the side selection that we have discussed earlier. Even when I swapped Sardaukar and Harkonnen, and would play Sardaukar after having selected what used to be Harks, the announcer would still say "Harkonnen" (using the voice set associated with the Mercenary side), and the hotkey would also be H. So the only way to change that is to make the game actually select the new sides at the selection screen (this would also supposedly remove the necessity to have two 1st missions for each side - one for the initialization and associated with an original playable House, and the other for regular use; e.g. both SCENA001.INI and SCENM001.INI are currently used for the Mercenaries' 1st mission).

[Nyerguds]

Try aligning them to 4 bytes when searching... so for example for the atreides eyes you can then search for either 28 00 00 00 50 00 00 00 or 50 00 00 00 28 00 00 00

[edit]

...or just put em in a row. I found em. Look at address 02F300 :)

Starts with Harkonnen, then Atreides... so the rest is most probably in the normal order of the sides.

Your data on the Atreides book is wrong, btw. It should be 72, 152, just like the data in the exe shows.

The order is Eyes, Mouth, Book, Shoulder. X and Y for each, one byte for each value.

[MrFlibble]

Wow, cool 8) Thanx a lot Nyerguds! :)

Your data on the Atreides book is wrong, btw. It should be 72, 152, just like the data in the exe shows.

Ah, stupid me. I put hex values instead of decimal for that one :O

[MrFlibble]

As you might have noticed, I've made use of the info on the Mentat animations' sprite offsets in the latest release of SD2SE, and since I find this approach of looking for already known data (such as animation offsets) in the EXE useful, I wish to continue this kind of research. Right now I'm interested in the House heralds in the build menu, which, for the regular Houses, looks like this:

Whereas all new sides in Super Dune 2 have it like this:

(also notice that the selection box on the unit list is white, while it is House-colored for regular Houses)

And this is how the actual CPS file within DUNE.PAK looks like:

Obviously the appropriate herald for each House is cut-and-pasted to both top and bottom positions on the left of the screen. However, for the sides in Super Dune 2, there are 5 chunks cut from the very top left corner of the screen (each chunk being 55x8 pixels) instead of the herald. Now the question: does this mean that the regular Heralds (each being 55x40 pixels) actually consist of 5 smaller chunks, or are cut out as a whole? If the latter is assumed, then it remains unexplained why the unused factions have the herald placeholders that consist of five segments - or, rather, one segment which is repeated five times (like the space for the herald is filled with this segment). On the other hand, the "Select your next conquest" screen for the unused sides uses whole House Atreides heralds (positioned at the bottom left and right corners of the screen).

Another question is, what values should I look for? If the program cuts out parts of fixed dimensions (e.g. 55x8 pixels), then the top left coordinate would suffice, and each herald would require 5 such coordinates; otherwise, I think the coordinates for the top left and bottom right corners of each sprite would be needed.

[Nyerguds]

Get the X and Y coordinates for each of them. The Ordos one should give a very nice indication.

BTW won't editing that cps give the no-remap bug again?