Request for source code for the unofficial Dune2000 patch

[lovalmidas]

Hello fellow commanders / modders,

 

I had spent the past few nights looking through some of the past research on the executable and data files, with the intention of uncovering past limits and removing some limitations of the game.

My main inspirations are:

• Data modding (without modifying the executable)
  • Uncovering what each data bit in each of the major .bin file means
  • Uncovering what each data bit in each of the major table file (.uib) means
  • Uncovering how the game registers a unit / building and links resources to it (bin, data.r8/r16, sounds.rs), allowing the possibility of adding new units / buildings.

 

• Executable modding (modifying the executable)
  • Removing size restrictions (example: the number of entries in Colours.bin), where possible
  • Merging the changes by other patches (if I ever figure out how the individual patches work, like the build queue patch)
  • Expanding the mission triggers system (Conditions / Events system)

 

My main investigative tool is Ghidra IDA. I use it mainly to read and interpret the Dune2000 executable and find where the data gets put in, and leave a trail of annotations / comments. It is like playing Minesweeper on a grid with almost 1.8 million cells (the size of the Gruntlords Dune2000 executable is about 1.795 million bytes).

 

One of the obstacles I face is that I am late to the party. Thus, I am re-discovering what others have found before

• e.g. Intelligent Games had two mission event action codes 12 and 13 defined and named (BLOXFILE, ATTRIBFILE), but an internal parser converts them to invalid code 20, and there is no handling of those codes, suggestive of a scrapped mechanic. I discovered this yesterday. Kloflac, FunkyFr3sh and co. discovered this in 2015.

 

So, I have changed my endeavor to collate what other people have researched, so that I can work on most updated information on the game's internals.

 

I found one of the resources to be very valuable in my initial research, as it collated and documented the list of changes made to the game since the last official patch (1.06): https://github.com/mvi/dune2k

Unfortunately, it is no longer maintained (https://forum.dune2k.com/topic/26661-dune-2000-106p-game-patching-bug-fixes-new-features/page/19/?tab=comments#comment-391397) and a lot of changes have since been made to the current Gruntlord / CnCNet version that were not present in this record:

• For example, the patches to fix several mission events (Trigger Actions as known in the RA/C&C world), described from here onwards: https://forum.dune2k.com/topic/26661-dune-2000-106p-game-patching-bug-fixes-new-features/?do=findComment&comment=390766 , are absent in this source. The addition of the PlayMusic (action code 21) is very valuable patch, and it would be much easier to tweak a C-source file that implements the additional function branch that calls the PlayMusic function and re-patch than re-hack the post-patch executable.

While I could run through both versions and make comparisons to append to my findings, I feel like a lot of time could be saved if the sources for those changes are available somewhere. In addition, I am quite confident I would trip over many of the same bugs from 2015 till now as I try to rebuild my interpretation of other people's changes to match the current game version. 

 

I am thinking of rebuilding the list of changes, and then use the research thus far to suggest new ones. This task is not an urgent task, as I would be spending my time reading all those BIN files first. But if I want to go about making changes to the EXE, without these prior information, it would be quite an inconvenience spending months to re-research what has been researched.

 

Basically, I am hoping that instead of playing a new game of 1.8 million cell Minesweeper, I get to continue some other person's game where some cells have already been uncovered and navigated. I get it that it has been a long time since those research were taken, and the game of research may have been stopped at some parts. I am willing to continue this game.

 

So, here, I ask if there are resources available that lists the source of changes that converted the original Westwood 1.06 patch to the current Gruntlords executable version, and any annotation that came with that research. Hopefully these are public; I am not sure how to go about getting permission to use knowledge that aren't public and put them in my fork of MVI's project.

 

 

[Klofkac]

On 6/30/2020 at 9:12 AM, lovalmidas said:

Hello fellow commanders / modders,

 

I had spent the past few nights looking through some of the past research on the executable and data files, with the intention of uncovering past limits and removing some limitations of the game.

My main inspirations are:

• Data modding (without modifying the executable)
  • Uncovering what each data bit in each of the major .bin file means
  • Uncovering what each data bit in each of the major table file (.uib) means
  • Uncovering how the game registers a unit / building and links resources to it (bin, data.r8/r16, sounds.rs), allowing the possibility of adding new units / buildings.

 

• Executable modding (modifying the executable)
  • Removing size restrictions (example: the number of entries in Colours.bin), where possible
  • Merging the changes by other patches (if I ever figure out how the individual patches work, like the build queue patch)
  • Expanding the mission triggers system (Conditions / Events system)

I'm glad to see someone new on the scene who is eager to continue with our Dune 2000 patching journey. It ended up in 2015 and we live with what those amazing guys made for us for a few years, but we're still finding demands for more things to patch to extend our campaign creation possibilities. For example back in the day I suggested to patch in a feature to be able place static bonus crates in a map, a la Red Alert 1, but this probably ran into some technical difficulties and never got implemented.

One thing I'm wanting the most is to increase the number of events and conditions there can be in a mission, because the current amount is quite low and limiting. There is some spare apparently unused space in .MIS file to place just a few additional events and/or conditions. The other possibility is to increase the size of .MIS files and add any arbitrary amount of extra events and conditions, but to keep compatibility with original files at same time. I was thinking of doing a direct binary modification of the executable, but that would be a bit tough. Having the full game patching source of the very latest patched executable would be extremely helpful.

The patches were later done primarily by @FunkyFr3sh. Did you already attempt to contact him directly? It would be very polite from him if he could provide our community the latest source code, so that we could work further based on his extremely helpful work he did for us. The best is to publish code on github, which elliminates the risk of not being able to reach the person in future and source code being lost forever. I'm having my own code of D2kEditor published on github already and I consider this being a matter of course for any community projects like this.

Thank you @FunkyFr3sh in advance, if you decide positively about sharing your modding treasure with us.

[lovalmidas]

I have not yet tried to contact anyone yet, I was spending my time learning how to fiddle with an IDA, and changing the executable currently takes a lower priority than figuring out how to change the various data files (in particular, templates.bin since it looks like the brain of the data).

As for modifying how the game loads the .MIS file, I will keep my expectations in reserve until the structure of the .MIS file (in the eyes of the current game) is fully discovered. The bytes for condition and event count are known to us, so that can be used as a trigger point for the game to load extensions if we need them.

[D2k Sardaukar]

I don't have much to add, only that - yeah, try to contact FunkyFr3sh directly - he knowns a lot about the executable and might be able to help.

If you do find some new stuff, please share - preferable with images.

Edit: You do know about the old modding wiki?

Some is still here: https://web.archive.org/web/20141023225803/http://d2kplus.com/wiki/index.php?title=Main_Page

Edited July 6, 2020 by D2k Sardaukar

[lovalmidas]

Yeah, I'll try to contact him.

There is some stuff I found with regards to where Colours.bin is stored, and which code references them. By changing the location of the data, I was able to preserve more house colors than the original 8 (currently tested until 16 colours, but it may be possible to test up to the practical limit of 256). I'll make another post about it once I know more information about the house alloc. index.

In the meantime I'll update my findings for the campaign.uib, and templates.bin.

I didn't know about the old wiki. Time for some reading I guess :D