Jump to content
Sign in to follow this  
lovalmidas

Templates.bin research

Recommended Posts

Posted (edited)

I am looking at one of the big .bin files. This is currently a very preliminary research, as at this stage all I can do is to recreate the past discoveries by TibEd and other people, before I can break new ground.

As such, this is a very preliminary research. I hope to share this here so in the future, other people can read and save on the discovery (I can also put my research on GitHub or something, but I have not put my mind into that)

 

Here is what I guess the game executable reads the Templates.bin file and splits it into different areas. These structural divisions is fundamental to how the game interprets the bin file. For instance, the game will reserve a block of memory for Unit data, and a different memory block for Building data.

UjoewkT.png

With the help of TibEd and running the game, these are my guesses on what each section belongs to.

gZsADr5.png

------------------------------------------------------

Text copy of above image

Spoiler

 

Templates.bin
    Units                    up to 0x3c (60) possible? 0x1e (30) used
    UnitBehaviours            
    Buildings                up to 0x64 (100) possible? 0x3e (62) used
    BuildingBehaviours        
    Weapons                    up to 0x40 (64) possible? 0x22 (34) used
    Explosions                up to 0x40 (64) possible? 0x3c (60) used


Allocations:
    
    Size(h)    Size(d)    Offset(h)        Section                                Comment
    0x3c00    15360    000000            Unit definitions                    0x3c instances x 0x100 bytes each            0x1E instances used
    0x68b0    26800    003c00            Building definitions                0x64 instances x 0x10c bytes each            0x3E instances used
    0x0700     1792    00a4b0            Weapon definitions                    0x40 instances x 0x1c bytes each            0x22 instances used
    0x0200      512    00abb0            Explosion definitions                0x40 instances x 0x08 bytes each            0x3C instances used
    0x0168      360    00adb0            ???                                    
    0x0168      360    00af18            ???                                    
    0x0004        4    00b080            ???                                    Value = 1D (unit count related?)
    0x0001        1    00b084            Unit count                            Value = 1E
    0x0004        4    00b085            ???                                    Value = 3C -> qty of next section (explosion count related?)
    0x0100      256    00b089            ???                                    0x3C int values
    0x0001        1    00b189            ???                                    Value = 3C (explosion count related?)
    0x0004        4    00b18a            ???                                    Value = 22 -> qty of next section (weapon count related?)
    0x0100      256    00b18e            ???                                    0x22 int values
    0x0001        1    00b28e            ???                                    Value = 22 (weapon count related?)
    0x0004        4    00b28f            ???                                    Value = 41 -> qty of next section (building count related?)
    0x01e0      480    00b293            ???                                    0x41 int values
    0x0001        1    00b473            Building count                        Value = 3E 
    0xAFC8    45000    00b484            Building name strings                0x64 instances x 0x1c2 bytes each
    0x0c80     3200    01643c            Weapon strings                        0x40 instances x 0x32 bytes each
    0x0c80     3200    0170bc            Explosion strings                    0x40 instances x 0x32 bytes each
    0x6978    27000    017d3c            Unit name strings                    0x3c instances x 0x1c2 bytes each
    0x0BB8     3000    01e6b4            Unit behaviour strings                0x3c instances x 0x32 bytes each
    0x1388     5000    01f26c            Building behaviour strings            0x64 instances x 0x32 bytes each
    0x0056       86    0205f4            ???                                    
    0x0100      256    02064a            ???                                    
    0x0064      100    02074a            ???                                    
    0x0064      100    0207ae            ???                                    
    0x0001        1    020812            ???                                    18
    0x0001        1    020813            ???                                    1A
    END                020814
   

------------------------------------------------------

At a glance, the game appears to support up to 60 unit types, 100 building types, 64 weapon types and 64 explosion types.

I attempted to change what I suspect to be the Unit and Building Count, and I have some varied responses:

  • Decreasing the values and loading a mission where such a value is needed leads to crash on mission load.
    • Setting Unit Count to 0x00 up to 0x19, and loading A1V1 leads to crash at mission load with error "MyVersionOfUnit didn't find a unit".
      ndQT9bj.png
       
    • Setting Unit Count to 0x1A just below the Sandworm ID (25), leads to crash to desktop (no error message)
       
    • Setting Unit Count to 0x1B just equal to the Sandworm ID (25), and loading A1V1 leads to working game, with graphically maimed animations and offset build icons
      DyYMK2G.png2i3iIFK.png 
      Haha, Construction Wind Trap
       
    • Setting Building Count to 0x3B (59), and loading O9V1 leads to crash with error related to the Imperial Palace (ID 59, or the 50thbuilding in the list).
      I2IjhCz.png
       
    • Setting Building Count to 0x3C (60), and loading H5V1 leads to crash with error related to the Harkonnen Special Outpost (ID 60, or the 61st building in the list).
      17nQx0A.png
       
    • Setting Building Count to 0x3D (61), and loading O4V1 doesn't lead to crash, but the player loses instantly (since the Ordos Special Outpost (ID 61, or the 62nd building in the list)... does not exist??). This happens on both the -spawn and normal launches of the game. I am not sure why this behavior is different from the Harkonnen version.

       
  • Changing other values in the surrounding region may lead to lead to very damaging graphical glitches.
    • Clicking on the building queue (to construct a building) may lead to crash due to 'BlitErrors'

 

Hypothesis: Unit Count and Building Count works like what I think they do, but their DATA.r8/.r16 folders may be arranged specifically to the exact number of items. Either this could be controlled by the .bin file, or it is hardcoded and any modder working on adjusting the counts need to also pad their DATA files so that the image sequences are aligned correctly.

This is the extent of my current research. More to come. 

 

 

Edited by lovalmidas
  • Upvote 1

Share this post


Link to post
Share on other sites

Looking great so far, loval. I look forward to when you check out unit sounds, because I need to be able to mod those unit reporting sounds! :P

Share this post


Link to post
Share on other sites
On 6/30/2020 at 10:30 AM, lovalmidas said:

I am looking at one of the big .bin files. This is currently a very preliminary research, as at this stage all I can do is to recreate the past discoveries by TibEd and other people, before I can break new ground.

As such, this is a very preliminary research. I hope to share this here so in the future, other people can read and save on the discovery (I can also put my research on GitHub or something, but I have not put my mind into that)

 

Here is what I guess the game executable reads the Templates.bin file and splits it into different areas. These structural divisions is fundamental to how the game interprets the bin file. For instance, the game will reserve a block of memory for Unit data, and a different memory block for Building data.

Good to see someone is taking a deep look into this.

I personally wanted to do some research of templates.bin file on my own, but I am quite busy with editor development, so did not really get into this much more than taking a quick look into it in a hex editor. I was thinking of loading this file in editor directly and reading some data from it, like the building and unit names, so editor would display them in UI without need to have building and unit names stored in separate editor's configuration files. But the names in templates.bin are not very appropriate, like Sietch is called as Fremen Barracks and other things like that, so I scrapped that idea.

I can see you are using some good tool for decompiling the code, which looks really nice and useful. I tried to use OllyDebugger and tried to read the assembly instructions directly, when I was examining the possibility to push the limits of events and conditions number. Ended up figuring out the game does not load .MIS file as a whole into continuous segment of memory, but it reads individual sections of file into randomly arranged segments of memory. Some editing of game exe would be definitely needed to increase limit of events and conditions number, and the hardest part is keeping the compatibility.

Adding the amount of building and unit types (without replacing existing) is one thing I wanted to see being possible. There are 100 building and 60 unit entries in templates.bin, so the game designers left really huge reserved space for much more stuff to be potentially added in future. Hopefully we can find a way how to take advantage of it. Thumbs up for your further research.

Share this post


Link to post
Share on other sites

Alright, I managed to look into how the game loads the Data.r8 / Data.r16 file, since previous tests determine that playing with unit counts messes with animations, and animations are images, and images come from the data resource files.

 

Here are the results, thus far:

unknown.png

* I suspect UnitTypes and BuildingTypes are simply indices, since they don't appear to carry more data than a name reference. I suspect those two to be relevant to how the sidebar icons are arranged. (e.g. why Wall appears before Refinery)

 

I am not very good at reading assembly, so having Ghidra IDA (it's open source) to decompile to a C-like flow helps a lot. This research has also been made much faster by MVI's old repository where the functions for processing images / BMP / TAR had been discovered, making it much easier to analyze the function loading the data resource file. 

 

Some major findings:

  • In analyzing the Data.R16 file, I discovered an issue with the existing Resource Editor. The editor consumes null bytes so the user does not see them. 
    On the contrary, the game does read the null byte as an empty entry (i.e. nothing is rendered). This is critical, as the game performs precise counting of the number of image frames an object has and read the same number of images in the data resource file to assign to that object. The empty entries act as 'padding', and are very important if we want not to replace entries, but to add to them and modify their ordering.

    For example, Templates.bin specifics 34 image frames for each of the 8 directions for Light Infantry, making a total of 272 images. There are 20 empty entries (situated around the death animations), so the existing editor sees only 252 entries.
     
  • Unit art is different from Unit. TibEd seems to be already aware of this.
     
  • The sections in the middle are all numbers instructing the game how many images are to be loaded for each unit art, building art, projectile art, animation (including explosions) etc. All these are used when the game loads the data resource file.

 

I attach a set of my findings below, in three files (within one zip):

 - A dataR16 folder, which contains the extracted images from the Data.R16 resource file. Each entry is named "test_xxxx_yyyyyy", where xxxx is the image entry ID, and yyyyy is the hexadecimal address of its starting location. Empty entries are represented by 1x1 transparent pixel (actually 0x0 to the game).

 - sym_addenum.txt, which carries my findings for the game loading Data.R16 file, including memory addresses of where each image is sent to. I consider the Data.R16 loading mechanism to be almost fully discovered.

 - Template_bin_sections.txt, which carriers my findings for the templates.bin file (seen in above image). 

dataR16.rar

 

I have also added some code that is able to parse the resource file and include the empty entries: https://github.com/nguoiyoujie/Spice2k/commit/1f3f8ace4f88eaa60ff42473b642da2f0ac9c63d . Credits to mvi, funkyfr3sh and co. for the original resource editor which allowed me to speed up other discoveries. This code was used to generate of the dataR16 folder as attached. (I should make some kind of test suite with the v1.06 files so I don't test on Program.Main()) :D

 

Edited by lovalmidas

Share this post


Link to post
Share on other sites
On 7/1/2020 at 5:43 AM, Fey said:

Looking great so far, loval. I look forward to when you check out unit sounds, because I need to be able to mod those unit reporting sounds! :P

On the unit sounds, if you are using TibEd to edit them, there is a bug:

unknown.png

The true sounds are:

  • AtreidesReporting1 = Reporting1
  • AtreidesReporting2 = Reporting2
  • AtreidesReporting3 = Reporting3
  • HarkonnenReporting1 = Reporting4
  • HarkonnenReporting2, (absent in TibEd)
  • HarkonnenReporting3, (absent in TibEd)
  • OrdosReporting1, (absent in TibEd)
  • OrdosReporting2, (absent in TibEd)
  • OrdosReporting3 = Reporting5

Each values takes an integer represent the sound ID.

 

The highlighted region to the right (hex 0x6C to 0x7B) are not seen in TibEd.

That could be the reason why you were unable to edit some reporting sounds.

At this time, while there is no template editor yet, you can use a hex editor to fix those values: 

  • HarkonnenReporting2 location = N * 0x100 + 0x6C, 
  • HarkonnenReporting3 location = N * 0x100 + 0x70, 
  • OrdosReporting1 location = N * 0x100 + 0x74, 
  • OrdosReporting2 location = N * 0x100 + 0x78, 

where N is the UnitID

 

Share this post


Link to post
Share on other sites
2 hours ago, lovalmidas said:

The highlighted region to the right (hex 0x6C to 0x7B) are not seen in TibEd.

That could be the reason why you were unable to edit some reporting sounds.

At this time, while there is no template editor yet, you can use a hex editor to fix those values: 

  • HarkonnenReporting2 location = N * 0x100 + 0x6C, 
  • HarkonnenReporting3 location = N * 0x100 + 0x70, 
  • OrdosReporting1 location = N * 0x100 + 0x74, 
  • OrdosReporting2 location = N * 0x100 + 0x78, 

where N is the UnitID

Awesome!! I'm so glad to know this is possible. I'm gonna have to contact my VAs and ask for more lines. Man, they're gonna hate me. 😅 But this is gonna be so worth it!!

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...