Leaderboard

According to information provided by you, I am now able to parse TEMPLATES.BIN properly, and compute DATA.R16 image entry indexes for unit art, building art, projectile art, animation art etc. Here I made a helper program that parses all the values (DATA.R16 image entry indexes are usually those 4-digit numbers) : Sedondly, I discovered the logic how the game determines the version of building or unit (for example Harkonnen get Harkonnen-style buildings, only Emperor gets Emperor-type Construction yard and Heavy Factory, only Smugglers get Smugglers-type Starport, Sandworm gets Atreides-style buildings etc). I can precalculate the unit/building versions, and here are the results (they should match the real game behavior): unit_side_versions.txt building_side_versions.txt @lovalmidas thank you very much for providing all the information you discovered, which pushed me way forward!

I'm continuing my investigation of building template data, and I discovered the meaning of almost all the bytes there! Thanks, althrough most of things you posted there I already discovered on my own, I did not know some of these and that indeed helped me. So here goes the meaning of building template bytes: D 0 Strength (hit points) B 4 Owner side (side bitfield) B 5 Armor type B 6 Turret turning speed (0 = fastest, the higher value the slower) B 7 Turret rate of fire (the higher value the slower frequency of shooting) B 8 ?? (Crewed field in TibEd, but does not affect whether infantry leaves building after destroying/selling) B 9 ?? (only zeros) B 10 ?? (only zeros) B 11 ?? (only zeros) B 12 Primary weapon (weapon type) B 13 Secondary weapon (weapon type) B 14 Sight radius (Armor field in TibEd, which is wrong!) B 15 Act like a turret? When set to 0, turrent won't shoot (1 on turrets, 0 on everything else) B 16 Tech level required for build B 17 Tech level required for 1st upgrade B 18 Tech level required for 2nd upgrade B 19 Tech level required for 3rd upgrade D 20 Building art index D 24 Barrel art Index D 28 Build cost D 32 1st Upgrade cost D 36 2nd Upgrade cost D 40 3rd Upgrade cost D 44 Build speed D 48 1st Upgrade speed D 52 2nd Upgrade speed D 56 3rd Upgrade speed D 60 Power consumption/production D 64 Prerequesite building 1 (building type index) B 68 Prerequesite building 1 owner side needed (side bitfield) B 69 Prerequesite building 1 number of upgrades needed (0 to 3) B 70 ?? (only zeros, probably byte padding) B 71 ?? (only zeros, probably byte padding) D 72 Prerequesite building 2 (building type index) B 76 Prerequesite building 2 owner side needed (side bitfield) B 77 Prerequesite building 2 number of upgrades needed (0 to 3) B 78 Turret requires enough power to work (0 = no, 1 = yes) B 79 Destruction explosion (explosion index) D 80 Tiles occupied by building - all tiles (tile bitfield) D 84 Tiles occupied by building - solid tiles (tile bitfield) B 88 Flags (1 = auto repair just when built, 2 = play buildup animation upon starting map, 4 = sell upon starting map, 8 = disappear upon starting map or after build, 16 = play animation always, 32 = play animation once, 64 = has animation, 128 = show "1" sign (like primary building)) B 89 Flags (1 = ??, 2 = building is selectable, repairable, can be sabotaged, will be set on fire when damaged and will release infantry when destroyed , 4 = can be captured, 128 = decays even on concrete) B 90 Flags (1 = building will still occupy space even after destroyed/sold, 16 = invisible, 32 = draw building skirt, 64 = do not decay without concrete + do not create concrete (used on Sietch), 128 = transparent animation graphics (used on light, heavy and hi-tech factory)) B 91 Flags (1 = cannot be sold) B 92 Special behavior (1 = Radar, 2 = Construction yard, 4 = Refinery, 5 = Silo, 6 = Repair Pad, 7-12 = production, 13 = Starport, 14 = Wall (links, no select cursor), 15 = Concrete, 16 = Turret (links), 17-19 = production) B 93 ?? (various values, probably some order? editing did not have any visible effect) B 94 ?? (only zeros) B 95 Health bar and selection size (0=16px, 1=24px, 2=32px, 3=64px, 4=96px, 5=32px). Sight field in TibEd, which is wrong! B 96 Building exit point 1 X coordinate (MEDIUM GUN TURRET, LARGE GUN TURRET has strange value 248) B 97 Building exit point 1 Y coordinate B 98 Building exit point 2 X coordinate (HARKONNEN PALACE has strange value 60) B 99 Building exit point 2 Y coordinate (HARKONNEN PALACE has strange value 247) B 100-131 Direction frame mapping B 132 ?? (only zeros) B 133 Animation speed (1 = fastest, the higher value the slower animation, 0 = crash) B 134 Art width (pixels) B 135 Art height (pixels) B 136 Building type index B 137 Number of buildup animation frames after which the final building is drawn (the number is about in the middle of animation) B 138 Buildup animation index B 139 Animation index B 140 Firing flash graphics (explosion index) There are some new great discoveries that make some new modding possibilities: - A building can actually be upgraded 3 times (a la Dune 2 heavy factory). This was never used in Dune 2000, but could be useful for mods. - Some new potentially useful building properties found under the Flags bytes (88, 89, 90, 91): Building cannot be sold, Building decays even on concrete (similar to Dune 2 behavior), Auto-repair when building is built without concrete. Those all Flags look to be similar to Tile Attributes in the manner that there are some properties that the game uses internally during gameplay and are not meant to be initially set in Templates.bin. For example when you click on a building to sell it, the game sets flag Byte88 = 4, and the building plays buildup animation in reverse and then disappears. There still remains one big mystery: How the game determines how many infantry are released from a building when building is sold? For example Construction Yard releases 4 infantry, Wind Trap releases 1 infantry, Silo releases no infantry, most other buildings release randomly 2 to 4 infantry etc. This did not change when I edited every possible value in Templates.bin, so this information must be stored elsewhere, or hardcoded in game executable. Also, editing Crewed filed dod not even affect whether infantry was released or not. I also did a research of BUILEXP.BIN file and found what is its purpose: When you shoot building and its health gets under half (and its graphics changes to damaged look), the building will be set on fire, and fire and smoke animations will play over the building (and fade out after some time). And BUILEXP.BIN defines the number of fire/smoke animations and their position on the building. For example modified outposts do not have any animations defined, I never noticed that until today! Here is the file structure, it contains 100 entries, 33 bytes each (total file size is 3300 bytes): B 00 Number of animations (0 to 8) B 01-08 Animation X-offset B 09-16 Animation Y-offset B 17-24 Animation graphics (explosion index, 11 = smoke, 39 = fire) B 25-32 Maximum number of animation frames that can play until animation loops Then I took a look at TECHPOS.BIN. This file is a big mystery: I tried to edit it (rewrite with all zeros), but nothing visibly changed in game! But after renaming this file, the game ended with error, so it definitely reads it. The file has 500 bytes, presumably 100 entries, 5 bytes each. And lastly, the two unknown sections in Templates.bin, one with 86 bytes and second with 256 bytes. The latter one (256 bytes long) is 64 entries per 4 bytes, and it has something to do with Explosions/Animation art. This looks to be Flags, which determine whether animation is transparent or not, etc. The 86 bytes one seem to consist of two sections, in the second half there are explosion indexes, which have something to do with unit behaviors (i.e. there are listed indexes of EX_HARVDUST0 to EX_HARVDUST7 in a row). Editing those values will change animations units are doing (i.e. Harvester harvesting spice and loading into refinery). Zeroing the first half caused various game crashes. @lovalmidas how is your research of Templates.bin and development of Spice2k going? I can see there has not been any new activity recently, so are you taking a (hopefully) temporary break? Like you are busy with other stuff but are still going to continue? You are probably going to develop a new graphics editor and Templates.bin editor (replacement of TibEd), which would be great. I hope my discoveries can help you go further.

So I was doing some research of Building Template data, the meaning of all the byte values. Some values are already covered by TibEd, all the others are available on the Unknown tab in TibEd. I concentrated on meaning of those unknown values. Here is my research do far. Letter B means a single-byte value, D means "Doubleword", a 4-byte value (marked as Long in TibEd). I was playing around with these values: D 80 Tiles occupied by building - all tiles (tile bitfield) D 84 Tiles occupied by building - solid tiles (tile bitfield) and there are very interesting and funny results: Any crazy shape concrete: Yay, a 4x4 concrete, very useful: A compact 2x2 windtrap. More Retro-Dune2 style, isn't it? Then I took advantage of this: B 90 Flags (32 = draw building bottom, 64 = do not decay without concrete + do not create concrete (used on Sietch), 128 = ?? (used on light, heavy and hi-tech factory)) Using value 64, I was able to made a building that won't decay without concrete. So we can finally screw concrete on other planets where it does not make sense, a la Red Alert! Probably someone of you ( @lovalmidas? @Fey? @Cm_blast? @Feda? ) already know meaning of some of the unknown values. For example I noticed you were able to make wall selectible with health bar and shift the selection frame. If you have such knowledge, I would appreciate if you share it here. I also made a complete table of buildings and all the values, which helped me a lot to discover meaning of those values. Here it is, it might be helpful for you too: buildings.txt One thing that I cannot still figure out is, how the game determines specific house's version of building, meaning, if you play as Harkonnen, you can build Harkonnen-style buildings. It looks it has something to do with the "Owner" bitfield, but there's some logic behind this I still don't understand.

I have not had time to read that yet, but the only thing that is really important in UnitTemplate / BuildingTemplate, in relation to data.R16 and rendering, is the UnitArt / BuildingArt / TurretArt entries (as seen in TibEd). If you want to render stealth, perhaps also the Special Logic entry. The idea is that art entries are divided into UnitArt (each entry with X frames * D directions), including turrets BuildingArt (1 BuildingBase + D directions (healthy frames) + D directions (damaged frames)), including turrets ProjectileArt (X frames * D directions) Animations (used by explosions, building buildup animations, building active animations, smokes or some special animations (harvester docking) The sections after those main sections (Unit, Building, Weapon, Explosion) are what is important for the game to inform itself how many of each frames each entry takes. template_bin_sections.txt Data.R16 image order, as known thus far. This includes the context of other images (some to be confirmed). Generally, unit image data starts from the 206th entry, and then the game uses the information obtained from Templates.bin in order to determine how many entries are loaded for one art. sym_addenum.txt I think an editor can afford some leeway to not require exact rendering since an editor is not the game, although you can go ahead with an exact rendering if you want. Carryall and Frigates can be resized to 32x32 without much confusion since a single tile is the atomic unit in the map format (you can't preload 5 infantry in the same tile, unfortunately). For stealth, the stealth 'static' animation seems to be entries 178 - 193. And then the stealth alpha mask seems to be entries 194 - 205. Knowing whether a unit is stealthed is also a bit tricky, since: It involves reading the unit's definition for Special Logic. There is a hardcode in the game that applies stealth for a unit named "STEALTH RAIDER" (probably a coding afterthought since the unit is added in a patch, and the special logic infrastructure is not as modular as I thought it would be)

Okay, so... Here it goes: I'm now able to load buildings and units graphics from DATA.R16 directly, and easily dynamically recolor images with colors from COLOURS.BIN. Thanks to your Spice2k source code (https://github.com/nguoiyoujie/Spice2k/blob/master/src/Dune2000/Structs/R16/ResourceElement.cs) I was able to understand the DATA.R16 format and how to parse it. How is your investigation of TEMPLATES.BIN formats? I could notice UnitTemplate struct format is ready, but BuildingTemplate not yet. That would help me too. Reading graphics data directly from DATA.R16 directly is indeed advantageous, but also some issues. The most important issue is drawing of stealth units (stealth fremen and raider) - those units seem to use regular unit's graphics and the stealth effect is made by the in-game renderer. Also the side color cannot be seen on a unit in stealth mode. I'm thinking how to render stealth units - maybe whole unit shape with a solid color (the side color). The second issue is, that the Carryall and Frigate are too much spacious and they occupy much more space than size of a tile. I wonder how that would look in editor.

For the reveal map 0 not updating the minimap, I found that the game updates the battlefield map and the shroud / minimap separately. For performance saving measures, the shroud and minimap is updated only within an NxN square centered around the reveal point (either a unit or a map location). For range 0, the square size is effectively 0x0 (or 1x1), meaning that effectively nothing is updated on the minimap. A quick fix would be to replace this logic with another that supplies the whole map dimensions. This fix may reduce performance, so I'll try to think of something to help branch this logic conditionally on range = 0. The results is sampled here, check the minimap: As for the minimap itself, this is interesting, but I have no in-depth knowledge of how saves recreate the minimap. Here is what I observed: Before save. (I am Ordos. Ordos structures to the north, Ordos units to the south. Map reveal triggers reveal Harkonnen base.) After loading Change map (dark grey is revealed before save, and shrouded after load) The battlefield itself follows the one after load. So the minimap is actually being revealed one tile beyond sight range into the shroud. Hmm... Edit: Added image for proof of concept

As part of an ongoing effort to collate research into an open-source code, I have created an editor that allows you to edit and preview the campaign map. The link to the project is here: https://github.com/nguoiyoujie/Spice2k (pre-built executables in /build/debug and /build/release) Preview: Discovered descriptions of the campaign.uib file format is here (may be moved in the future ) (Edit:) https://github.com/nguoiyoujie/Spice2k/blob/master/doc/spec/uibCampaign.md

Alright, I managed to look into how the game loads the Data.r8 / Data.r16 file, since previous tests determine that playing with unit counts messes with animations, and animations are images, and images come from the data resource files. Here are the results, thus far: * I suspect UnitTypes and BuildingTypes are simply indices, since they don't appear to carry more data than a name reference. I suspect those two to be relevant to how the sidebar icons are arranged. (e.g. why Wall appears before Refinery) I am not very good at reading assembly, so having Ghidra IDA (it's open source) to decompile to a C-like flow helps a lot. This research has also been made much faster by MVI's old repository where the functions for processing images / BMP / TAR had been discovered, making it much easier to analyze the function loading the data resource file. Some major findings: In analyzing the Data.R16 file, I discovered an issue with the existing Resource Editor. The editor consumes null bytes so the user does not see them. On the contrary, the game does read the null byte as an empty entry (i.e. nothing is rendered). This is critical, as the game performs precise counting of the number of image frames an object has and read the same number of images in the data resource file to assign to that object. The empty entries act as 'padding', and are very important if we want not to replace entries, but to add to them and modify their ordering. For example, Templates.bin specifics 34 image frames for each of the 8 directions for Light Infantry, making a total of 272 images. There are 20 empty entries (situated around the death animations), so the existing editor sees only 252 entries. Unit art is different from Unit. TibEd seems to be already aware of this. The sections in the middle are all numbers instructing the game how many images are to be loaded for each unit art, building art, projectile art, animation (including explosions) etc. All these are used when the game loads the data resource file. I attach a set of my findings below, in three files (within one zip): - A dataR16 folder, which contains the extracted images from the Data.R16 resource file. Each entry is named "test_xxxx_yyyyyy", where xxxx is the image entry ID, and yyyyy is the hexadecimal address of its starting location. Empty entries are represented by 1x1 transparent pixel (actually 0x0 to the game). - sym_addenum.txt, which carries my findings for the game loading Data.R16 file, including memory addresses of where each image is sent to. I consider the Data.R16 loading mechanism to be almost fully discovered. - Template_bin_sections.txt, which carriers my findings for the templates.bin file (seen in above image). dataR16.rar I have also added some code that is able to parse the resource file and include the empty entries: https://github.com/nguoiyoujie/Spice2k/commit/1f3f8ace4f88eaa60ff42473b642da2f0ac9c63d . Credits to mvi, funkyfr3sh and co. for the original resource editor which allowed me to speed up other discoveries. This code was used to generate of the dataR16 folder as attached. (I should make some kind of test suite with the v1.06 files so I don't test on Program.Main())