lovalmidas

> Extended limits to maximum of 1024 events and 256 conditions (needed to extend MIS file format) It is refreshing to see someone achieve what I could not. It looks like you have also handled the support for the old capacity and format. Great work! The theoretical limit to the number of different types of events and conditions should be 255 (0xFE) or 256 (0xFF), because that is storage of the type is one byte. The game has an internal check for the event and condition ID so as to produce an error if the read ID exceeds the old limit, but the patch can easily override that. The rest is a matter of developing the switch behavior. Some additional suggestions based on my rusty knowledge, and what I had wanted to do before: - In Event 12 / 0x0C - Switch My Side, there is the option of setting the enable AI. I suggest that we keep that option as it is, but also create an event to set only the AI for a house (similar to Event 06 / 0x06 - Set Build Rate and similar events). I guess it can be called Set AI, and the parameters can even be extended to include House and AI Enabled for a start, but also allow the future inclusion of EnablePractice, BuyUnits, UpgradeBuildings, RebuildBuildings, DontSell, GuardBase and GuardBuildings (more if you are willing to use the full set of 48 bytes in the event to its full potential). A possible favorite for mappers and scripts could be the SpecialWeaponDelay, but that may take up several bytes. - In Event 08 / 0x08 - Set Cash, a good expansion to the event is a switch (we can use the byte reserved for Deploy Action in Reinforcement / Unit Spawn) to determine different behaviors, such as: > 00 - Convert Spice to Cash and Set Cash (the default behavior, spice is set to zero / silos are emptied and the cash amount is set) > 01 - Preserve Spice and Set Cash (preserve the spice amount, and set Cash to the value - Spice, unless there is not enough Cash, then deduct from Spice) > 02 - Convert Spice to Cash and Add Cash (spice is set to zero / silos are emptied, the previous spice amount is added to cash, and the value is added to Cash) > 03 - Convert Spice to Cash and Subtract Cash (spice is set to zero / silos are emptied, the previous spice amount is added to cash, and the value is subtracted from Cash down to 0) > 04 - Convert Spice to Cash and Add Spice (spice is set to zero / silos are emptied, the previous spice amount is added to cash, and the value is added to Spice. Spice may be lost in case of insufficient spice storage) > 05 - Preserve Spice and Add Cash (preserve the spice amount, and the value is added to Cash) > 06 - Preserve Spice and Subtract Cash (preserve the spice amount,, and the value is subtracted from Cash down to 0) > 07 - Preserve Spice and Add Spice (preserve the spice amount, and the value is added to Spice. Spice may be lost in case of insufficient spice storage) > 08 - Preserve Spice and Subtract Spice (preserve the spice amount, and the value is subtracted from Spice down to 0) - Add Event "Add Unit to Starport Shop" (or a similar name), with two parameters: Amount and ID, to add the specified quantity to a unit availability on the Starport order tab. The IDs on the original game shall be: 00=Trike, 01=Quad, 02=Harvester, 03=Combat Tank, 04=MCV, 05=Missile Tank, 06=Siege Tank, 07=Carryall, but can change if Templates.bin is modded. This gives some control over the Starport shop to the mission creator. - In Conditions 00 - Building Exists and 01 - Unit Exists, I do not know how feasible it is, but we can see if conditions Building Count and Unit Count can be made. The search algorithm has to be retrieved from the XX Exists function and studied so it can be made to count instead. XX Exists will be a special case of XX Count where the required count is 1. This can be useful to enforce a defense of several bases (e.g. Protect all 5 Refineries) without consuming precious house allocations to implement via Building Exists. - For the following conditions, possible use of a special House value -1 (0xFF) as "Any House" to indicate a full map search inclusive of all 8 house players (implementation could simply be to run the function 8 times, one for each house and stop on success): > 00 - Building Exists, > 01 - Unit Exists, > 08 - Spice Harvested (if the specific side is shown) - Add Condition "Spice Storage", to compare the amount of Spice Storage similar to "Spice Harvested". This could be useful to enforce fail conditions where the player has lost the ability to acquire new silos to meet a certain "Spice Harvested" quota. - Several more conditions can be made if the in-game statistics can be read. Possibilities include: > "Buildings Destroyed" > "Buildings Lost" > "Units Killed" > "Units Lost" > "XX Building Owned" > "XX Unit Owned" - Outside of the events mechanism, I think it may be possible to allow larger reveal radii by extending CIRCLES.bin. But it also involves a patch into the EXE so that it reads more of CIRCLES.bin than the present 8 entries. As a refresher, CIRCLES.bin comprises of 8 sets of masks (00 - within circle, FF - outside circle), when read from the front, spells the values for 1x1, 3x3, 5x5, 7x7, 9x9, 11x11, 13x13, and 15x15 (radius 7). This could impact sight radius and map reveal events.

In TibEd, this is encoded in the flag "MoveInfantryZone" for each unit, which on a hex editor represents byte (0x100 * n + 0x11 in Templates.bin file, where n is the Unit ID). My experiments seem to tell me that setting determines if the unit is an Infantry or a Vehicle: MoveInfantryZone (in TibEd; this looks more like "IsInfantry". Values: 00 , 01) * If not an infantry: (1) The unit occupies a full square and not a sub-square. (2) The unit no longer moves into infantry-only zones. (3) The unit loses its firing and death animations (and perhaps the death voice too?) (4) The unit emits smoke at half-health (5) The unit is no longer eligible to be ejected from a sold building. In this case the next available infantry, e.g. the Trooper is used instead (6) The unit is eligible for worm attack The last point is the item you are looking for. Unfortunately, this control comes as a single switch that controls several things at once. It is not yet certain if finer controls exist for each individual items. If not, you might need some heavier editing (either through other files, e.g. hide the smoke by figuring out invisible smoke, or editing the game executable itself which is not trivial) if you need just the worm attack.

I have not had time to read that yet, but the only thing that is really important in UnitTemplate / BuildingTemplate, in relation to data.R16 and rendering, is the UnitArt / BuildingArt / TurretArt entries (as seen in TibEd). If you want to render stealth, perhaps also the Special Logic entry. The idea is that art entries are divided into UnitArt (each entry with X frames * D directions), including turrets BuildingArt (1 BuildingBase + D directions (healthy frames) + D directions (damaged frames)), including turrets ProjectileArt (X frames * D directions) Animations (used by explosions, building buildup animations, building active animations, smokes or some special animations (harvester docking) The sections after those main sections (Unit, Building, Weapon, Explosion) are what is important for the game to inform itself how many of each frames each entry takes. template_bin_sections.txt Data.R16 image order, as known thus far. This includes the context of other images (some to be confirmed). Generally, unit image data starts from the 206th entry, and then the game uses the information obtained from Templates.bin in order to determine how many entries are loaded for one art. sym_addenum.txt I think an editor can afford some leeway to not require exact rendering since an editor is not the game, although you can go ahead with an exact rendering if you want. Carryall and Frigates can be resized to 32x32 without much confusion since a single tile is the atomic unit in the map format (you can't preload 5 infantry in the same tile, unfortunately). For stealth, the stealth 'static' animation seems to be entries 178 - 193. And then the stealth alpha mask seems to be entries 194 - 205. Knowing whether a unit is stealthed is also a bit tricky, since: It involves reading the unit's definition for Special Logic. There is a hardcode in the game that applies stealth for a unit named "STEALTH RAIDER" (probably a coding afterthought since the unit is added in a patch, and the special logic infrastructure is not as modular as I thought it would be)

Yep. This is only a piece of the puzzle. I have not yet investigated how allocation indices drive the selection of the player house, but at some point I hope to make an edit to the executable so that the player house is effectively "alloc. index mod 8" instead of "Max( 7, alloc. index )", or a more flexible method, if possible. Loading custom colours.bin is probably the easier (and probably the preferred way in modded content), when you are willing to work with third-party launchers. The benefit of extended Colours.bin file with > 8 colors is mostly research points into finding out how the game handles allocation indices (Funkyfr3sh modded the game so that indices > 2 use Ordos unit voices & Announcer / EVA.) If this can be customized into a mapping (or extended to support more Announcers), that would be sweet (though this may require modifications to templates bin as well). With the templates bin format known, loading from Data.R18 may be possible. But I am still working on a template bin editor (that integrates with an Data.R18 editor) and it is definitely not a trivial endeavour. xD An easier alternative would be to extract only the images as a 8bpp image (I'm trying to get my Data.R18 editor to do that instead of exporting as true color RGB). Recoloring a 8bpp image is much easier as the palette is already embedded in the image. The current editors only export as a true color RGB so you may have to wait for me to finish my version, or try a hand in parsing the data.R18 file yourself.

The game happily reads whatever values that are in there so no crashes. It also happily reads beyond the size of the colours.bin file as padded 0x00 values. The memory space also initializes with 0x00 values. So extra colors were seen as black. I have a customized colours.bin with 3 house color sets (3 x 16 colors) and another with 16 house color sets (16 x 16 colors) to experiment with an exe modification that moves the memory where game stores and subsequently reads colours.bin, to allow proper rendering of 16 colors. Basically, the limitation of 8 house color sets is caused by the fact that the game allocates only enough space for them, and fills the space right after with other data, and it does not protect against invalid house index and happily reads from data meant to be for something else.

I only tried hex value 0x11. And it only reveals the tile if there is an unrevealed tile in some of its 8 directions (which ones is not yet certain). In effect, a unit moving towards a blob of shroud to its west will not reveal them. It doesn't depend on the direction the unit is moving (although that does the impact the visuals as the map is only updated in a limited NxN square region around the unit), it seems to check the surrounding tiles to determine if it should uncover this tile. At some time I could generate some screenshot or video to illustrate this.

For the reveal map 0 not updating the minimap, I found that the game updates the battlefield map and the shroud / minimap separately. For performance saving measures, the shroud and minimap is updated only within an NxN square centered around the reveal point (either a unit or a map location). For range 0, the square size is effectively 0x0 (or 1x1), meaning that effectively nothing is updated on the minimap. A quick fix would be to replace this logic with another that supplies the whole map dimensions. This fix may reduce performance, so I'll try to think of something to help branch this logic conditionally on range = 0. The results is sampled here, check the minimap: As for the minimap itself, this is interesting, but I have no in-depth knowledge of how saves recreate the minimap. Here is what I observed: Before save. (I am Ordos. Ordos structures to the north, Ordos units to the south. Map reveal triggers reveal Harkonnen base.) After loading Change map (dark grey is revealed before save, and shrouded after load) The battlefield itself follows the one after load. So the minimap is actually being revealed one tile beyond sight range into the shroud. Hmm... Edit: Added image for proof of concept

Great work! Probably a no-need-to-fix, I got a EInOutError when I loaded it with my working folder, which has some modified bin files. The cause appears to be the use of a modded Colours.bin which contained only colors for 3 houses (96 bytes) for my experimentation, instead of the expected 8 houses. The application closes after acknowledging this error message. A possible improvement could be to indicate to the user which file has an issue, but it is more of a nice to have. Enjoy your vacation nonetheless!

I apologize for spamming 'research topics'. I wanted to hijack an existing research thread for this file, but I didn't find a dedicated thread for it. The format of circles.bin is straight forward: A visual of the data There is a check that the game makes at address 0x0044e3ba to determine if the map reveal range is more than 7. If so, the game crashes with the message "Shroud circles only defined up to size 7." This much is probably already known. When loading a map, the game loads this file in these exact segments, storing them in these memory locations: Reference pointers to starting from 0x00547ff0 Location of data: 0 = 0x00548044 (1 byte data) (referenced by 0x00547ff0) 1 = 0x00548038 (9 byte data) (referenced by 0x00547ff4) 2 = 0x00548018 (25 byte data) (referenced by 0x00547ff8) 3 = 0x00517b60 (46 byte data) (referenced by 0x00547ffc) 4 = 0x00517b08 (81 byte data) (referenced by 0x00548000) 5 = 0x00517a88 (121 byte data) (referenced by 0x00548004) 6 = 0x005179d8 (169 byte data) (referenced by 0x00548008) 7 = 0x00517d00 (225 byte data) (referenced by 0x0054800c) With these locations, it may be possible to perform some modifications on the game executable to extend the maximum range of a reveal (either by Unit Sight or by Reveal Tile event). We will need: Modify the address of the reference pointers to support the same number of pointers as the desired range. Prepare memory locations for additional data. Have the game assign the reference pointers to the memory locations of additional data. Modify the function loading the file so it loads the required number of bytes to support the desired range. Supply the game with a large enough Circles.bin file to support the desired range. For a range up to 32 cells, 7690 bytes of data is needed. For the full range of 255, that would be 516130 bytes, too costly and impractical for a simple function. This is probably a case of trading space for processor efficiency by loading what is essentially a lookup table for each tile reveal instead of doing time-expensive math. This is a hypothesis, which has to be attempted and tested.

As part of an ongoing effort to collate research into an open-source code, I have created an editor that allows you to edit and preview the campaign map. The link to the project is here: https://github.com/nguoiyoujie/Spice2k (pre-built executables in /build/debug and /build/release) Preview: Discovered descriptions of the campaign.uib file format is here (may be moved in the future ) (Edit:) https://github.com/nguoiyoujie/Spice2k/blob/master/doc/spec/uibCampaign.md

Yeah, I'll try to contact him. There is some stuff I found with regards to where Colours.bin is stored, and which code references them. By changing the location of the data, I was able to preserve more house colors than the original 8 (currently tested until 16 colours, but it may be possible to test up to the practical limit of 256). I'll make another post about it once I know more information about the house alloc. index. In the meantime I'll update my findings for the campaign.uib, and templates.bin. I didn't know about the old wiki. Time for some reading I guess :D

Nice. I have not yet tried the missions yet due to me messing around with other files. I'll definitely try them out someday! :D

On the unit sounds, if you are using TibEd to edit them, there is a bug: The true sounds are: AtreidesReporting1 = Reporting1 AtreidesReporting2 = Reporting2 AtreidesReporting3 = Reporting3 HarkonnenReporting1 = Reporting4 HarkonnenReporting2, (absent in TibEd) HarkonnenReporting3, (absent in TibEd) OrdosReporting1, (absent in TibEd) OrdosReporting2, (absent in TibEd) OrdosReporting3 = Reporting5 Each values takes an integer represent the sound ID. The highlighted region to the right (hex 0x6C to 0x7B) are not seen in TibEd. That could be the reason why you were unable to edit some reporting sounds. At this time, while there is no template editor yet, you can use a hex editor to fix those values: HarkonnenReporting2 location = N * 0x100 + 0x6C, HarkonnenReporting3 location = N * 0x100 + 0x70, OrdosReporting1 location = N * 0x100 + 0x74, OrdosReporting2 location = N * 0x100 + 0x78, where N is the UnitID

I have not yet tried to contact anyone yet, I was spending my time learning how to fiddle with an IDA, and changing the executable currently takes a lower priority than figuring out how to change the various data files (in particular, templates.bin since it looks like the brain of the data). As for modifying how the game loads the .MIS file, I will keep my expectations in reserve until the structure of the .MIS file (in the eyes of the current game) is fully discovered. The bytes for condition and event count are known to us, so that can be used as a trigger point for the game to load extensions if we need them.

Alright, I managed to look into how the game loads the Data.r8 / Data.r16 file, since previous tests determine that playing with unit counts messes with animations, and animations are images, and images come from the data resource files. Here are the results, thus far: * I suspect UnitTypes and BuildingTypes are simply indices, since they don't appear to carry more data than a name reference. I suspect those two to be relevant to how the sidebar icons are arranged. (e.g. why Wall appears before Refinery) I am not very good at reading assembly, so having Ghidra IDA (it's open source) to decompile to a C-like flow helps a lot. This research has also been made much faster by MVI's old repository where the functions for processing images / BMP / TAR had been discovered, making it much easier to analyze the function loading the data resource file. Some major findings: In analyzing the Data.R16 file, I discovered an issue with the existing Resource Editor. The editor consumes null bytes so the user does not see them. On the contrary, the game does read the null byte as an empty entry (i.e. nothing is rendered). This is critical, as the game performs precise counting of the number of image frames an object has and read the same number of images in the data resource file to assign to that object. The empty entries act as 'padding', and are very important if we want not to replace entries, but to add to them and modify their ordering. For example, Templates.bin specifics 34 image frames for each of the 8 directions for Light Infantry, making a total of 272 images. There are 20 empty entries (situated around the death animations), so the existing editor sees only 252 entries. Unit art is different from Unit. TibEd seems to be already aware of this. The sections in the middle are all numbers instructing the game how many images are to be loaded for each unit art, building art, projectile art, animation (including explosions) etc. All these are used when the game loads the data resource file. I attach a set of my findings below, in three files (within one zip): - A dataR16 folder, which contains the extracted images from the Data.R16 resource file. Each entry is named "test_xxxx_yyyyyy", where xxxx is the image entry ID, and yyyyy is the hexadecimal address of its starting location. Empty entries are represented by 1x1 transparent pixel (actually 0x0 to the game). - sym_addenum.txt, which carries my findings for the game loading Data.R16 file, including memory addresses of where each image is sent to. I consider the Data.R16 loading mechanism to be almost fully discovered. - Template_bin_sections.txt, which carriers my findings for the templates.bin file (seen in above image). dataR16.rar I have also added some code that is able to parse the resource file and include the empty entries: https://github.com/nguoiyoujie/Spice2k/commit/1f3f8ace4f88eaa60ff42473b642da2f0ac9c63d . Credits to mvi, funkyfr3sh and co. for the original resource editor which allowed me to speed up other discoveries. This code was used to generate of the dataR16 folder as attached. (I should make some kind of test suite with the v1.06 files so I don't test on Program.Main())