[Dune 2] Dune II editor with 1.07 support

[Nyerguds]

CURRENT LATEST VERSION: 1.17.0
http://nyerguds.arsaneus-design.com/dune/dune2edit/d2editor_v1.17.0.zip


LATEST SCREENSHOT:






I've made a separate topic for the developement of this editor, since it was kinda off-topic in the SD2 thread.

Anyway... the current version can finally read the exe file! It's not much, but it can follow the references in the exe file to find the unit's name and WSA file strings.

At the moment, that's the only thing it does though, in terms of actual editing-related stuff. I still need to implement the structures data, and define all of the different data types to edit before I can do much more.

At the moment it's not too useful for editing itself, but anyway, it's coming along nicely

All future versions will be dumped into this folder:
http://nyerguds.arsaneus-design.com/dune/dune2edit/


[update]

I just added the structures list, and adapted the program to make it read the units and structures list straight from the exe file, using the name references
I also implemented the data structure for reading ownership values.

Screenshot:

[MrFlibble]

Looks very good Excellent work!

These are the types I need lists of:

ownership (got that one)
commands (unit sidebar commands)
Movement types
weapon types
structures (for required structures - is this a bit switch list too?)
units (for construction options of buildings - or are the values the same as the normal unit order?)

AFAIK, the order of unit commands, units and structures is the same as the list of their names in the DUNE.ENG string table file (with the exception of warheads, which do not need text strings). Perhaps this order also coincides with the list of their internal variable names within the EXE itself, although I have not checked that. For the units on buildlists, and probably buildings in requirements section too, the value is their number on the internal list (e.g. Carryall is 01, 'Thopter is 02 etc.)

[Nyerguds]

Ehh... can't you have multiple structures as prerequisites? In that case I can't see how these values would all be squeezed into 2 bytes...

I don't think I'm going to start with reading .eng files too though. I'll just hardcode those strings -_-

Anyway, the very first really working feature: OWNERSHIP!



Due to the control system behind it, the changes are actually made from the moment you press the spacebar. You see the name go from uppercase to lowercase in the list right away. ENTER or ESC closes the list.
Since the controls are 100% the same as those behind the other lists, this list will automatically scroll too when it'll be used to display a list of units, structures, commands etc that goes beyond 6 items.

The new version is uploaded as d2editor_v1.00b.zip
Enjoy your new modding tool, lol.

By the way, do you have ANY idea where the building graphics are in Dune II? I was trying to figure out what those entries you indicated as 'related to building graphics' actually did, but I can't find any SHP files for them.

[Nyerguds]

New feature implemented: Movement type.

Next is the unit factory construction options, for which I can just use an adapted version of the same function that draws the units in the second box

[Nyerguds]

Fully Implemented:
-Integer types
-unit factory construction options
-Sidebar commands

On that topic, the "Die" command is hilarious


Another feature that's fully implemented now: changing references

This is very handy for changing the WSA pictures, but be VERY careful when editing name references. If you screw them up, both the editor and the game will malfunction.




This version is practically complete... just a few more types to be added, but now I've enabled integer editing you can do those manually too. There's still a bug with negative numbers though.

http://nyerguds.arsaneus-design.com/dune/dune2edit/d2editor_v1.00c.zip

Enjoy

[MrFlibble]

Ehh... can't you have multiple structures as prerequisites? In that case I can't see how these values would all be squeezed into 2 bytes...

Whoops, sorry, I didn't think of that. Honestly, I don't know, but the 1.0 editor surely contains this info.

By the way, do you have ANY idea where the building graphics are in Dune II? I was trying to figure out what those entries you indicated as 'related to building graphics' actually did, but I can't find any SHP files for them.

They're not in SHPs, but in a file called ICON.ICN (inside DUNE.PAK), which contains all tile images (Dune II buildings are composed of tiles rather than single graphics). I don't know whether there's a program that can read/edit ICON.ICN, but the "data unpacker" that is supplied with the Dune Legacy v0.91-0.92 can read this file and convert the data into BMP images (only to be used with DL though).

[Nyerguds]

Well I just wanted to see if there was any relation between the numbers in the exe and the frame order like there is with the icon ID number...

Anyway, besides adding the last data types, all that really needs to be done now is the negative numbers thing. I hope you find this editor useful

[edit]

Negative numbers fixed
Man, thank goodness I remember my classes of computer architecture. This stuff is gold.

As for the structures lists... I know, I just couldn't be bothered doing tons of byte comparisons again
Then again, if they're bit switches, I can be pretty sure the order in the old 1.0 editor is probably correct. It was correct for the commands, anyway

[edit again]

Wow... for the weapons type I could just use EXACTLY the same procedure used to select factory construction options
Yes, this means you get all REAL units in the list too. No, from what I've tested here they don't work as projectiles, but I'm giving the full list anyway
(making a rocket launcher spit out attacking 'thopters didn't work )

Oh, also, fixed a minor bug that prevented the last unit on the list to appear. Though I doubt anyone could use a frigate as either a projectile or buildable unit

[Nyerguds]

All right, I figured it all out

The main problem I had with the prerequisite structures was that they were wrong in your text file; they start one byte earlier and are theoretically 4 bytes long. Only 3 of those are used though.

Unless anyone finds some serious bugs, or info on the unknown values, this is the final release... so, enjoy

d2editor_v1.01.zip


The only things I might still add are a special key to edit any value as bare bytes, an ego-trip screen (AKA 'About' screen) and perhaps compatibility with Dune II v1.00.

[edit]

Added compatibility with version 1.00

d2editor_v1.02.zip


[edit again]

Agh, dammit, I still need to implement a way to select the 'is repair facility' and 'units can enter' booleans 

What's with that repair facility option anyway? It's enabled by putting it on 80h... and I can't turn any other structure into a repair facility with it... all it can do is disable it on the actual repair facility.
That's also very odd concerning enabling the entering of units. It doesn't work on the repair facility when you disable the 80h value

[Nyerguds]

v1.3:
-Hex editing implemented; press [F3] to edit a value as raw bytes
-Type-sensitive help-function implemented; press F1 to get data-specific help.
-Ego-trip screen implemented
-widened third frame to make it touch the right-hand edge of the screen.
-Boolean editing implemented, but only used for the true boolean 'units can enter' option.
(On a related note... disabling the repair facility option does prevent units from moving into it, but a Carryall will stil pick up damaged units and drop them in, and when that happens the repair facility will still repair them)

d2editor_v1.03.zip

OK... so, uhm, I've implemented absolutely every idea I ever had for the editor. I guess that's it


[edit]

ACK!

v1.4:
-Fixed a bug that caused only one byte of the construction options to be read and written, which made the whole thing not work when values were changed from FF FF ("None") to something different (For my editor, [FF 05] was the same as [00 05]. The game disagreed with that )
-Fixed a bug that caused some messy bits to be written when changing prerequisites lists.
-Changed single bytes from unsigned to signed (I should just make 2 kinds, but it's all in those unidentified data anyway, so I can't be sure what is meant. There's always [F3] anyway )
-Wrote a little algorythm to do a smart abbreviation on the structures strings for the prerequisites display, which makes sure you can see the difference between all strings right away. (for example, Concrete -> Conc but Concrete4 -> Con4 )
-Drastically optimized layout drawing time. (try having both the old Dune II 1.0 editor program and my program open at the same time.. it's not pretty )





d2editor_v1.04.zip

I hope it's bug free now

[Nyerguds]

v1.5:
-Optimized the construction of layout frames
-Added F2 for binary editing. The principle is exactly the same as the selection lists for other bit switches: Use arrows to select a bit, and press [SPACE] to enable/disable it



d2editor_v1.05.zip

Whew. Any more stuff I can implement? lol.

[Thumper]

Looks really nice that editor. I think you know a lot about hex. Do you also know a way to get 16 bit images from dune 2000? Someone already made an 8 bit image editor, but why no 16 bit? Is 16 bit more difficult?

[Nyerguds]

Uhh... the only thing I know about that is that these 8 bit images all use the same 256-colour palette, while the 16 bit ones are high-colour. But I doubt that the format of the archive they're stored in would be different.

Did the creator of that 8-bit image extractor leave any information behind about the format of the archive or the images?


Really though, there isn't much to 'know' about hex. The hexadecimal system is pure math... the numeral system with base 16. Meaning that you go through 16 values (0-9+A-B-C-D-E-F) before the value that's displayed as "10" (which then means 16 of course).
Binary is just the same with only 2 values, 0 and 1, so binary "10" is 2.


As for images, 8 bits is 1 byte, which is a value of 0 to 255. When you have an 8-bit bitmap, each pixel is represented by 1 byte. To get the actual colour, you have to look up the colour on that index in the colour palette, which obviously can only contain 256 different colours then, since the bitmap can't refer to any more values. For Dune 2000, I know that the colour palette is a separate file.

Normally, "true colour" images are 24 bit, which is 3 bytes, meaning the values aren't references to a palette but the actual Red/Green/Blue colour values themselves, with one byte (0-255) for each colour value. So the actual file is just a big list of RR GG BB RR GG BB RR GG BB etc, one triplet of bytes for each pixel.

16-bit images are "high-colour": they just squeeze the actual colours into the 16 bits, by using 5 bits for two of the colour values, and 6 for the 3rd one (usually green) since there's one bit left anyway. This gives the two 5-bit colours a range of 0 to 31, and the third one a range of 0 to63, which, as you see, is still a LOT less than the 0-255 of 24-bit images.

You can't easily see the colours in the byte that way, since bytes divide thte bits up in blocks of 8 which doesn't correspond to the way the 16 bits are divided by 3 for the colours. In binary, this would look like this:

[RRRRRGGG] [GGGBBBBB]
As you see, the middle colour is split over both bytes.

http://en.wikipedia.org/wiki/Highcolor


Mind you, only pure BMP images actually work like that straight away. Most image formats first use these systems, and are then compressed to take up (a LOT) less space. So just knowing about the colour depth isn't enough to make a program to view them; I would also need to know how they are compressed.


(man man, het zit hier vol Hollanders  )

[MrFlibble]

That's really a great job you did Nyerguds! The only thing, I haven't had any free time to test it yet...

Have you identified at least some of those unknown values in unit/structure stats? Any clue to where specific house data might be located? I wish I knew more about disassembling so I could at least try to locate them myself (and sufficient free time too, of course)

[Nyerguds]

Uh I told you this just reads the basic structure... it has nothing to do with finding these exception values.
I've been working at this for about a week non-stop, so I haven't had time to look into that stuff

And no, I haven't been able to identify any of the unknowns... I did try though but I can't see much that corresponds to the kind of data in C&C.

C&C had huge numbers of switches that could be enabled and disabled, like Stealth, Crewed, Can Be Built, Can crush, Can harvest, etc... Dune doesn't seem to have anything like that since 90% of these things seem to be defined by the movement type. The only one like that that I could find was the "Unit Can Enter" option for buildings, which is pretty useless to enable on other buildings, since there's no way of getting the unit out.

[Thumper]

Thanks for the information, Nyerguds. I think I understand 8-bit and 24-bit, however the 16-bit is more difficult.

I'm not educated in programming or hacking and making a program to read the 16-bit will be way out of my capabilities, but I'm from time to time doing some coding so perhaps one day I will figure it out, haha.

If you would like some more info about the "Dune 2000 Image Converter", you can download it here: http://dune2k.com/Downloads/43 and it looks like the code is in an .pal file.

[Nyerguds]

Yeah, that's indeed the Dune 2000 8-bit colour palette.

Oh wait, Siberian_GRemlin made that? Heh, he's still around you know. You can just message him on PPM. Maybe he'll get around to finishing it if you ask him

[MrFlibble]

Nyerguds, do you know, from your C&C experience, where the information about how different terrain types affect unit movement speed depending on movement type might be located?

[Nyerguds]

No... RA has terrain types in rules.ini, but I've never seen anything in C&C that wold indicate anything like that. In fact, I doubt it. There doesn't seem to be any terrain where units go faster or slower, not even tiberium. And all I ever found about C&C terrain in that aspect were the passability grids.

Not even the boat movement mode is real... the gunboat and Hovercraft can move over land as well, if used incorrectly. Though the Hovercraft does detect beach tiles somehow and (usually, not always) uses them as unload point.

C&C1 terrain structure example: "SH33"

-offset of passable cells list=00502F38
-Unknown001=2
-Y-dimension=3
-X-dimension=3
-Unknown004=0
-String Offset=004F132B (ref to "SH33")
-Unknown006=0Ch
-Unknown007=2Eh
-value of ID byte on map=34h

This tile is made from 3 different kinds of terrain, though. The grass, then the beach and then the water. I do know that you can't build on some of the beach tiles you CAN move on, but I haven't seen anything in the data that might be responsible for causing that. Mind you, I've never even put this data in UGE because there are hundreds of these terrain entries...

btw, if this data would be found in Dune II, it'll be easy to implement in the editor. All lists use the same system, so even the basic units<->structures list can be made into a scrolling one with just one value (list length) changed from 1 to 2 (it starts from 0, you see )

[MrFlibble]

Nyerguds, I have finally tested the program, and identified a few unknowns

First of all, the "Sight" byte was incorrectly defined by the guy who created the old editor. The real bytes that define unit and building sight range is what is currently Unknown 8 in your editor.

The incorrect "sight" value actually has to do something with how the AI builds and controls units. For example, combat tank, siege tank and devastator have the value of 2, while launcher, deviator and sonic tank have the value of 3. In the game, the AI normally builds more combat/siege tanks than launchers. When I changed the launcher value to 2 and combat tank value to 3, the AI built lots (I mean it) of launchers and only one combat tank.

Another value I identified is Unknown 46, which defines what sound is played when the unit attacks. I can't give you exact values for each sound right now, but you'll figure 'em out: infantry and light vehicles have the "machine gun" sound, tanks the "tank gun", sonic tank the "sonic wave"; the launcher and deviator seemingly have none, possibly because the sound is attached to the respective projectiles - I haven't tested that.

[Nyerguds]

ohh, that's awesome!

I'll get working on a sounds list

The launcher thing seems to fit, yes.. when I changed its weapon to sonic it didn't make any sound anymore. You can always just look at the sound values on the projectiles then though

[edit]

Huh, this is odd.. I've found another version of the dune II 1.07 executable. So bizarre... it's actually a different size.
I think it's the 3-language CD version of dune II... I'll implement it right away.

btw I couldn't find the Sonic wave sound anywhere, and oddly enough, it didn't play ingame either. Do you have a version of the game that has it?
[edit]
Nevermind... it's generated
I can only hear it when playing with DOSBox...

[Nyerguds]

lol, these sounds are crazy... it aren't just the .VOC files; it also includes everything the game generates when no sound samples are played. And most sounds that DO play SB samples (.VOC files) sometimes still play their 'low-tech' alternate, too. Like the trumpet when infantry is killed.

So far I got:

(didn't test all before 20 but the ones I tested were all empty)

20 - concrete slab placement

22 - twinkling sound
23 - intro: dramatic chord
24 - Carryall drop off

30 - infantry kill (vscream1.voc) / trumpet
31 - infantry kill (vscream5.voc) / trumpet
32 - infantry kill (vscream2.voc) / trumpet
33 - infantry kill (vscream3.voc) / trumpet
34 - infantry kill (vscream4.voc) / trumpet
35 - infantry crush (squish2.voc)
36 - target confirm beep

38 - intro: click (click.voc)
39 - intro: missile (missile8.voc)
40 - missile in sand (exsand.voc)
41 - tank shot (excannon.voc)
42 - rocket (rocket.voc)
43 - Sonic wave
44 - building crumble

46 - series of ting sounds
47 - cannot place here beep

49 - tank shot (excannon.voc)
50 - tank shot (excannon.voc)
51 - exploding unit sound
52 - money increase sound
53 - money decrease sound
54 - small rocket hit

56 - tank shot (excannon.voc)
57 - tank shot (excannon.voc)
58 - infantry gun (gun.voc)
59 - infantry gun (gunmulti.voc)
60 - "Construction complete" in current mentat voice
61 - pling-plong sound
62 - radar sound (staticp.voc)
63 - Worm eat (wormet3p.voc)
64 - Mini Rocket (misltinp.voc)

[Nyerguds]

Looks like that's all...

v1.6:
-Added decimal editing (F4) for all data types
-Improved internal procedures for reading & writing bytes
-Added sounds list
-Added compatibility with european v1.07 (CD version)

http://nyerguds.arsaneus-design.com/dune/dune2edit/d2editor_v1.06.zip



Oddly enough, the MCV and Harvester use 0 as sound value, where the rocket launchers use -1.
For that reason, I've put -1 as "none" on the list and 0 as "Not applicable".

[MrFlibble]

Very cool Your editor actually helps a lot to get a better understanding of Dune II's functions. That's a huge step forward, so great thanks!

I think I'll do some more testing soon, and if I identify more unknowns, I'll surely tell you

Huh, this is odd.. I've found another version of the dune II 1.07 executable. So bizarre... it's actually a different size.
I think it's the 3-language CD version of dune II... I'll implement it right away.

I've heard about the differences between the European and patched US versions of the game before. How much do they differ? It's important for me if I want to make SD2SE an "add-on" that gets installed on an official version of Dune II. Do the European and US versions have different game file structures (I mean the files inside PAK archives)?

[Nyerguds]

Well, the CD version'exe has 3-language support... but for the rest, it doesn't seem to different. I just added it as "1.07-CD" in my proggie. It's added in exactly the same as the 1.00 version; by giving an alternate start offset for the unit and building lists, and a third value for correcting the name references to get to the string offsets.

On the 3-language version I've seen an odd bug: it doesn't center the text on any of the ingame menus. I wonder if that was just because I messed around with the files, but I kinda doubt it... all I did was dump the 1.07-CD exe in the normal 1.07's files.

As for the pak files, I got no idea... I know there's an extra one for each language, containing the strings for that language, together with one generic Mentat voice set used for all houses.
Also, IIRC the title screen said "the battle for Arrakis" instead of "the building of a dynasty".

btw, I got that european version from Abandonia

Very cool Your editor actually helps a lot to get a better understanding of Dune II's functions. That's a huge step forward, so great thanks!

See, that's exactly why I based it off the C&C Structure Editor (press F12 in the proggie, it's credited )
That program was the first one ever to allow people to edit even unidentified data entries, and it's what really got me started with editing this kind of stuff.

I think I'll do some more testing soon, and if I identify more unknowns, I'll surely tell you

Do note though, I am absolutely not sure if all bytes are divided correctly. Some of the pairs of 1-byte values might actually be single 2-byte values and even some 2-byte values might have to be split and joined with different bytes. In C&C, the program structure made this easy to see, but here it's just raw data, so I can't be sure at all...

[Nema Fakei]

Wow, looks impressive.

[Nyerguds]

Thanks

(I just hope you didn't mistake that screenshot of the C&C Structure Editor for the final version of my editor, lol )

[MrFlibble]

Save for the colourful interface, they're nearly identical in terms of functioning

[Nyerguds]

Indeed they are... though as I said, in C&C it's easier to be sure the data bytes are divided correctly :\

Also, the Structure Editor normally changes the resolution to a length of 50 lines, so you can get way longer lists. But apparently in Windows when I changed it to 25 lines the program adapted seamlessly.

[Nyerguds]

New version 1.7:

-Changed version label for 3-language version exe from "1.07-CD" to "1.07-EU"
-Added the ability to edit a value from the units/structures list (by pressing space/enter/F2/F3/F4) without having to switch to the options list itself.
-Pressing F2-F3-F4 in editing mode also cancels the edit now, so you can simply tap it twice to quickly see the value and close the edit frame again.
-Pressing F11 toggles Full Hexadecimal Mode, in which all data in the last column is displayed as hexadecimal values, and where the default editor (space/enter) is also the Hex editor.
-Supports the demo exe file from ftp.westwood.com/pub/dune2/previews/demos/
(it's hilarious to change all the "Attack" commands to "Die" and see the automated demo kill off each of its units with a single click )
(oh, and this board needs support for FTP links )



http://nyerguds.arsaneus-design.com/dune/dune2edit/d2editor_v1.07.0.zip

[Ultraq]

With all this progress on these Dune 2 modding tools, it must be a good time to be you eh MrFlibble