Format80 decompression

[Nyerguds]

A more reasonable request... can you ALWAYS show a 3x3 tiles grid? That will make it easier to change the foundation dimensions and will make sure the image doesn't suddenly blow up to gigantic proportions when editing a single frame.

You can just grey out & disable the unused tiles bhen the dimension of the set that's being edited is smaller than 3x3. But show them anyway.

[ArrakisResearchCo]

The grid will "auto-show" regardless of foundation dimensions but will fit the foundation dimensions of the specific TileMap reference (one of the 26). So if one of the TileMap references - say the Repair Facility - is changed from 3x2 to 3x3 the grid will auto-resize to the new dimensions. I do have to copy/shift the previous setup into the new dimensions and then fit the new "empty" tiles with new tile references. My plan is to drop concrete slabs into these "empty" tiles. I have to re-write the TileMap reference setup first though: to read the dimensions from the EXE. This will present a problem: if your editor has been used to change the foundation dimensions the ICON.MAP file will not match and will result in under-/overflow and seriously messed up references. Alternatively I have to add code to patch the EXE from the Tile Editor when changing the foundation size in the editor. I'm still looking into these things :)

[Nyerguds]

It's just that it's handier that it doesn't change size whenever you click on something different -_-

[dvalin]

For Doon Lunacy I've moved all code handling the native data format into

a separate library for easier and better maintenance etc., while also

making it possible for others to benefit from as well.

So you might just have interest in the library which you'll find at

https://code.launchpad.net/~doonlunacy/doonlunacy/libeastwood .:)

The library is written in C++, so I'm not sure of how much use it

is for you guys with Visual Basic. I don't really have any knowledge

of VB nor development on Windows at all, but I guess for someone

with both VB and C++ skills it shouldn't be hard to create some

VB bindings to make it possible to use with VB..

I'm not really sure on what specific use you have in mind for these

things either, but if it's basically just about reading and extracting

game data, then writing tools for such using the library is a very

easy task. I've been thinking about the need for such for a while

since I haven't really seen any other free and portable tools out

there supporting all the formats (only tool coming close is sandtools

which isn't actively maintained and still lacks support for some

of the data formats), so I plan on getting such tools in place

sooner or later. I might also create some python module using

the library as well, since it's a very easy language to learn

and use, while also being available for most platforms this

seems like something that could be of great use to many people,

maybe if I just write the python module someone else could

easily write the actual tools in python using this. :)

The library supports near all of the formats used in Dune 2,

even the format used for text used with cutscenes, mentat etc..

Only formats that I can recall it missing support for is

those used by .c55 & .xmi midi music (support for .adl is in

place, but support for the others is needed as .adl is the

format Dune 2 uses when you have midi only, so it has some

extra instruments being played some places used as sound

effects in place of the missing digital sound effects which

sounds a bit odd when playing the sound effects in addition.

Also interesting note is that when using midi only, midi

intruments are used for sound effects in the game. When

using digital sound effects most sound effects uses the

.voc samples, but there's still some sound effects using

midi instruments then as well, ie. like the Sonic Tank

and also some of the machine gun during the intro seems

to be using what I suspect is the midi instruments for

machine gun or something. :D) and .pcs for pc speaker

which I also would like to support since it gives

some extra retro feeling for some of the pc speaker

nostalgics out there. I'm at least one of those as I

didn't have a sound blaster back then and had to

use pc speaker, watching the intro with pc speaker

under dosbox gives me such nostalgic sensations.:O)

In the future I'm planning on adding support for

other formats as well, ie. support for more of the

C&C data formats. With freecnc being more or less

dead, picking up much of their code and supporting

C&C with the Doon Lunacy engine could be interesting,

just think of the cheezy opportunities that can be

made possible with playing ie. Nod vs Atreides vs

Fremen vs Russia vs GDI vs The Allied. Who do you

think would win in a fight between group of fremen

troopers, Tanya and a GDI commando? :P

For anyone wondering about Doon Lunacy's current

state development kinda stopped at the end of the

summer as school work occupied most of the time in

stead, but as someone expressed interest in

contributing to development I found some motivation

for doing a bit work again the other day, so maybe

2009 will be the year where a playable version

will finally be released. :O)

I'm not really following this forum more than sporadically

every now and then a few times a year, but I might

try follow it closer if anyone has interest in the

library or tools etc. using it, for anyone who would want

to get in direct touch I'm available on #dunelegacy @ irc.freenode.net

and also by mail (you'll find my address on the launchpad

project page).

I hope this might be of use to you. :)

[MrFlibble]

Wow dvalin, you've done a lot of work and research. Keep it up! :)

Which blue colour(s) exactly cause the remap problem? Is it all of those?

I did not test thoroughly, but it looks like all light-blue colors after the red HK set are part of the remap scheme. Perhaps they wanted to make different colors for units, buildings and/or infantry, Brotherhood of Nod (TD) style (I remember it is mentioned in the book that Harkonnen uniforms were blue - maybe that's the case?). Anyhow, if they are all remapable, some substitute should be then found among the remaining colors - editing the palette as I proposed earlier is certainly out of the question, as those colors (of the other Houses) are also used elsewhere.

[ArrakisResearchCo]

Nyerguds: I'll keep your request about the zoom view in mind :)

dvalin: Thanks for the hint. I've already looked at your fine library but it took me a while to figure out which repository tool was needed to download the library. Bazar does the trick.

[MrFlibble]

Hm, I still can't find a good solution for the IX remap bug - there are no other colors quite close to those which are remapable in the palette, so it's either editing it (a prospect I don't like, since it will require either finding some unused colors, if there are any, or replacing some used ones and editing the graphics where they are present) or somehow editing the EXE to exclude the colors in question from the remap list (any ideas about this Nyerguds?)

Otherwise the tool has been very useful, allowing me to fix the broken frame of the gun turret in the v1.0 tileset and to change the first frame to what it looks like in v1.07 (the checkerboard frame which is used when an old savegame is loaded).

[ArrakisResearchCo]

After I fixed my (de)compressor my House of IX didnt have any visible remapping with wrong colors. Maybe there's a structure bug or something or maybe some of the tiles have been encoded wrongly / maybe it has been patched already. I'm not sure why the wrong colors persit.

Anyhow - I hope my editor has been of some use and just let you guys know: I'm still working on a few more tweaks to my editor. I'm working on adding the option to edit foundation size for the structures in the editor and to keep the same zoom view regardless of foundation size. Things are a bit slow on the project at the moment tho because of other things :)

[Nyerguds]

the game remaps too much. That's the bug. It doesn't happen for Harkonnen because the original remap is red, and they're already red, so the remap is skipped for them.

[MrFlibble]

So it's EXE editing to fix that, right? Any ideas on how to find the required data within it?

[Nyerguds]

Nope.. not a clue.

[MrFlibble]

I've kinda fixed the remap bug with gray color for my SD2SE project (the Sardaukar remap was especially ugly):

But it's still not the same...

[Nyerguds]

still, it doesn't look too bad :)

[MrFlibble]

Here's a new, better version:

I believe it's the closest thing to the original colours available in the game palette.

[Nyerguds]

ooh, I like it! Yes, that is better than the grey :)

[MrFlibble]

Definitely. I included this version in both Super Dune II: Classic Edition and in Dune 2 eXtended.

[Nyerguds]

and in the game patch? This is definitely patch material :)

[401Kill]

Just came back to this thread, its been a long time since I had any time to think about such things!

The VB effort expired a while back, and is probably dead for good, since I am now rewriting as a cross-platform app in Mono C# (as good an excuse to learn C# as any I guess :P). Only done the PAK reader so far (the easy bit!), and a quick VOC -> OGG conversion using PACPL - but thats kinda Linux dependant at the moment, need to implement a more universal solution (probably just a VOC -> WAV algorithm for starters). Looks like you guys have come up with a wealth of info on the graphics side of it, will need to read the thread in detail now!

I am particularly interested in dvalin's work on the music, I was hoping to find some info on converting XMID into a (more) usable format.. eg normal MIDI. Any information on that front would be greatly appreciated! I can see he's done a lot of work on the file formats, and I shall try to make what sense I can of the C++ :P

Apologies for reviving an old thread and then wandering off topic, and many thanks for all info! :)

[Nyerguds]

As far as I know, WinAmp can convert XMid to normal midi. You just open its file details and there's a Save button.

[401Kill]

As far as I know, WinAmp can convert XMid to normal midi. You just open its file details and there's a Save button.

Thanks. I'm trying to find a reasonable cross-platform, automated approach - either CLI or a library would be best, since I don't think my format hacking skills are up to decoding it yet lol. Will look into any possible CLI interface to Winamp.

Also, I now have a C# routine which decodes all the WSA files correctly, except 'static.wsa' - is there any known issue with that file? I read somewhere that the WSA format used in 1.07 differs from 1.0, is 'static.wsa' perhaps a leftover in the earlier format?

[Nyerguds]

The radar screen? Huh. Well XCC Mixer opens it just fine.

[MrFlibble]

What version do you have? AFAIK all WSA files in 1.07 were updated to the new format, which is recognized by XCC Mixer.

[401Kill]

What version do you have? AFAIK all WSA files in 1.07 were updated to the new format, which is recognized by XCC Mixer.

I'm on 1.07 European. Fair enough, in that case it's most likely a bug in my code. Just seems strange that all the other WSA files work ???

According to my routine, 'static.wsa' is 64 x 64 pixels, which I would guess is correct for the radar screen, and contains 23 frames. The offsets of which I find to be:

Offsets	int[23]

[0]	102
[1]	2271
[2]	5686
[3]	7770
[4]	11123
[5]	13458
[6]	16002
[7]	17359
[8]	18167
[9]	18212
[10]	18257
[11]	18302
[12]	18361
[13]	18381
[14]	18389
[15]	18397
[16]	18405
[17]	18429
[18]	18437
[19]	18474
[20]	18482
[21]	18506
[22]	20677

Are these the correct values?

[XTF]

IIRC there are files with 16 bits offsets and files with 32 bits offsets. That might be an issue.

[401Kill]

IIRC there are files with 16 bits offsets and files with 32 bits offsets. That might be an issue.

Thanks for pointing that out, double checked and static.wsa uses 32-bit offsets though.

Also checked the first offset against the raw file data, it appears to be correct as far as I can tell.

The problem seems to be that the Format80 decoding routine is using a 4096 byte (64 x 64 px) output buffer based on the image dimensions, however the EOF marker (command 128) is not encountered until the Destination pointer is at 4099 - ie. the decoded image is 4 bytes too long. If I increase the output buffer size, I can marshal the resulting byte array into a bitmap which looks ok, but then the second frame fails to decode.

I've tried tweaking the start offset slightly, but that results in a complete failure to decode the first frame - so I think it's probably correct.