Format80 decompression

[ArrakisResearchCo]

Ok. I've finished the compression function and run a test.

I've replaced all the colors in the plain sand tile with some red colors effectively making it red sand instead. See the attached screenshot - it's the HitSquad v1.07 version I've tried tile patching.

[Nyerguds]

nice :)

Now try saving some other image in there ^_^

Oh, do you have any idea what determines the radar colour? I can't help but notice it didn't turn red...

[ArrakisResearchCo]

I've been pretty busy today so I havnt really had the time to go on but before I left this morning I started working on the pixel editor. When I have an editor it will be easy to change the tiles.

"radar colour" ? You mean the minimap at the bottom right part of the screen ? I have not looked into that part because I have not yet stumbled across anything that compares pixel values to tile numbers. My guess is that DUNE2.EXE internally somewhere handles tile-to-color conversion to create the minimap.

[Nyerguds]

Hm, that's possible. If I find a tiles list I may find it. And since I've already messed around with stuff that turned out to be tile-related (the changing of tiles when spice is harvested) I bet I can find it again :)

[ArrakisResearchCo]

If you should stumble over the tile/color relations please let us know. If you should stumble over any information about spice value I'd really appreciate getting a few hints in that direction. With your permission Nyerguds I'd like to post a page containing your your findings and information found in your source code for your editor.

I'd also like to wish everyone a merry Christmas :)

[Nyerguds]

Information I release is free. Do whatever you want with it. There's a reason I released the source of my editor ;)

And a merry christmas to you too :)

[MrFlibble]

So, any news on the project's progress yet?

[ArrakisResearchCo]

I've had a lot of things on my hands since Christmas along with a windows driver crash when I put in my new ATI card so I havnt had that much time for my projects. I'm having a few design issues with the tile editor - I'm hoping that my calendar will allow me to get back on track during the next week. At least for the tile editor.

[ArrakisResearchCo]

Tile Editor update: I've resolved the UI design issues I had and I'm back on track.

Currently D2TE (Dune 2 Tile Editor) is able to load ICON.ICN / ICON.MAP, display all tiles, display all tile references found in ICON.MAP (each structures seperate frames - tiles used for each frame are highlighted) and you can drag-n-drop tiles to a selected frame (thereby editing which tiles are displayed in the frame).

The contents of IBM.PAL is hardwired into D2TE and palette selection is also up and running.

I still need to implement the pixel editor (pen / color-replacer / color-picking). Color count restriction is already built but not yet implemented since the pixel editor isnt running yet.

A load/save dialog also needs to be constructed. Currently I have a few hardwired load menus for test purposes while I'm working on the editor. These will be removed and replaced with proper dialogs.

Attached is a screenshot of the editor in 800x600.

[ArrakisResearchCo]

First test runs of the changed made by the Tile Editor has resulted in :

(Testing on v1.07-HS)

Green "Fog-of-war" .. although Dune2 internally seems to ignore the completely black Fog-of-war tile and simply just paints everything black where it's supposed to be. Border of the Fog-of-war is green as it's supposed to be.

Windtraps where the structure itself is colored like the Starport Facility. Sandish / skin colored.

Full animation returned for the Repair Facility. In v1.07-* the middle sequences of open/close animation are missing.

Full animation returned for the Hi-Tech Factory. In v1.07-* the middle sequences of open/close animation are missing.

Concrete slabs now have house color.

The tiny red lights on the Palace and Spice Refinery have been changed to yellow instead of red.

The Tile Editor currently prevents saving if one or more tiles have too many colors. Limit is 16 colors pr. tile. Tile references in the TileMap (ICON.MAP) can be changed by simple drag-n-drop. Looks funny when some of the animation frames have been changed :)

The pixel editor and TileMap editor are functional now. The last thing I need to add is the ability to add/remove tiles and add/remove frames in the TileMap.

I'm not entirely sure about this but I'm guessing that those entries in the TileMap list that contain landscape parts, wall parts, turrets and so on should be kept locked in regards to the number of frames they have. So I'll leave those locked for now.

[Nyerguds]

Amazing progress man :)

Yeah... the main purpose of this was to edit buildings and to change their foundation dimensions. Can you implement that too, when adding functionality to add and remove frames from the TileMap?

One request... could you change the colour picker so we actually see and manage the 16-colour palette and are able to manually replace colours on it? Of course, this should automatically be sorted again after replacing a colour.

You could add it as interface option... "manually manage 16-colour palette"

[ArrakisResearchCo]

Thanks Nyerguds :)

I'll see what I can do about the foundation dimensions. We would have to use your editor to change the foundation dimensions in the .EXE though.

I chose to hardwire the IBM.PAL in the editor so I would only have to load the two ICON.* files since they are the ones changed by the editor. Changing the palettes in the game shouldnt be a problem.

For my web design purposes I write a palette editor a few years back that allows easy palette building. See the attached screenshot of "Blend My Colors". I could add this interface to the Tile Editor and load all the palettes from the game into this palette editor. How would that be ?

I'm not entirely sure what you have in mind about "automatic sorting when replacing a color" - could you explain please ?

I've already added my 8-bit bitmap class to the Tile Editor for export purposes and I'm also thinking about adding "grey-scale" export options using the GRAYRMAP.TBL swap table from the game but if palettes are changed this swap table should also be changed.

[Nyerguds]

this:

Each of these 16 byte blocks is sorted lowest palette index first highest palette index last. If less than 16 colors are used the rest of the 16 byte block is filled by zeros. Note that if palette index zero (the first black color) is in the tile a leading zero is placed in the block otherwise the first (and lowest palette index) is the first byte.

Since these blocks are all sorted there's a chance that the same color contents of a tile may be repeated and if repeated already stored blocks are re-referenced.

I'd like to have that internal colour palette to work with so it's kinda logial to auto-sort it to the 256-colour palette indexes order right away when replacing a colour on it.

[MrFlibble]

Wow. Very impressive :) Is it downloadable yet?

[ArrakisResearchCo]

Nyerguds:

The reordering of the data blocks when "compressing" the tiles into the ICON.ICN file is based on each pixel entry in each tile. When ever a tile has to be compressed again the code will read all 256 pixel value entries in the tile and count up the number of colors found in the tile. Each color reference will then be stored in a seperate array that will be reordered lowest color to highest color. The actual compression is made up of zero to 15 indexes in this new array. I'll supply the source code for the Tile Editor when it's finished - should be fairly easy to read the VB6 code used. As long as you respect the 16 color limit per tile this new array can be created and re-used when compressing the tiles. It's all in the code and I've commented a large portion of the class that does all the work so it should be fairly easy to read and understand :)

MrFlibble:

I have not yet posted a download for the Tile Editor. I'd like to finish the last few things in the editor before releasing it but I'll post info when I've released it. And not to worry: I'll release the editor and the source code for it before I finish writing the new site for dune2.dk :)

[Nyerguds]

I see. I just though it'd be more logical to restrict it right away by only giving the user a 16-colour palette.

See, the problem I have with that is that if you choose a colour there's no easy way of knowing if using that will add a new colour to the palette or if it's already used.

So in that case, when working on one tile can you please show some sort of markers on the 256-colour palette to indicate which colours are already used, together with some sort of "colors used in this tile" indication?

[ArrakisResearchCo]

I've tried keeping the interface and workspace as simple to use as possible. The color picker allows you to pick any of the 256 colors as you see fit .. but: if you place the 17th color on a tile the entire tile will be shown in red colors in the tile-view area (top-left of the screen) thereby showing the this particular tile contains too many colors.

I've put in three tools: pen, color-replacer and color-pick. The last of these tools currently has the wrong tool icon: there's a magnifying glass displayed - I'm working on putting in a color-pick icon instead (which will be in the final release). When using the third tool you can pick a color from the tile (doesnt matter if it's in single tile mode or tilemap mode). When using the third tool to pick a color from the tile, the selection will be activated in the palette / color picker thereby showing which color is currently selected. Note that all three tools have double functions: left-click to use color on the left mouse button or right-click to use color on the right button. The tools default to having palette index zero (the transparent black) on the right mouse button while the left mouse button defaults to palette index 255 which is a white color. If you use the color-pick tool to select a color from the tile you can press either of the two mouse buttons to lock this color to that button. Note that only the color used on the left mouse button will be "highlighted" in the palette.

When I started designing this tool I didnt put much thought to keeping the user informed about which colors are used in each tile. As long as the editor internally keeps track of this limit and somehow warns about and restriction violations it should be enough. It doesnt really matter which 16 colors are used for each tile as long as the total number of colors used in the tile doesnt exceed 16. I've solved this by re-drawing the tile in all red colors in the TileView area thereby warning that this specific tile contains too many colors. Before attempting to save the editor runs through all tiles and checks that the total color count of each tile doesnt exceed 16. If just one tile has more than 16 colors the editor prevents you from saving and warns that X number of tiles have too many colors.

I have been thinking about planting some kind of indicator on the zoomed view along with the red-coloring of faulty tiles but I havnt done anything about this yet.

When I release the editor I'll put in a readme.txt file that describes the functions in the editor and what to look for.

[ArrakisResearchCo]

A little experimenting later ... I've added a "huge" visual warning to tiles containing too many colors. I've attached a sample screenshot of the editor.

The first tile displayed has 17 colors. Notice that the tile has been painted red in the TileView area (top-left of the window) and I've added a red/orange square thingie over each zoomed pixel of the tile. The palette shows one of the bluish colors with a small black box around it - showing that this is the color selected for the left mouse button. The view changes from standard zoomed view to this "error-view" the second the 17th color is added to the tile. Screenshot also shows that the actual pixel colors are behind the red warning squares on the tile. Once the 17th color is replaced on the tile the zoomed view returns to normal - and the tile will be repainted normally in the TileView area aswell.

I'll describe the limits and restrictions in the documentation for the editor when I release it but please let me know if this is enough or if I should change something or make it different.

I've decided to pre-release a beta version of the editor. NOTE! please that I wont be supporting this version in any way because the final editor is to be released in the near future.

The editor requires that you load from DUNE.PAK and will only save to DUNE.PAK again. Make a backup of your DUNE.PAK before attempting to edit the tiles ! "Save to" saves to a different DUNE.PAK file if needed. "Save" will save to the same DUNE.PAK as you loaded. The "TileMap" menu has not yet been implemented so no entries in this menu work yet. You can add/remove tiles. The editor requires VB6 runtimes SP6. The magnifying glass tool is NOT a magnifier - it's a colorpicker - use it to pick a color in a tile. When selecting TileMaps you can drag-n-drop tiles from the TileView onto the zoomed TileMap frame. A green border will be shown around the tile in the frame where you can drop the new tile.

Get the editor here (216Kb) :

http://www.dune2.dk/D2TE(Beta).exe

This is the program file - not an archive and not an install package. Please play with the editor and let me know of anything that should be changed or if it crashes for any reason. If it crashes please note what you did before it crashed and please try to make it crash the same place again.

[edit: may 4th '10] Changed the URL to the correct new site

[Nyerguds]

If just one tile has more than 16 colors the editor prevents you from saving and warns that X number of tiles have too many colors.

And how do you FIX it if you don't know which colours ARE used on the image? I mean the way to fix this is normally to replace the "17th colour" by one of the 16 that are allowed and already on it, right?

Unless your colour picker tool also works on the image, and not just on the palette, there really is no easy way to find those colours... unless you display the 16-colour palette too.

[ArrakisResearchCo]

Well .. I'm having a hard time trying to answer your question Nyerguds :)

As you know - the basics of the editor is to allow visual editing of the images that make up the tiles. In case the person drawing a new tile or changing an existing tile ends up having too many colors in the tile that person would have to manually correct the contents of the image to stay inside the 16 color limit. I could build in a hardwired restriction that prevents the user from adding the 17th color but if the user trieds to change a number of shades or colors - say change the overall color of the Starport Facility to something else - it would be nice if the user could temporarily be allowed to have too many colors in the tile as long as the end result keeps the 16 color limit. Auto-correcting for the limit would prevent the user from making the adjustments to the image contents as needed. It would be up to the user to correct any tiles that have too many colors.

The three tools above the palette: pen, replacer and picker - work outside the palette. The third tool, the Picker, only works on the zoomed image (the one in the middle of the window) and can be used to pick up a color from the zoomed image. This will select the picked up color in the palette. What happens inside the palette is specific to the palette itself. The top part of the palette displays the two colors selected: the border is the color locked to the right mouse button while the color in the middle is the color locked to the left mouse button. The pen allows you to draw with these two colors, while the replacer only replaces a pixel if one of the colors match according to which button is pressed.

I think it's very hard trying to describe how the editor works and how the tools can and should be used ... I'll try setting up a page that describes how everything works in the editor ..

[ArrakisResearchCo]

I've setup a page describing the editor a bit ...

http://www.dune2.dk/d2te/

You can get the beta of the editor here :

http://www.dune2.dk/D2TE(Beta).exe

Grab a copy of your DUNE.PAK file and load the copy into the editor using File->Open. The first link hopefully describes the tools enough - if not please let me know :)

[edit: may 4th '10] Changed the URL to the correct new site

[MrFlibble]

I've tried the tool and it woks good so far.

But the IX remap problem seems to be rooted in the fact that three of the light-blue colors used there are in fact part of the remap color section (marked by a purple rectangle in the attached image). If you look at the remapped IX sprite with screwed-up colors, you'll see that each of the Houses has those colors which are parallel to the bluish colors that follow the red (Harkonnen) remap palette. It is especially noticeable with the Sardaukar and mercenaries, since the corresponding colors in their respective palettes are not purple and yellow, but dark brown and gray.

There are seemingly no other instances where the light-blue part of the remap palette is used, so perhaps the best solution to the bug would be to somehow reduce the range of colors used for remap (unless, of course, it does not screw up something elsewhere :))

Another solution would be to edit the House palette parts (in IBM.PAL) that correspond to the light-blue colors, but that might possibly affect other images in the game.

[ArrakisResearchCo]

After I cleaned up the original code and rewrote my own (de)compressor I left out the parts that replace colors in the tiles. The result should be that all tiles should be correctly indexed in regards to the colors used. There are minute differences beteween the versions but primarily different selections of the cyan colors used in the IX Research Center. I've tried loading ICON.ICN from all four versions and all seem to be displayed without the remapping errors we have seen on other screenshots.

Changing the house colors in the palette will definately screw things up in the game. At least if I look at the changes I've tried making with my editor - like when I changed the concrete slab to use some of the Harkonnen colors instead. Result was that all concrete slabs were colored like the house they belonged to and sometimes when a unit died on one of my slabs it changed color to that enemy units house. Looked pretty cool :)

If you want to draw something on a tile that has to be the house color you should use the harkonnen colors to draw it. Notice that my editor shows all house specific parts in the harkonnen colors. The game replaces these pixels with the respective house colors when needed.

[Nyerguds]

Which blue colour(s) exactly cause the remap problem? Is it all of those?

The third tool, the Picker, only works on the zoomed image (the one in the middle of the window) and can be used to pick up a color from the zoomed image.

Ah, okay.. then there's no problem, since your image itself can serve as palette to pick the colours from :)

[ArrakisResearchCo]

Yup :)

As I said: I've tried making the editor as simple to use as possible. The latest beta release has the graphics changed for the third tool.