Dune 2 Overlays

[TrueBrain]

MarkJ asked me to share my knowledge of the Dune2 Overlays. In the last few days I have been reading through them. I have to say, it is rather elegant Sorry if this is the wrong place to post this random technical blabla, but I thought this would be as good place as any other.

Dune2 Overlays is in fact a simple memory manager. Every time a function is called that is not inside the memory, the oldest overlay is unloaded, all overlays are moved (to the right in fact), and the new overlay is loaded on the left. Then the function you requested is called. In more detail:

When your binary is loaded at 0x03F1 (DOSBox Debugger does this in general), the memory range for overlays is between 0x3FF1 and somewhere in the 0x4400. Around 0x3400/0x3500 is a jump table. This is located in the executable just below the VDISK. Look for CD3F (INT 3F). The first entry is the start. The second entry is much more important.

At the second entry (0x34790 in Dune2 1.00 executable) there starts a jump table. For each overlay there exists, there is such a table. I have a few details on them, but for the general idea that is not really important. Sufficient to say that it contains the offset (I guess from the top of the table) to the real overlay code (which is always below), the size of the overlay, and the amount of entry points. Then follows an array of 5 bytes indicating each entry point. They all start with CD3F. Follow are 2 bytes which are seemly gibberish. They turn out to be very important

When you want to call an overlay function, you jump inside this table. Normally, there is an INT 3F. Interrupt 0x3F is a software interrupt which is the Overlay Manager. It handles everything. When called, the first it does is opening Dune2.exe and getting the filepointer for it. Then it jumps into the real manager. When it comes back it closes the file, and returns. Important here that it always returns 2 (!) bytes backwards. So when nothing would change, it would keep on calling INT 3F.

The manager itself looks at the return pointer. Based on that it knows the header of that overlay table. It checks if the overlay is loaded (it shouldn't). Then it starts on the right side of the memory, unloading overlays, moving all other overlays to the right, till it has enough free memory to load the requested overlay on the left. It will always load it to the most right as possible, so pointers are generally > 0x3FF1. It loads the overlay and runs relocation (behind every overlay is a relocation table. It relocates code to find other functions OUTSIDE the overlay. Inside the overlay ALL calls are relative. Overlays are generally moved around often, so it would be impossible to keep relocating that).

Now it looks in the overlay header to find all entry points to the overlay. It rewrites all CD3F there to jumps (EA). The offset of the jumps (entry points) are the 2 bytes after CD3F. The 2 bytes after that are of course the code segment of where the overlay is loaded. Everything is closed up, and jumped back. Because of the -2 in offset, it returns at the begin of the jump. It jumps in the overlay, and it continues happily. When an overlay is unloaded, the above create EA entries are re-rewritten back to CD3F, and the story starts from the top again.

There are a few more tricks and ins/outs to it, but this gives the general idea. One other important thing I like to mention: from time to time a return value is changed to CS:0000, where CS is one of the overlay jump tables. It seems to be doing some cleanup then (maybe a GC?). I don't know. Special code handles that case, and I haven't got the time to look into that

Where which overlay currently is, is stored in the memory, right after each other. It also keeps track how often it loaded an overlay it seems, maybe some kind of LRU. Not sure.

To jump into an other conversation about what segra said: it should be relative easy to reuse empty space to insert your own code. 'All' you have to do is create an entry point in the right overlay header, and always jump there to call it. The Overlay Manager does the rest for you.

Anyway, sorry about the lengthy explanation. Soon I will publish most of this above in plain C code, so it will be easier for everyone to read I am sure there are errors in my story above, but it at least gives the general idea!

[segra]

Thanks for the explanation,

been wondering about this for awhile now

and yeah c code would be much easier lol :)

[TrueBrain]

http://svn.opendune.org/filedetails.php?repname=opendune&path=%2Ftrunk%2Fdecompiled%2Fcs__261F.c

http://svn.opendune.org/filedetails.php?repname=opendune&path=%2Ftrunk%2Fdecompiled%2Fcs__217E.c

;) Ghehe. I know, it is not really C, it is assembly-wrapped-in-C. I am pretty far with cleaning up parts of that code, I will commit that soon :)