OD2: Dune 2 engine recreation

[segra]

Hey guys,

been working on this 2-3 months now, but its getting close to time to move on, so figured id post it all
http://od2.sourceforge.net/

heres a video of it in action
http://www.youtube.com/watch?v=nKmwj2oUaE8

yes another dune2 clone. Ported by hand to c++, just to see if i could, its not entirely complete but scenarios are playable
Currently its utilizing an older version of Libeastwood (about a 3month old version), with a few mods of my own so ive added the entire thing into the svn so its compilable by whoever wants to


while the exe-resource manager will support multiple versions of the game, currently the only one with all the exe offsets is the 1.07HS release.


ive added the Win32 release exe of the latest svn to the project files, and havnt tried compiling on anything else... but it should without too much trouble

- no text anywhere, wasnt sure how to use the font class in libeastwood so i just left it alone
- scenarios never end, there is no checking if any flags have been met
- mission number counting is currently wrong, so structures/units appear in the build list earlier than they should (easy fix, just havnt had time)
- no fog of war, which means the AI pretty much kicks in straight away (lots of the AI is based on whether the player has 'revealed' the unit, action ambush is good example)
- mini-map doesn't take the scale into account, meaning you see the entire map on scaled down scenarios
- some hardcoded unit stuff is still missing like deathhand, sonic blast (TODOs exist in all locations of missing code)
- lots of functions are named sub_XXXXX
- frame rotating and some other unit draw flag effects are still unhandled (sonic blast/sandworm)


there is a built in debugger altho its fairly primitive currently and was originally added to debug the emc scripts
most of the commands are useless currently, just never had the time to implement them

in the game window, press CTRL D
cmd line will display 'debugger entered'

to change between scenarios, simply type   scen x   (x is the scenXXX.ini number)
to exit debugger, type x and hit enter


its currently impossible to turn on script debugging for a unit from the debugger, ive always worked straight out of visual studio and just modified the variable for whichever unit i wanted to watch from there


the AI wont rebuild its base, as the building script function 'Explode' is missing (this adds the building to an array, which is checked in buildings loop.. and once the array has an item, the ai begins rebuilding it from the const yard)


immediate build (for human only, theres no unit limit and the AI goes out of control and unlimited credits are enabled by default

[proyvind]

awesome!

I must say that I'm both happy that you made use of libeastwood, while also very surprised about using libeastwood
from three months back, it was in such an utter crappy state back then!

hehe, I sure hope you plan on updating to using latest, it's pretty much kickass compared and can probably solve some of your issues such as the v1.07hs strict requirement etc., and also of course that you'll have interest in working together on the library (it should be in freakishly nicer shape now than last time I enquiried you about it, thus more appealing;).

Also if you'd have interest in teaming up and joining efforts on game engine as well, that would make me ecstactic and something I bet would be of great benefit to both of us (and others;). )

Either way, I'm gonna dig through your codebase now, I'm *extremely* curious about what your code might have to offer and that I can swipe from you, especially any changes for libeastwood.

[proyvind]

oh, also, just to point it out, I'm aware that the doon lunacy codebase suffers from a lot of the same mistakes that libeastwood did, so don't feel intimidated by it if you ever were to consider joining efforts, it's about to receive the same overhaul that libeastwood just had.

[proyvind]

oh, also, could I request for you to rather create another bzr branch on launchpad for libeastwood rather than importing libeastwood to your own svn repository?

It's pretty darn simple and it'll make it a lot easier for both of us to track changes and perform any merges we'd might want to do.

[segra]

to be honest, im not sure if im even going to do anymore work on this or not. kinda over it now so i just put that repository up for anyone who wants the code/history of it

altho i realise the game engine itself is actually pretty close to completion so its tempting, but with the OpenDune team, whats the point now that ive achieved my objective


the resource manager actually needs a clean, i was going to use the new libeastwood except it didnt drop straight in and i didnt want to spend the time cleaning it up. was more interested in getting as much code as possible ported and working

only small changes where made that are really required by od2, others where just to get it working (mostly trouble with passing in/out data... lol, but i see you've cleaned all that up now

what i do remember tho,

IcnFile > getTileIDs/getTileCount where added (used by mapCellAnim class, frameCalculate function)

[proyvind]

to be honest, im not sure if im even going to do anymore work on this or not. kinda over it now so i just put that repository up for anyone who wants the code/history of it

altho i realise the game engine itself is actually pretty close to completion so its tempting, but with the OpenDune team, whats the point now that ive achieved my objective


the resource manager actually needs a clean, i was going to use the new libeastwood except it didnt drop straight in and i didnt want to spend the time cleaning it up. was more interested in getting as much code as possible ported and working

only small changes where made that are really required by od2, others where just to get it working (mostly trouble with passing in/out data... lol, but i see you've cleaned all that up now

what i do remember tho,

IcnFile > getTileIDs/getTileCount where added (used by mapCellAnim class, frameCalculate function)

I've separated the tile map to a separate MapFile class now, so those functions aren't really needed anymore

Your EMC interpreter is anyways something I most certainly have great interest in, I'll pick it up and merge it with libeastwood sometime soon.

I bet much of your map code etc. might come in handy as well, code seems in general clean, most of the code of ours in Doon Lunacy is still based on the fugly remains of the original Dune Legacy code which were everything but clean..

I still have more code of yours to go through, I hope to find more precious treasures... )

[TrueBrain]

Wow, cool, and you made this based on IDA output? That is impressive ... nice job

[MrFlibble]

Great work, segra I hope you'll find time to complete the project.

while the exe-resource manager will support multiple versions of the game, currently the only one with all the exe offsets is the 1.07HS release.

BTW, there's an important thing that the developers of all Dune II remakes that use the original game files should take into account: the VOC.PAK file is different in the US release and the European three-language releases. In the latter, the infantry speech files all have a Z prefix (e.g. ZAFFIRM.VOC), to support speech in other languages. In the US version, there is no prefix (e.g. AFFRIM.VOC). If you use a wrong version of VOC.PAK in the original Dune II, there will be a crash if digital speech is enabled. I also had problems running Dune Legacy 0.95b2 because of this.

[Xaroth]

A truly nice piece of work you made there, Segra.

[segra]

Thanks for the input guys,

most of the code was produced directly from IDA, only things which I decided weren't worth reimplementing (C specifics like the unit/struct arrays replaced with maps, etc). path finding function returns a vector instead of using hardcoded array.. the gui (was planning on having classic/new eventually)

alot of time was spent with IDA open in one half of the screen, visual studio in the other

[raminator]

would it be possible to port this to pocketpc?

[Minniatian]

Hello Segra!

I would like to announce you that I can help you with OD2 project with my Dune 2 material which has been completed through last years. This material consists of many de-compiled fragments and routines of original Dune 2 code as well as description of many things related to Dune 2. Are you interested in it? If so I might publish here informations that I think should be useful in OD2 project. Maybe it could be useful in OpenDune project aswell.

The routine which I published already is the seed generator translated fully to C (with original random function). It shows that original Dune 2 was written in C or similiar language and proves that some functions can be faithfully converted to C with some effort.

I have a (far from complete though) list of functions divided into sections that appears in Dune 2 with significant names, arguments and value returned. I think it will be useful here but I warn that I didn't touch it through last few years, but I can return to it and try to complete it, but it still requires much work. I have also the description of main loop of the game.

Can you tell what informations do you require by now so I can look if they appear in my material and share these informations? This might be good start of our cooperation. I will also try to read through the forums to find what informations are still needed. OK, so it's all for now. I'm glad that I can help and my material can find a use.

Cheers.

[MrFlibble]

Hey Minniatian, I think the developers of OpenDune might take interest in your data as well You should upload your database somewhere anyway, since many people here will probably want to take a look

[Minniatian]

Hey Minniatian, I think the developers of OpenDune might take interest in your data as well You should upload your database somewhere anyway, since many people here will probably want to take a look

OK. I'll gather the material, describe it and upload as soon as possible.

[MrFlibble]

BTW, how is your Amiga "Dune 3" project going?

[Minniatian]

BTW, how is your Amiga "Dune 3" project going?

Well, I stopped working on it by far, because I was busy with my other projects on AmigaOS.

[proyvind]

Eyo!

Haven't touched any of these things in a few months, but since OD2 is written in
C++ and using an older version of my library, I've been thinking of looking into
it and what use I might have of it's code and all ever since it were first announced.

I figure that porting the thing to using latest libeastwood of mine and to a more gnu'ish
dialect of C++ shouldn't be that hard, so I've started doing this now and for anyone
who'd find it of any interest, can find my progress here:
https://code.launchpad.net/~proyvind/doonlunacy/od2

Not much yet, I only started a half hour before I went to job this morning, but eventually
I guess it shouldn't be that big of a job to get it just working for me (with even font support
added when I get around to it! . From there I'm thinking of trying to merge things with
my Doon Lunacy project. Currently I haven't touched the code for it in like a year (only
focused on libeastwood) and I know of so many mistakes and much ugliness in the
code that should be fixed and even then I still have yet to implement actual gameplay,
so perhaps I'll just base myself on OD2 and merge my code into it along the way as
I progress, or I might do the opposite...  Time will tell, I at least wanna release something
playable sometime soon, so just porting OD2 seems like a less demanding project
on this with far less amount of code and without shitloads of functionality that won't be
of any use anytime soon anyways..

Oh well, I'm babbling, just thought that I'd let you know, if I ever manage to release something
playable, you're probably even likely to find interest in it.

PS: I'd still appreciate if anyone on win32 could nail down the last couple of remaining
issues to get libeastwood and the python bindings / tool working, then I'd release
a new version of it and provide you with another tool I surely think could be of use
to you, especially as it's far more complete and full fledged than most other Dune 2
tools out there. :p

cya!

[proyvind]

I've finished doing the initial work of getting OD2 building and running on Linux now at least..

Let's see how much further I get from there, I've figured out that if anything, I'll merge the code into doon lunacy rather than the other way
around at least..

[MrFlibble]

Nice progress Hopefully it will help make DL gameplay more close to the original

[proyvind]

I figured that I'd just provide some status update report on the progress of mine:
I removed the last remaining piece of code getting the original data structures from DUNE2.EXE the OD2 engine last night, introducing replacements for these in libeastwood with code resembling more the original c++ code in implemented in more generic fashion now.

The more I refactorize and clean up the disassembly code in C, I'm getting a much better understanding of the remaining code as well,
it's quite interesting to see in how similar fashion much of this code was implemented compared to much of our older in dune legacy/doon lunacy.
Not only do I think the code will be easier to fit into my own engine when I'm done decompiling it, but I think it should be quite easy for others with their
own engines to integrate. All the data structures are easily available from the library, with the OD2 implementation using it,
both aiming at (and progressively succeeding in;) being written in a very generic fashion, making it relatively easy
to implement in different languages from as well.
Myself I'm thinking about implementing python classes for my Dune2File class in libeastwood, then reimplement the
c++ code decompiled from Dune 2 in python using them for use in doon lunacy, eventually separating libeastwood from the engine completely (ie. no linking
or use of it's c++ API), making use of python interfaces that'll be easier to figure out and play with and the engine less tied into it, making it more generic (buzz
word/fetish of the day;) for supporting other games as well eventually..

So yeah, that's kinda describes some of my thoughts and efforts on the subject, you can follow my progress in more detail in my branches (as you'll see,
they've been quite active lately ):
https://code.launchpad.net/~doonlunacy/libeastwood/trunk
https://code.launchpad.net/~proyvind/doonlunacy/od2

News of my progress might not be that interesting to gamers at this moment, I haven't really added any new functionality
visible for people only playing the game, nor made any new discoveries to share on these things interesting to most either
(the OpenDune guys does a lot better job on this;), but at least people will notice that my project(s) are still alive and making
progress.
Hopefully it will look a bit more interesting to other devels with similar projects of theirs, perhaps even being of some
use to them, and if I'm lucky, getting a bit help, feedback and motivation in return for me to ever reach a milestone others
might care about.. :p
I sure know that making guesses on a lot of these values is a lot of annoying hazzle, involing losalotsa trial and error getting
what you're satisfied with, having somewhat the same behaviour kinda matching Dune 2's..
Ie. creating separate classes for each unit, structure, missile etc. in the game, figuring out frame offsets etc. used
for each of these I sure know feels a "bit" tedious (trying to get the cutscene playback, with timing, looping etc. correct, and still
failing was definitely the worst one for me, which I never even got in an okay state, for the playback itself nor design-wise;p)..

So for all you doing your own Dune 2 engines in C++ at least, I hope to grab some of your attention, what you'll mainly find
here is various components (of your choice which in specific you'd like to use, mostly individually if you'd like a well;) that
you can easily plug into your engine without doing major rewrites. This is the main gain and focus of mine as well, rather than
a complete engine matching ~identical original gameplay (OpenDune has this focus already, succeeding a lot better at it, with
far more progress:o).


Now the remaining component of OD2 that libeastwood's not fully complementing at the moment is EMC stuff, I've merged in
the code of DST in the past, now I need to merge in the appropriate parts of the EMC interpreter into that code, then the major
work left of on this one is the dark voodoo of the EMC scripts+functions that I need to identify and figure out (my biggest fear
so far, hoping to dip into OpenDune efforts getting to much of it first, before me ;p).

Blablablabla, at least some new rambling and Dune 2 related news again in here, haven't been much of that over the last month..

[segra]

Hey proyvind,

Wow, good work on OD2! It is very nice to see all those weeks of porting code havnt gone to waste!

I havn't attempted to download or compile it yet, but ill definitely be taking a look at the SVN from time to time.

Thanks for your hard work!