Decompiling

[XTF]

Two games were 'decompiled' with an emulator / self-written decompiler and then manually ported to C++ (AFAIK):
http://www.opendune.org/
http://www.openttd.org/

I've been thinking, wouldn't this be possible with Red Alert 2, Yuri's Revenge, Renegade and others too?

There's no need to do the entire game in one go, it could be done one function at a time. Even having assembly as output that could be modified and recompiled easily would be nice.

[DCoder]

OpenDune claims there are 1229 functions in the original game.
IDA tells me there are 18355 functions in YR 1.001. This includes piles of template specializations, thunks, CRT functions and other crap, but still, it's 15x the size.
The largest function in the game is BuildingTypeClass::LoadFromINI, at 19475 bytes.

How do you imagine this recompiled stuff should be connected to the game? Replace the first instruction of the original function with a JMP to the new one? Syringe can do that.

Assembly output should be fairly simple with IDA - last weekend I started writing (for other purposes) an IDA plugin to spit out the asm instructions from a function without any decorations IDA normally places around them, wrapping that in __asm {} would be all that's needed for a start. But I really doubt that that approach would get you comprehensible code faster than Hex Rays does.

[XTF]

How do you imagine this recompiled stuff should be connected to the game? Replace the first instruction of the original function with a JMP to the new one? Syringe can do that.

For example

Assembly output should be fairly simple with IDA - last weekend I started writing (for other purposes) an IDA plugin to spit out the asm instructions from a function without any decorations IDA normally places around them, wrapping that in __asm {} would be all that's needed for a start. But I really doubt that that approach would get you comprehensible code faster than Hex Rays does.

What about generating function prototypes and pointers (so the originals can be called) and variable types and definitions?

BTW, does IDA allow you to put the 'DB' into version control and to merge two DBs?

[DCoder]

So, once you have function prototypes and variable types... how is your work different from YR++/Ares?

Edit: Re IDA version control, no, as far as I know it doesn't have any functionality like that.

[XTF]

Does Ares have those for all functions and vars? For RA2?
What about other games?
Once you've got them, you could decompile one function at a time (or more) to a recompilable format.

The end goal is to 'port' as much code to an easily editable format as possible.

[DCoder]

YR++ has a lot of declarations for YR 1.001 I wouldn't mind having more of them, and for RA2/TS as well. Heck, I might build a starting point from YR++'s files over the weekend and toss it on github.

I still think that a lot of code in YR/TS is already quite readable when processed by Hex-Rays, but if you want to make your own conversion, good luck. The one part of the game that makes little sense to me in Hex-Rays is the audio code, it was probably written by a serious old school C wizard with a mountain of "I know what I'm doing, shut up" casts.

Have you considered how long such a process would take? 18000 functions... Best of luck, seriously.

[XTF]

YR++ has a lot of declarations for YR 1.001 I wouldn't mind having more of them, and for RA2/TS as well. Heck, I might build a starting point from YR++'s files over the weekend and toss it on github.

It'd also be nice to not have to keep two copies of that data (one C++, one IDA).

I still think that a lot of code in YR/TS is already quite readable when processed by Hex-Rays, but if you want to make your own conversion, good luck. The one part of the game that makes little sense to me in Hex-Rays is the audio code, it was probably written by a serious old school C wizard with a mountain of "I know what I'm doing, shut up" casts.

Have you considered how long such a process would take? 18000 functions... Best of luck, seriously.

I'm not sure yet whether I'm going to work on this.
But again, the goal would be to have easily editable output, not just to have comprehensible output.

[Graion Dilach]

Yeah, one IDA wouldn't hurt to be shared at least in a Dropbox within the Ares developers, for people who can understand assembly but can't disassemble. Like me, who goes nuts right now due to that crash in AttachEffects and has no idea to figure out that reason.

Rest I think... I dunno. I doubt it would worth it. I mean, I know you favor RA2 because YR is less balanced, but on the other hand, YR is a more advanced engine in many aspects (starting point selection, the YR-only logics) which from coding point, voids the reason.

If we would speak about an entire rewrite, like OpenTTD did, then the new engine should be flexible, (worths to mention IMO that when OTTD came, TTDPatch was the etalon (it's much like RockPatch) and OTTD was written to took the TTDP behaviour) but in this case... I don't see what would be the gain of the backporting.

Especially that most of your pro players would even burn you for touching their game.

[DCoder]

IDA can parse C headers and import definitions from them. It is theoretically possible to write a tool to convert the C++ headers YR++ uses into C headers. Or vice versa, IDA plugins that turn its structs into C ones (though converting those to C++ would be trickier).

As for easily editable, I don't see much problem there - write a single hook at the function's starting point, replicate the entire function body in your own code, edit as desired, done. The catch is of course that you would need to understand not just that one function, but all the ways it interacts with the rest (data structures being changed, map/cell flags being set, etc.)

[XTF]

If we would speak about an entire rewrite, like OpenTTD did, then the new engine should be flexible,

OpenTTD is not a rewrite, it's a 'port' of the existing assembly to C/C++.
Only after that parts were rewritten.

[AlexB]

Porting YR like that would stop actual progress being made in other areas for a long time, and possibly introduce subtle bugs. Usually I'd say "If it ain't broken, don't fix it", but as this game is made by Westwood, that wouldn't make sense.

[XTF]

Porting YR like that would stop actual progress being made in other areas for a long time,

Not really, as you don't have to do it all at once. Besides, if YR isn't broken, what's Ares doing?

[Graion Dilach]

Adding in new features. Similarly like OpenTTD. There aren't many new stuff alone, modders have to use them up.

[AlexB]

Not really, as you don't have to do it all at once. Besides, if YR isn't broken, what's Ares doing?

Recoding small pieces at a time would still sum up, maybe even more than doing it all at once. The time spend there cannot be used to actually fix bugs or code new features, and there's still a chance of accidently changing behavior and thus introducing small bugs while transcribing and refactoring.

Ares already replaces many functions, but as far as I know not to make it more perfect, but because it would be tedious to add hooks every five instructions. Every full rewrite is done to serve a concrete purpose.

Of course YR is bugged, that's why I said the premise in that commonplace isn't true, thus it doesn't apply.

[XTF]

Then we come back to the question: what goals would you like to achieve?
For major updates it's (IMO) quite handy to have all relevant code (in C++) available.