From nothing to a first lifted function
The client does not boot yet. I want that said first, because everything below is groundwork. This is a private educational proof of concept for the Mac client of a private PSU server. Nothing from it goes public without approval from the Clementine staff, and I am not affiliated with SEGA or the Clementine team.
The plan is to statically recompile the Windows client into C, build it on the player's own Mac, and run it against a runtime I write. Before any of that works I need a way to run the original code and a way to check my translation against it. This post is how those two things came together.
Finding the functions
The first question was how much code there is. I wrote a discovery pass that walks the executable from its entry point and every address it can prove is a function, following calls and jumps.
It finds about 27,000 functions in about 0.3 seconds. That is 1.34 million instructions, and it accounts for 92% of the code bytes. Along the way it recovered 1,453 jump tables, which are the compiler's way of turning a switch statement into an indirect jump. Those matter because a lifter that cannot resolve them cannot follow the code at all.
The remaining 8% is mostly padding and data that looks like code, plus a few spots I have not chased.
An interpreter first
I did not start with the lifter. I wrote an interpreter for 32-bit x86 first, so I would have something slow but trustworthy to compare against.
It passes 1,053 unit tests (981 when I first counted) and runs about 112 million instructions per second. That is not fast, but it is enough to run the program's startup code.
Floating point took the most care. The game uses x87, MMX and SSE, so I added all three. Then I did something I would recommend to anyone. I cross-checked every instruction against Rosetta 2, which runs real x86 code on this Mac. A small test program feeds random inputs to a snippet, runs it in my interpreter and under Rosetta, and compares the results. It covered roughly 93,000 random inputs.
It found 7 real bugs in my first x87 and SIMD code. A few of them, in plain words:
fxchswapped the top of the stack with itself. The disassembler library reports the instruction with two operands and I read the wrong one. The compare instructions had the same slip.- With the precision set to 24 bits, I rounded results through a 32-bit float. That also clamped the exponent range, which real hardware does not do.
1e30 * 1e30is a finite number there. fpremcomputes a remainder, and when the two numbers are very far apart it only reduces part of the way and sets a flag so the program loops. Mine did not follow the hardware's rule for how far each pass reduces, and the game's own loop depends on that rule to finish.
I would not have found any of these by reading my own code. The integer side had no mismatches that mattered.
Running the startup code
With the interpreter in place I could map the executable into memory, fill in its imports and let the C runtime start up. It runs for about 70,000 instructions and touches 66 distinct imports (the Windows functions the program asks for). Then the game's own initialisation fails, because it calls Direct3DCreate9 and I have not wired that up to the graphics layer yet.
That is the furthest the original code has ever run on this machine. It is a clean stop, at a place I expected.
The first lifted function
The lifter reads the same instructions and writes C. The first function it handled is a 28-byte bounded string length. The output looks roughly like this (a made-up sketch of the shape, not a quote):
/* 00401000 mov ecx, [esp+4] */ cpu->ecx = rd32(cpu, cpu->esp + 4);
/* 00401004 xor eax, eax */ cpu->eax = 0;
/* 00401006 cmp eax, [esp+8] */ flags_sub32(cpu, cpu->eax, rd32(cpu, cpu->esp + 8));
/* 0040100a jae 0040101a */ if (cond_ae(cpu)) goto L_0040101a;
It is ugly on purpose. Every instruction becomes a line, with the original address in a comment, and the flags are computed explicitly. Cleaning that up can come later. Correct comes first.
To check it, I compile the C into a library, then run the original bytes in the interpreter and the lifted version on the same random inputs. For the string length, they agreed on 1,000 of 1,000 inputs. Seven more small integer functions matched, ten functions that use x87 matched, and one function that makes a call matched too.
How much lifts today
The lifter already handles 19,187 of the 27,158 functions it was given, which is 70.6%. The biggest blocker is indirect calls, 6,355 of the failures, where the target is only known at run time. Jump tables and the less common x87 operations make up most of the rest.
What is next
Indirect calls are the next job, since they block the most code. After that I need the Win32 pieces the startup code reaches (threads, the registry, a real allocator), and the Direct3D bridge so that Direct3DCreate9 has something to call. Input, sound and networking have not been started.
I will write the next post when something on screen is the lifted code's doing.