PSU Clementine for Mac
PSU lives on the Clementine server. Support the people who run it. Support Clementine on Patreon (opens in a new tab)

← Dev blog

From interpreter to lifted, and a clean DMG

Portrait of Masamuene, Harold's character in PSU

By Harold Davis
devlog recompilation packaging

This is a private educational proof of concept for the Mac client of a private PSU server. Nothing public ships without approval from the Clementine staff, and I am not affiliated with SEGA or the Clementine team.

The last post ended with the interpreter drawing the title screen at 8 to 11 frames a second. This one covers the lifted build running the same screen, what broke, a launcher and an audit.

The lifted build runs the game

The lifter turns the game's x86 code into C, which I compile on the Mac that has the install. After I taught discovery to treat code addresses that appear as immediates as function starts, 27,752 of 27,771 functions lift. The 19 left over stay with the interpreter. The earlier 31 were privileged instructions and decode noise in places that do not run, and I have not yet checked that the 19 are the same kind.

That change did more than raise a count. In 300 presents of the title screen, trips back into the interpreter fell from 7,825 to 216, and the steps they took fell from 258,429 to 5,816.

The check I trust most is the golden trace. The lifted run and the interpreter make the same 685 calls into Windows up to WinMain, and the same 103,206 Direct3D 9 calls over 300 presents, in the same order. Three lines differ in the raw diff, and all three are the clock.

Timing, for 300 presents: 29.7 s interpreted and 14.6 s lifted. CPU time was 23.9 s against 8.1 s. The first 25 presents went from 13.8 s to 3.9 s.

The title screen cannot show how fast lifted code is, though. The game paces itself at about 25 fps, with Sleep(16), a timer-driven limiter and the display's vsync all adding up to roughly 39 ms a frame. I do not uncap it, because the Clementine rules say not to. A sample of the lifted process showed the guest threads waiting in sleeps and event waits for over 99% of the samples. The compiled code is a fraction of a percent. The interpreter only looks slow in the loading stretches, where it drops to 11 fps. Once loading is over, it also sits at the game's pace.

Two bugs the real game found

My test suites were green. The differential run compared 168,765 function runs between the interpreter and the lifted library from random starting states, with 0 mismatches. That was the last full run, on the library from before the seeds and the checks, and I have not repeated it on the current one. Then the game ran, and two things went wrong.

Compiled code cannot be pre-empted. The game depends on time slicing. Threads spin on memory that another thread will set. The interpreter hands over its lock every 1024 instructions, but compiled code only yields at calls into Windows, and a spin loop makes none. My first run sat in a loader thread for two minutes at full CPU while the main thread waited. The fix is a check at every function entry and loop head: a volatile flag, set by a host thread every millisecond, that sends the thread into the scheduler.

I estimate the checks cost 2 to 3% of CPU time in the loading stretch. It is an estimate, and I can say why. I counted 935 M executed checks over the first 25 presents, and measured 0.08 to 0.1 ns per check on real loops from the exe. I cannot run the game without checks to compare, because it hangs in its own polling loops.

A function start one byte into a mov. After three presents the game's loader failed and the game shut itself down, every time. I bisected the library by lifting only the first N functions and leaving the rest to the interpreter. It came down to a 36-byte function at 0x770030. Discovery had also started a "function" at 0x770053, one byte into a mov where the second byte decodes as junk. The jne after it ended up owned by that bogus start, so the real function handed off to the interpreter at that branch, and the cmp before it was dropped as dead. The branch ran with no flags. The fix is to inline a jump target into the caller when its owner does not really contain it. The random-state test could not see this, because it started every function from random registers and never ran the code in order.

The launcher

psu-launcher prepare takes a player's install to a running lifted client. It checks the install without writing to it, copies it with clonefile into Application Support, and lifts the exe from the copy. The player's saves are kept across re-clones. On the real exe:

A test makes the install read-only, runs the whole flow, and checks that every file is byte for byte unchanged.

The audit

The hard rule is that the DMG holds only our tools and runtime. psu-audit fails the build if it finds anything that could come from the game. It looks at file extensions and magic numbers, known hashes of the game's executables, files identical to ones in the install, markers of generated C, and a handful of audited words. It also fingerprints the real exe: any 64-byte window of its sections turning up anywhere in any file is a hit, found with a rolling hash.

I tested that last rule by planting 4 KiB slices of the real exe in copies of one of our own binaries. The audit caught them. The packaging script audits the staging folder and then the DMG itself, and deletes the DMG if either fails. The one I built is about 5 MB (5.2 MB), with 14 files, and the audit passes.

What I have not verified