Tprrt's Blog

Introduction

The earlier article on cycle-level GBA optimization covered the software 3D rasterizer, where two "obviously correct" changes measured as regressions once link-time optimization's inlining was counted. Here the same thing happens in the text renderer, with two more wrong turns that have nothing to do with LTO.

Two framer-engine examples with a lot of on-screen text ran far below the 60 Hz target, both idle, nothing moving, no input needed to reproduce: input_tester (76 of a 96-slot text sprite budget active) at roughly 18 FPS, and wordle, a word cloud of rotated labels, at roughly 14.5. phase_probe.py, the gdbstub cycle-bracketing script used in earlier articles, put about 69% of a frame's cost inside one function, gba_text_render_system. Which part of it?

Finding where the cost lives

The function processes a handful of text entities, each with one or more lines, each line made of several hardware sprites (one GBA OAM entry per glyph group). Three plausible places for the cost to hide: the glyph layout math, the screen-space projection that turns a world position into a pixel coordinate, or the OAM attribute writes themselves, once per glyph and so the most frequent of the three.

arm-none-eabi-objdump -dS ruled out the OAM writes: they are inlined integer stores with no library calls. The projection math was different: cx/cy (an entity's anchor in screen space) and py_base/px (where each line starts) were float, and the ARM7TDMI has no FPU, so every subtract, add, multiply, floorf() and float-to-int conversion goes through a software float routine.

Static inspection shows that a call exists, not how often it runs. Two breakpoints measured that: one at the function's entry, one at the first line of the innermost per-sprite loop body. The delta between consecutive hits of the second breakpoint is one sprite's own cost when it stays on the same line as the sprite before it, and the same delta plus a line's or entity's own setup when it doesn't. In a steady frame of input_tester, a same-line sprite-to-sprite step cost about 515 cycles. A step that also crossed into a new line or a new entity cost 7400 to 17000, roughly fifteen times as much. Those transitions are rarer than sprites but frequent enough to dominate a frame. The per-entity and per-line projection math was the cost, not the per-sprite work.

Attempt 1: fixed-point that rounded the wrong way

The engine's software 3D pipeline already uses fix_t, a Q12 fixed-point type, where one hardware multiply (fix_mul) replaces a chain of soft-float calls. Reusing it for text looked like the obvious fix: convert the camera position, ortho extent, scale and entity position to fix_t and project in integer math.

It looked right in every example. Before trusting that, I wrote a host-side sweep test comparing the fixed-point result with the float one across a grid of realistic camera, ortho, scale and position values. 16 of 192 combinations disagreed, every one by exactly one, in the same direction:

mismatch: cam=0.0 ortho=5.0 scale=28.8 pos=0.0 expected=144 actual=143
mismatch: cam=0.0 ortho=5.0 scale=28.8 pos=2.5 expected=72  actual=71

The existing fix_from_float() helper (shared with the 3D rasterizer, which depends on its exact behavior) truncates toward zero. Four conversions chained together — camera, ortho extent, scale, position — each truncate a little short, and on values that land exactly on an integer in float math, the accumulated shortfall floors to one below the right answer. Text at 144.0 pixels would have shifted by a pixel on specific camera positions, with no visible sign.

Attempt 2: the float path that measured slower

Instead of fixing the rounding, I kept everything in float and replaced (int)floorf(x) with a hand-rolled version. objdump showed the floorf() body at 74 instructions, handling NaN, infinity and subnormals that bounded screen-space input never produces. A truncate-then-adjust version is exact for any finite value and looks much cheaper:

static inline int gba_fast_floor(float x)
{
    int i = (int)x;
    return (float)i > x ? i - 1 : i;
}

It is exact, verified like the first attempt, and shorter. cycle_probe.py disagreed: the steady-state average on input_tester went from 727556 to 748028 cycles/frame, a ~2.8% regression. The original (int)floorf(x) is two soft-float calls (floorf, then the float-to-int conversion). The replacement is three (truncate, convert back to float, compare), and each call pays its own call/return/Thumb-interworking overhead. floorf()'s large body is cheaper than one extra call.

What actually worked

Fixed-point, with the rounding fixed: a local gba_fix_from_float_rounded() that rounds to nearest. The shared fix_from_float() stays untouched because the 3D rasterizer depends on its truncation. The sweep test now passes, including the exact-integer cases and the narrowest float margin around an integer boundary.

The second change mattered as much: every frame-constant term (camera position, ortho extent, screen-space scale) is converted to fix_t once per call, not once per entity. Only the entity's own position is converted per entity. fix_mul is one hardware multiply, and flooring a fix_t to a pixel coordinate is an arithmetic right shift, with no library call.

A rotated text entity still needs a float cx/cy for the sprite-rotation math. Computing it from scratch next to the fixed-point version made a rotated entity pay for both, which was slower than before for that case; cycle_probe.py on wordle's word cloud confirmed it. Deriving the float value from the fix_t one (one int-to-float conversion and a multiply by a constant) halved that overhead. A rotated entity still computes both.

Checking it

Cycle counts do not show correctness, and a projection rewrite can be off by one pixel without looking wrong. The check is the flicker article's OAM sampling, made byte-exact: connect to the emulator's GDB stub, let a few frames pass so startup settles, read the 1 KB of hardware sprite memory, and compare it with the same read from the unmodified ROM.

IDENTICAL: 1024 bytes of OAM match exactly

This holds for both input_tester (unrotated text) and wordle (rotated): all 128 sprite entries have the same position, tile, palette bank and affine flags in both ROMs.

The numbers

Steady-state averages, cycle_probe.py against framer_gba_wait_vblank:

input_tester is the representative case: a HUD-style scene with static, unrotated labels. wordle's word cloud is a rotation showcase and the one case where the change gives a little back. Closing that gap would need two code paths, fixed-point when nothing is rotated and float when something is. I did not add them: more branching in a dense function, for a demo scene rather than the 18 FPS HUD case.

What is left

~3.5% is not the fix for an 18 FPS scene. The per-entity layout call and framer_color_bank_acquire() were never bracketed like the projection math, and either may hold more cost. Next step: the same two-breakpoint measurement on them.

Introduction

The article about bullet pools ended with a number: on the Game Boy Advance, a pool of bullets stops being fully visible at 121 of them, and on the Nintendo DS at 126. Both consoles have 128 hardware sprite entries (OAM), and a few of them always go to the player, the target and the text. Past that number, the renderer runs out of entries and has to stop drawing.

Which bullets stop being drawn? The engine's first answer was the last ones in the array, which is wrong for a reason unrelated to performance. The second answer is cheaper and older, and an emulator can check that it works.

The first answer: the same ones, every frame

A sprite batch is drawn in array order, and the loop stops as soon as there is no OAM entry left. With 200 bullets and room for 121, the same 79 bullets, the ones at the end of the pool, are skipped in every frame. They are never drawn, and a bullet that is not drawn still moves and still hits the player. The player dies to something that was never on the screen.

The engine already had one tool for this, and it is the right one for most of the problem: a priority per batch. A batch with a lower priority number is drawn first and so gets its OAM entries first; the batches that matter least lose theirs. On the GBA, two stationary pools of 100 bullets that together overflow OAM behave as expected: the one created second but given priority 0 kept all 100 of its sprites, and the one created first, with priority 200, kept 22.

/* The enemy's bullets decide whether the player lives. */
framer_sprite_batch_set_priority(world, enemy_pool, 0);
/* The player's own shots and the sparks come after. */
framer_sprite_batch_set_priority(world, shot_pool, 200);

Priority decides between batches. It says nothing about what happens inside the one that does not fit, and the pool that loses is still cut off at the same element every frame.

The second answer: start where you stopped

The consoles' own games had this problem first. A Famicom can draw eight sprites on a scanline and sixty-four in all, and the shoot-'em-ups of that era did not pretend otherwise: when a boss made of many segments crossed a line, some of its sprites were missing. Players know the fix as flicker: the missing sprites change from one frame to the next. Every sprite is on screen on some frames, and at 60 frames a second the eye reads a dim sprite rather than a hole. Recca is a well-known example. The hardware did not do this; the games did, by rotating the order they wrote their sprites in.

The same rotation fits a batch in a few lines. Each frame, a batch that has opted in starts drawing at the element where the previous frame stopped, and wraps around to the front when it reaches the end:

int n = framer_sprite_batch_count(batch);
int start = framer_sprite_batch_start(batch);

for (k = 0; k < n; k++) {
    i = start + k < n ? start + k : start + k - n;
    q = framer_sprite_batch_get(batch, i);

    if (s_next_oam_slot >= GBA_OAM_HW_SLOT_COUNT) {
        framer_sprite_batch_resume_at(batch, i);
        return;
    }
    /* ... write the OAM entry for q ... */
}
framer_sprite_batch_resume_at(batch, 0);

When the whole batch fits, the loop finishes, the resume point goes back to zero and nothing rotates. When it does not fit, the element that was refused is the one the next frame starts with. The first frame draws elements 0 to 120 of 200, the second draws 121 to 199 and then 0 to 41, the third starts at 42, and so on. Every element is drawn about 60% of the frames, and which ones are missing keeps moving.

framer_sprite_batch_set_flicker(world, shot_pool, 1);

Checking it

The claim to test is narrow: with flicker on, every bullet is drawn on some frame, and with it off, the same bullets never are. A screenshot cannot show that, since one frame looks like any other, so the check reads the OAM.

The test program is the bullets example with a pool of 200 stationary bullets laid out in a grid of 20 by 10, which never move. A script connects to the emulator's GDB stub, lets the program run for a random fraction of a second, stops it, reads the 1 KB of OAM, and picks out the bullets, which are the 8 by 8 sprites that share the most common tile. It does that dozens of times and counts how many different positions it ever saw.

On the GBA, in mGBA, with room for 121:

The "off" row is noisier than it should be: the per-sample count reached 128 and 122 positions were seen, where 121 is the most that fits. I did not track down the difference. It is small next to 122 against 200, but the sampler is not exact on the GBA.

On the DS, in melonDS 1.1, with room for 126, the program toggles the flag itself every 180 frames, so one launch gives both rows, and the script reads the current flag from a global at each stop:

In both cases the number drawn at any moment is unchanged: flicker changes which sprites are drawn, not how many. The figure shows three GBA samples per setting, taken a second or so apart, with OAM positions mapped back onto the 20 by 10 grid:

Filled dots are bullets present in OAM, hollow ones are bullets that were not drawn. With flicker off, the same 79 are missing in every sample. With it on, the gap moves.

All of this ran on emulators (mGBA, melonDS), none on hardware, and the per-scanline limits were not checked.

What is left

The rotation only deals with the whole-OAM limit, the 128 entries. Both consoles also limit how many sprites can be on one scanline, and that limit can be hit by a wide row of bullets before the 128 total, so it is next. Also planned: a way to report how many sprites a frame used, so a game can see its budget without a debugger, and a priority for single sprite entities and text, which today are always drawn before every batch.

Introduction

The previous article described the static, no-heap Entity Component System framer-engine runs on GBA and NDS: fixed tables of component slots and entities, sized for a machine with no malloc(). With today's component set the table has room for 64 real entities. A caravan-style vertical shoot-'em-up, the engine's first playable milestone, wants several hundred bullets on screen in a dense pattern. The entity table cannot grow to fit that, and the pattern cannot cap itself at 64 shots.

The usual answer is not to make every bullet an entity. The simulation, a flat array of structs advanced by one system, was easy. Drawing that array without opening a second path in the renderer took two attempts. The first shipped, worked, and was wrong in a way no test would catch. The second was smaller, matched a pattern the engine already had, and surfaced three bugs across three rendering backends, none about bullets.

The first answer: a queue that isn't an entity

Hundreds of short-lived things, updated and drawn every frame and never addressed individually, suggest a per-frame queue: a flat array of draw commands (position, sprite, frame), reset each frame, appended to by whatever wants something drawn and consumed once by the renderer. A bullet pool fits exactly, pushing one entry per live bullet after the physics step, and needs nothing from the ECS: no entity, component or query.

That was the problem. Everything else the engine draws (a sprite, a tilemap, a HUD label) is an entity with a component, and every renderer's draw phase is systems iterating queries. A queue fed by application code and drained by the renderer is a second path. The entity-inspection API cannot see it, capability queries do not know it exists, and the debugging tools built on "the ECS is the whole scene" miss it. Bullets appeared in the right place at the right speed, but elements without an entity bypass the engine's core model. It is a different design that happens to render the same pixels.

The second answer: an entity that stands for many

framer-engine already had a component for this class of problem: a Tilemap is one entity whose component points at a grid the owner keeps, and the renderer draws the whole grid from that entity. Nothing in "many drawn things, one external memory, one component pointing at it" is specific to tiles. The rewrite generalizes it as SpriteBatch, and the bullet pool becomes one more caller instead of a bespoke path:

struct framer_batch_sprite {
    int32_t x, y;       /* center, world units, fixed point */
    uint32_t texture;
    uint8_t frame;
    uint16_t size;       /* drawn size, world units, fixed point */
};

struct sprite_batch {
    const void *first;   /* owner's element array */
    const uint16_t *count; /* owner's live-element count */
    uint16_t stride;      /* bytes between elements */
};

A framer_bullet has its own struct framer_batch_sprite as its first field, so the pool's storage array is the SpriteBatch element array, with no copy and no second buffer to keep in sync. framer_bullet_pool_entity_create() gives the pool's one entity a BulletPool component, stepped each frame by a physics system, and a SpriteBatch component built from the same array, drawn each frame by the renderer. Every draw phase still iterates a query and draws what it finds. The bullets stay out of the 64-slot entity table, and everything drawn is reachable from an entity.

Every renderer backend has to learn to draw the new component independently, which gave each one a chance to get it wrong.

Bug 1 (Vulkan): a guard that was checking the wrong component

Each backend's sprite system is imported once per world, so it needs a guard against importing twice. Comparing the stored world pointer with the current one breaks when a different world is allocated at the same address, which every test here does by tearing a world down and creating a fresh one. The convention in the renderer imports is to also check that a component the import function itself registers is present on the world.

The check written for SpriteBatch tested the wrong component:

/* Wrong: Sprite is registered by the renderer's own import, before
 * this function ever runs -- it says nothing about whether THIS
 * function has run on this particular world. */
if (s_world == world && FRAMER_COMPONENT_IS_REGISTERED(world, Sprite))
    return;

On a fresh world reusing a previous world's address, Sprite is already registered by the caller, earlier in the same framer_sprite_system_import(), before the guard runs. The check reports "already set up" on the first call and the function returns without registering SpriteBatch or its draw system. Sprites drew; batches never did.

A test caught it. The new pixel-readback test for SpriteBatch (draw an element, read the pixel back, expect it filled) passed on the software and OpenGL backends and failed on Vulkan, whose suite happened to run a batch test on a world at an address a prior test had used. The fix checks the component this function registers:

if (s_world == world && FRAMER_COMPONENT_IS_REGISTERED(world, SpriteBatch))
    return;

The OpenGL and software backends had the identical guard, found by inspection before either failed a test. Both got the same fix and were verified.

Bug 2 (OpenGL): reading back a write that hasn't happened yet

The OpenGL backend compiles its default shaders lazily, on first request, and caches them in the world's Shader singleton component:

const struct shader *shaders = FRAMER_GET(world, FRAMER_ID(Shader), Shader);
if (!shaders) {
    unsigned int sprite = framer_shader_create(...);
    /* ... compile the other three default shaders ... */

    FRAMER_SET(world, FRAMER_ID(Shader), Shader, .sprite_shader = sprite, /* ... */);
    shaders = FRAMER_GET(world, FRAMER_ID(Shader), Shader);
    if (!shaders)
        return 0;
}

switch (type) {
case SHADER_TYPE_SPRITE: return shaders->sprite_shader;
/* ... */
}

This assumes a component write takes effect immediately. It does not: a write issued inside a running system is staged and merged once every system in that step has finished. A second system in the same step can see it, but the call that made the write cannot read it back. That is an ordinary property of this ECS. Every other caller of framer_shader_get_default() came after something upstream (loading a level, spawning a player) had already asked for a shader on an earlier frame, so the singleton existed.

The SpriteBatch draw system is the first thing in a new scene with nothing upstream: no mesh, no plain sprite, just a bullet pool's entity. On the first frame its render system is the first caller of framer_shader_get_default(). The write lands, the read-back in the same call finds nothing, the function returns 0, and that frame renders every batch element with no shader bound. The ECS timing is fine; the fix stops reading back what was just staged:

struct shader created = {.sprite_shader = sprite, /* ... */};

framer_component_set(world, FRAMER_ID(Shader), FRAMER_ID(Shader), &created);
return shader_of_type(&created, type);

Bug 3 (all three desktop backends): drawn correctly, in the wrong order

The first two bugs were "nothing drew". In the third, every bullet drew at the right position, size and texture, but the entity that fired them disappeared behind its own bullets once enough piled up at its position.

SpriteBatch and the per-entity Sprite draw are separate systems in the same ECS phase, and nothing said which goes first. On GBA and NDS the hardware settles it: OAM sprites carry a priority, and entity sprites are drawn at a higher priority than batch elements, so an entity is in front of a batch whatever the system order. Sampling the OAM through mGBA's debug stub mid-run confirms it: the emitter (the round blue sprite at the center) sits in front of the spiral it fired:

The bullets example's spiral pattern, OAM contents read directly off a running GBA — the emitter draws in front of every bullet it fires, by hardware priority, regardless of draw order.

The three desktop backends (OpenGL, Vulkan, software) have no such guarantee: draw order is their only ordering mechanism, and the two systems registered in whatever order the import code called them, which put SpriteBatch after the entity sprite system. Sampling framebuffer pixels at the emitter's position every fifty frames of a Vulkan run shows the failure: the emitter's blue for hundreds of frames, then, once enough live bullets reach that pixel, whatever bullet is passing through it:

frame  350: 2850c8 5a96ff 5a96ff 5a96ff 2850c8   (emitter blue)
frame  750: 2850c8 5a96ff 5a96ff 5a96ff 2850c8   (emitter blue)
frame  800: c81e28 ff7828 ffdc50 ff7828 c81e28   (bullet red/orange)
frame 1200: c81e28 ffdc50 fffff0 ff7828 c81e28   (bullet, still)

The fix declares the order explicitly instead of relying on registration order, on every desktop backend:

static const struct framer_system_dep batch_deps[] = {
    {.before = "sprite_render_system", .after = NULL}};

framer_system_register(world, "sprite_batch_render_system",
                        FRAMER_PHASE_ON_UPDATE,
                        sprite_batch_render_system, bq, batch_deps, 1);

A new renderer test pins it down: a red entity sprite and a white batch element on the same pixels, asserting the pixel reads back red.

Verification: one bug per backend, so one check per backend

The design is not backend-specific, but each backend had its own bug, so each was checked separately. The desktop pixel-readback tests (software, OpenGL, Vulkan) ran the same three scenes. examples/bullets was cross-compiled for GBA and NDS and its OAM sampled through mGBA's debug stub, the technique the dithering article used to catch a hardware-only bug. Every GBA example ROM was booted headlessly to check that a related fix did not regress startup: one more component pushed the fixed-stride storage layout past EWRAM, so component storage is now pooled.

One result was the same everywhere I checked: with the spiral pattern running continuously, the live bullet count settles in the high 90s on GBA, OpenGL and Vulkan alike.

Where this goes next

FRAMER_STATIC_MAX_COMPONENTS (40) and FRAMER_STATIC_MAX_ENTITIES (104, leaving 64 for game use) both grew slightly to fit the new component without pushing the pooled storage past EWRAM on GBA. What remains for the bullet pool is measuring and documenting the per-target bullet ceiling. The capacity constants (examples/bullets uses 128 on GBA/NDS, 1024 on desktop) were picked before there were numbers to size them against.

Introduction

The two previous GBA articles, on cycle-level optimization and the no-heap ECS underneath it, were about working around hardware with no float unit and no polygon fill. This one is about an older constraint on the same hardware: the GBA's Mode 4 framebuffer is 8bpp indexed, and its palette is a single unbuffered 256-entry table with no double-buffering and no VBlank gating on writes. Reassigning a palette slot recolors every pixel already on screen that uses it, immediately, mid-frame if the timing is unlucky.

That rules out blending per pixel and allocating the result a palette slot on the fly. Blending would also be too slow. framer-engine's GBA backend rasterizes 3D faces in software on a 16.78 MHz ARM7TDMI with no hardware alpha unit, and the earlier optimization article spent its length shaving cycles off this rasterizer's inner loop. Blending means mixing source and destination colors and resolving the result to a palette index for every covered pixel of every translucent face, every frame. Dithering avoids that: the per-pixel decision is one threshold compare against a precomputed matrix, with no blend math and no palette search. Translucent faces are drawn to roughly the fraction of screen cells their alpha implies, using a fixed Bayer matrix, so the density of drawn pixels approximates the alpha.

That part worked the first time. What failed, twice and in two different ways, both visible only on real hardware, was making overlapping translucent objects dither differently enough from each other to stay visible. The second bug is a quieter version of the first, and the fix that stuck came from building three ROMs and looking at them, not from a smarter formula.

The setup: one matrix, one offset per object

The dithering is a small pure function. Given a matrix of thresholds and a per-object level (roughly alpha out of 16), a pixel draws if the matrix value at its screen coordinate is below the level:

static const unsigned char DITHER_BAYER_4X4[4][4] = {
    {0,  8,  2, 10},
    {12, 4, 14,  6},
    {3, 11,  1,  9},
    {15, 7, 13,  5},
};

static inline bool dither_should_draw(int x, int y, int level)
{
    return DITHER_BAYER_4X4[y & 3][x & 3] < level;
}

That is enough for one translucent object, not for two. examples/translucent_cubes draws three overlapping cubes (red, green, blue) at the same alpha, back to front. With no per-object variation they test the same screen cells against the same level, so they draw to the same pixels, and the nearest cube overwrites the farther ones where they overlap instead of blending with them. Each object needs its own dithering phase: a draw-set that differs from its neighbors' while still covering the right fraction of pixels.

Bug 1: shifting the coordinate collapses sixteen phases into two

The obvious way to give each object a phase is to shift the sampled coordinate before indexing the matrix: object i looks up BAYER[(y + py) & 3][(x + px) & 3] for some per-object (px, py). That gives sixteen (px, py) pairs and, apparently, sixteen phases. It passed review and looked right in the emulator.

It did not look right in the screenshots from hardware. With three overlapping equal-alpha cubes, the middle one, green, was almost invisible wherever it overlapped blue.

examples/translucent_cubes at the commit that shipped the coordinate-shift phase. Green and blue have collapsed into the same draw-set as another object and vanished.

Enumerating all sixteen shifts against the 4x4 matrix at that alpha level showed the cause: only two distinct draw-sets exist among the sixteen shifts. The 4x4 Bayer matrix has translational symmetry, so every shift is equivalent, cell for cell, to another shift with the same parity of (px + py). Two objects in the same parity class draw to identical cells, and the nearer one erases the other. Three objects and two parity classes guarantee a collision.

The fix: rotate the compared value, not the sampled coordinate

The working version leaves the coordinate alone and rotates the value read from the matrix by a per-object offset before comparing it with the level:

static inline bool dither_should_draw(int x, int y, int level,
                                       int offset)
{
    return (int)((DITHER_BAYER_4X4[y & 3][x & 3] + offset) & 15) <
           level;
}

DITHER_BAYER_4X4 is a bijection: its sixteen cells hold the values 0 to 15 once each. The cells with (value + offset) mod 16 < level form a level-sized contiguous arc of that range, and as offset goes from 0 to 15 the arc's start visits every residue once. Sixteen offsets give sixteen distinct draw-sets at every level, by construction and not just for the alpha I tested, as sixteen Caesar shifts give sixteen distinct ciphertexts. This shipped and the green cube reappeared. The next report from hardware was different: not invisible, but rougher than the other two.

The demo after the value-rotation fix. All three cubes are visible, but the thinnest sliver, drawn last and forced onto a worse offset, already reads rougher than its neighbors.

Bug 2: only two offsets out of sixteen are smooth

"Rougher" was exact. At the alpha level examples/translucent_cubes uses, offset 0 produces a perfect alternating checkerboard:

offset=0                     offset=1
#.#.#.#.                     #.#.#.#.
.#.#.#.#                     .#.#.#.#
#.#.#.#.                     #.#.#.#.
.#.#.#.#                     .#.#.#.#

Only one other offset (8, half the matrix's sixteen values away) produces the other perfect checkerboard, disjoint from offset 0's. Every offset in between is the same pattern with a few cells flipped, less regular the farther it sits from 0 or 8. Three overlapping objects need three distinct offsets, which the previous fix gave, but with only two perfect offsets in a 16-value matrix the third object is forced onto an irregular one. That is a structural property of a small matrix, not a bug in the value rotation.

A bigger matrix buys a finer gradient, not an exemption

An 8x8 matrix (64 distinct values) does not remove the "only two perfect offsets" property; the argument above does not depend on matrix size. It gives a much finer gradient around the two perfect points, so a third offset near either one looks close to perfect. The caller contract is unchanged: levels still run from 0 to 16, and the comparison is rescaled by 4 to match the 8x8 matrix's 0-63 range:

static inline bool dither_should_draw(int x, int y, int level,
                                       int offset)
{
    return (int)((DITHER_BAYER_8X8[y & 7][x & 7] + offset) & 63) <
           level * 4;
}

The third cube went from "noticeably wrong" to "close to right". That still did not fix the problem that mattered.

The bug the roughness metric couldn't see

The offsets came from a 64-entry table ranked by how close each offset's own pattern is to a perfect checkerboard: offset 0 first, offset 32 (the 8x8 matrix's other perfect point) second, then the next closest. For three visible objects the table handed out 0, 32, 1.

Offset 1 is individually an excellent choice: one flipped cell out of sixty-four. The ranking never asked how much offset 1's draw-set overlaps offset 0's. The answer is 31 of 32 drawn cells identical. The third object was again drawing to almost the same pixels as the first, which is the original bug in a better disguise: "individually smooth" and "well separated from neighbors" are different properties, and the ranking had conflated them.

Testing three sequences instead of guessing

Fixing the separation analytically hit a trade-off. Offsets chosen only to maximize separation, spread evenly over the 64-value range, land on values like 16 and 48, which render as regular vertical or horizontal stripes instead of checkerboard noise:

offset=1 (roughness-ranked)      offset=16 (separation-first)
#.#.#.#.                         #.#.#.#.
.#.#.#.#                         #.#.#.#.
#.#.#.#.                         #.#.#.#.
.#.#.#.#                         #.#.#.#.
#.#.#.#.                         #.#.#.#.
.#.#.#.#                         #.#.#.#.
#.#.#.#.                         #.#.#.#.
#..#.#.#                         #.#.#.#.

The stripe pattern shares 16 of 32 cells with its neighbors instead of 31, a large improvement by the numbers. But axis-aligned stripes are a textbook case of a dither that is more noticeable than diagonal checkerboard noise. More computation could not say whether that trade looks better than the near-miss checkerboard.

So I built three complete 64-entry offset sequences: the original roughness-ranked table; a separation-maximizing one, a bit-reversal permutation (a Van der Corput sequence, where entry i is the 6-bit binary reversal of i and every prefix is evenly spread over the offset range); and a capped-overlap middle ground. I compiled each into a translucent_cubes.gba ROM and ran them in mGBA. The separation-maximizing sequence won, stripes included.

static const unsigned char GBA_DITHER_OFFSET_SEQUENCE[64] = {
    0,  32, 16, 48, 8,  40, 24, 56, 4,  36, 20, 52, 12, 44, 28, 60,
    2,  34, 18, 50, 10, 42, 26, 58, 6,  38, 22, 54, 14, 46, 30, 62,
    1,  33, 17, 49, 9,  41, 25, 57, 5,  37, 21, 53, 13, 45, 29, 61,
    3,  35, 19, 51, 11, 43, 27, 59, 7,  39, 23, 55, 15, 47, 31, 63,
};

The demo with the bit-reversal sequence: three distinct, evenly separated offsets, with no object drawing to nearly the same cells as another.

Verifying a change with nothing to count

The GBA performance article relied on cycle_probe.py, a deterministic cycle count diffed between builds. It does not apply here: this is an output correctness question, and the claims are combinatorial. I checked each of them by construction in Python before writing the C: the sixteen distinct draw-sets of the value rotation, the exact pixel count at every level and offset of the 8x8 matrix, and the even prefix spread of the bit-reversal sequence. Each check enumerated every offset, level and pair. Every change also got the full desktop test suite, a GBA cross-compile and a headless mGBA run of examples/translucent_cubes. For the final sequence choice I looked at the rendered result, the only step that caught what the math could not.

Where this goes next

The bit-reversal sequence is one fixed table shared across every scene. It optimizes pairwise separation along the sequence, not for the specific set of objects overlapping in a given frame. For a small, fixed handful of translucent objects that is enough. A scene with many more would exhaust the best-separated entries and reach ones no better than the original bug. Assigning offsets jointly across the objects in use each frame is the next step if that happens.

Introduction

The previous article was about squeezing cycles out of framer-engine's GBA renderer. This one covers what has to be settled before that work is possible: how to run an Entity Component System, the pattern the engine is built around, on a console with 32KB of fast RAM, 256KB of slow RAM, no operating system and no usable malloc().

framer-engine answers with a second, much smaller ECS implementation behind the same interface as the desktop one. Component definitions, systems and queries are identical between a PC build and a GBA build. Only one file differs: which world.c is linked.

Two ECS backends, one interface

The default backend is Flecs, a full-featured ECS used for desktop and handheld targets with a real OS (PSP, 3DS, Switch, PC). It provides archetypes, queries, observers and relationships, and all of that bookkeeping assumes a heap that grows and shrinks as entities and component types come and go. The GBA has 32KB of IWRAM and 256KB of EWRAM, both fixed-size and fully accounted for from boot, with no MMU and no OS to page anything in. There is no heap.

A second full-featured ECS sized to that budget would be a lot to maintain for a problem that does not need it. GBA-class scenes hold a handful of objects (examples/simple_cube uses one mesh and a camera, examples/spinning_shapes three), not thousands, and the component types are fixed at compile time. src/ecs/static/world.c, about 550 lines of plain C, is sized to that instead of to Flecs:

/* Entities */
static bool s_alive[FRAMER_STATIC_MAX_ENTITIES];
static uint64_t s_comp_mask[FRAMER_STATIC_MAX_ENTITIES];

/* Component data store: row = component slot, column = entity slot.
 * Each entity slot is FRAMER_STATIC_MAX_COMPONENT_SIZE bytes wide. */
static uint8_t s_store[FRAMER_STATIC_MAX_COMPONENTS]
                       [FRAMER_STATIC_MAX_ENTITIES *
                        FRAMER_STATIC_MAX_COMPONENT_SIZE];

Every array is a fixed-size global sized by compile-time constants. Nothing in this file can fail by running out of memory unpredictably. It can only run out of array slots, a number known before the program starts.

Both backends sit behind the same include/framer/ecs.h: framer_world_t, framer_component_register(), framer_query_create(), framer_system_register() and the FRAMER_GET/FRAMER_SET/ FRAMER_FIELD macros. A component defined with FRAMER_COMPONENT_DEFINE(Velocity) and a system registered with framer_system_register() compile and run unchanged on either. A single Meson option selects the backend, -Decs=flecs or -Decs=static. GBA's cross file pins it to static because Flecs needs an OS, and the build refuses to configure otherwise.

Entities are array slots

An entity here is a 1-based index into those parallel arrays. framer_entity_create() claims the lowest dead slot, first-fit:

framer_entity_t framer_entity_create(framer_world_t *world)
{
    int i;

    for (i = 0; i < FRAMER_STATIC_MAX_ENTITIES; i++) {
        if (!s_alive[i]) {
            s_alive[i] = true;
            s_comp_mask[i] = 0;
            if (i + 1 > s_entity_high)
                s_entity_high = i + 1;
            return EID(i);
        }
    }
    return 0; /* pool exhausted */
}

That is the whole allocator. There is no free list and no fragmentation, since every slot has the same size and a compile-time address. The price is deliberate: destroying entity 5 and creating a new one immediately returns ID 5 for a different logical entity, and there is no generation counter to tell them apart. For a handful of objects that mostly live for a whole level, that is acceptable. test_entity_slot_recycled_first_fit pins it down so it stays a known property.

Components: a bitmask and a flat array

Each entity slot carries one uint64_t bitmask with one bit per registered component type. framer_component_set() is a memcpy into a fixed-stride row of s_store plus a bit set:

void framer_component_set(framer_world_t *world, framer_entity_t e,
                          framer_id_t id, const void *data)
{
    int ei = EIDX(e);
    int ci = cidx(id);
    /* ...bounds and liveness checks elided... */
    memcpy(&s_store[ci][ei * FRAMER_STATIC_MAX_COMPONENT_SIZE], data,
           s_comp[ci].size);
    s_comp_mask[ei] |= ((uint64_t)1u << ci);
    s_any_mask |= ((uint64_t)1u << ci);
}

A query is a precomputed mask built from the component ids it asks for, and matching an entity is one AND and one comparison ((s_comp_mask[ei] & mask) == mask). The uint64_t is also the ceiling on distinct component types in a world: 64. That is plenty for a retro scene, and it stops anyone building something Flecs-shaped on top.

A silent NULL deref vs. a loud abort()

The most important property of this backend is what happens when a limit is hit. framer_component_register() originally returned 0 when a component was too big or the registry was full, like the rest of the API. A 0 component id then does damage downstream: framer_query_create() skips it when building the mask, and the system that registered the query calls FRAMER_FIELD() on a field that was dropped, dereferencing NULL on its next matching entity. This happened on a real GBA build: a Text component of 288 bytes against a 64-byte cap, and 17 component types against a cap of 12. The crash was a NULL-pointer SIGSEGV inside a render system, two layers away from the mistake, not "your component is too big".

The fix removes the silent path:

if (size > FRAMER_STATIC_MAX_COMPONENT_SIZE) {
    fprintf(stderr,
        "framer_component_register: \"%s\" is %lu bytes, "
        "exceeding FRAMER_STATIC_MAX_COMPONENT_SIZE (%d)\n",
        name ? name : "?", (unsigned long)size,
        (int)FRAMER_STATIC_MAX_COMPONENT_SIZE);
    abort();
}

It uses abort() instead of assert() or a GCC/Clang builtin because this backend also has to compile under cc65 and SDCC for 8-bit targets, and C89 abort() is the one failure primitive guaranteed everywhere. The tests check the contract, not just the arithmetic: test_ecs_static_limits.c forks a child, registers one component past the limit, and asserts the child died of SIGABRT. That proves the fail-loud path fires.

On a backend made of fixed-size arrays, every hard limit is a wall. The choice is hitting it with a clear message at the offending call, or with a corrupted query and a crash three calls away. This backend fails loudly wherever a hard cap exists.

Sizing the pools, and a second silent-failure bug

Each target's meson.build picks FRAMER_STATIC_MAX_COMPONENTS, FRAMER_STATIC_MAX_ENTITIES and FRAMER_STATIC_MAX_COMPONENT_SIZE for that platform. No universal default fits, because the engine registers its full set of core components (Transform, Velocity, Sprite, Light, Text, Camera and so on, 19 today) regardless of what an example uses:

# Embedded (system == 'none'):
-DFRAMER_STATIC_MAX_COMPONENTS=19
-DFRAMER_STATIC_MAX_ENTITIES=96
-DFRAMER_STATIC_MAX_COMPONENT_SIZE=64

MAX_ENTITIES=96 has its own bug behind it. Each registered component type, not each entity created, reserves one sentinel slot from the same entity pool, so the entities available to a scene are MAX_ENTITIES minus the number of component types. At MAX_ENTITIES=64 and 19 components that left 45 usable slots, enough for examples/simple_cube but one short of examples/input_tester's 48 (47 on-screen panel entities plus a camera). The failure was silent again: framer_entity_create() returned 0 once the pool filled, and the caller wanting one more entity for a gamepad-axis label never got it. The text did not appear and nothing in the logs said why. Raising the cap to 96 (77 free after the 19 sentinels) fixed it. On GBA that costs about 544 bytes of IWRAM (three per-entity arrays scale with the cap), which I checked against the build's free IWRAM margin before landing. On a 32KB budget you measure.

As in the performance article, arm-none-eabi-size -A on a current GBA build shows where this backend's memory goes. The flat component store (19 × 96 × 64 bytes ≈ 114KB) goes in EWRAM's .sbss section: too big for IWRAM, and zero-initialized by the startup code at no ROM cost:

#ifdef GBA
#define _FRAMER_STORE_ATTR __attribute__((section(".sbss")))
#else
#define _FRAMER_STORE_ATTR
#endif

static uint8_t _FRAMER_STORE_ATTR
    s_store[FRAMER_STATIC_MAX_COMPONENTS]
           [FRAMER_STATIC_MAX_ENTITIES * FRAMER_STATIC_MAX_COMPONENT_SIZE];

Everything else (alive flags, masks, the per-frame iterator's entity list) is small enough for IWRAM, the GBA's fast 32KB memory, where the CPU wants its hot data. On the current examples/simple_cube build that is roughly 19.5KB of 32KB used. That leaves room for a component or two, and the number needs rechecking whenever the component count grows, as the 96-entity change did.

The scan loop

framer_world_progress() walks every registered system in phase order and, for each, scans entity slots up to the high-water mark (s_entity_high, one past the highest slot any entity or component sentinel has occupied) looking for bitmask matches. The sticky s_any_mask check skips the scan entirely for systems whose component type no entity has ever had. That was the biggest win in the previous article's performance work, covered there. The design point here is that this is a linear scan over a flat array, not a sparse-set or archetype lookup. That is right at GBA scene sizes (tens of entities) and wrong at thousands, where you would reach for Flecs.

What this design gives up

Two backends exist so that these trade-offs are made once, explicitly, per target:

• No entity generations. IDs are recycled immediately and look identical to the entity that held them before.
• No archetypes, no sparse sets. Matching is a linear scan with a bitmask test, not a cache-friendly iteration over exactly the matching entities. Fine at dozens of entities, wrong past that.
• 64 component types in total, for the whole world. Not per query: every component type in the engine, shared across every system. The current 19 leaves room, but adding a component means someone must recheck that ceiling.
• No relationships, hierarchies or observers. Flecs has all three. This backend has entities, components and queries.

Flecs has these capabilities and this backend does not, and they are the features that cost a heap, dynamic bookkeeping and memory use unknowable at compile time, none of which a bare-metal ROM can afford.

Where this goes next

Every retro target on the roadmap has the same constraints: no OS, no heap, a fixed memory map and a known component set. The 32-bit-era consoles mentioned at the end of the previous article, starting with the PlayStation 1, have far more RAM and a real GPU but still no OS and no desktop heap. I expect them to use this static backend with much larger pool constants, not Flecs and not a third ECS. Whether that holds on a real PS1 build is for a later article.

Introduction

framer-engine is a small cross-platform ECS game engine I am building, with backends from desktop OpenGL/Vulkan down to bare software rendering on 8/16-bit consoles. The Game Boy Advance backend renders textured and shaded 3D meshes (cubes, cones, spheres) through a CPU-only software rasterizer on a 16.78MHz ARM7TDMI with no FPU and no hardware polygon fill. Every float operation and every divide is a library call, and every pixel is a CPU read-modify-write into VRAM.

examples/simple_cube, captured from mGBA. The simple_cube numbers below are measured on this demo.

Every number below comes from scripts/debug/gba/cycle_probe.py, a script that sets a breakpoint on the engine's vblank-wait function inside headless mGBA and reads the emulator's cycle counter on every hit. mGBA's CPU emulation is deterministic: the same ROM with the same inputs gives bit-identical cycle counts every time, so a claim is "frame N now costs X fewer cycles", not "it looked smoother". I discard the first ~100 frames as warm-up (caches, lazy first-frame setup) and average the steady-state window after that.

That discipline matters more than any single trick below, because twice an optimization that was obviously correct measured as a regression. Those are at the end.

The hardware constraint

The GBA display is locked to the LCD's scanout rate. A frame takes 280896 system clock cycles to display whether or not your CPU work fits in it. If it does not, you display every other frame or worse, and the frame rate quantizes to 59.73 / n for whatever integer multiple of the budget a frame costs. There is no GPU to defer to and no graceful way to miss the deadline. The goal is a CPU-side frame cost under, or close to, 280896 cycles.

Two limits drive every technique:

1. No FPU. Any float/double multiply, divide, sqrtf(), sinf()/cosf()/tanf() compiles to a call into ARM's soft-float runtime: a function call plus a software algorithm per operation.
2. No hardware rasterizer. Mode 4's bitmap layers are VRAM written by the CPU. Every pixel a triangle covers is computed and stored by the ARM7TDMI.

Technique 1: fixed-point math instead of float

The most basic change: the hot path (per-vertex transform, per-pixel rasterization) uses Q12 fixed-point integers through a small fix_t type (src/backends/renderer/common/sw3d_fixed.h):

typedef int32_t fix_t;

#define FIX_SHIFT 12
#define FIX_ONE   (1 << FIX_SHIFT)   /* 4096 == 1.0 */

static inline fix_t fix_mul(fix_t a, fix_t b)
{
    return (fix_t)(((int64_t)a * (int64_t)b) >> FIX_SHIFT);
}

The int64_t intermediate looks expensive, but on ARM it lowers to a single hardware SMULL (signed multiply, 64-bit result), with no library call. A float * float on this target is a soft-float call doing mantissa and exponent bookkeeping in software.

Division still hurts, because the ARM7TDMI has no hardware divider either way: a fixed-point divide costs an integer library call (__aeabi_idivmod and friends) instead of a float one. The perspective divide exploits a property of that specific division to cut the cost:

/* fix_div()'s general implementation widens to a 64-bit intermediate to
 * stay correct for arbitrary numerators, but the perspective divide's
 * numerator is always FIX_ONE, so FIX_ONE << FIX_SHIFT never exceeds
 * 32 bits. */
static inline fix_t fix_reciprocal(fix_t b)
{
    return (fix_t)(((int32_t)FIX_ONE << FIX_SHIFT) / b);
}

Replacing the general 64-bit fix_div() with this 32-bit reciprocal at the one call site whose numerator is always FIX_ONE saved ~50,000 cycles/frame on examples/spinning_shapes.

Technique 2: a LUT instead of sinf()/cosf()

framer_transform_get_matrix(), the shared cross-platform transform code, builds rotation matrices with cglm's glm_rotate_{x,y,z}(), which call cosf()/sinf(). On desktop that is a couple of FPU instructions. On the GBA it is a soft-float libm round trip per axis, per object, per frame.

examples/spinning_shapes, captured from mGBA: three objects rotating on all three axes every frame. Every spinning_shapes number here comes from this demo.

The GBA backend carries its own 256-entry sine table (sw3d_raster.c), reads cosine from the same table at a quarter-turn offset, and interpolates linearly between samples:

static const fix_t gba_sin_lut[256] = { /* ... */ };

static void gba_fast_sincosf(fix_t angle_turns_256, fix_t *s, fix_t *c)
{
    int idx = angle_turns_256 & 0xff;
    int cidx = (idx + 64) & 0xff; /* cos(x) == sin(x + tau/4) */

    *s = gba_sin_lut[idx];
    *c = gba_sin_lut[cidx];
}

256 entries is ~1.4° between samples, far finer than a 240x160 screen shows, so the interpolation error never appears as jitter. Swapping it in for the float sin/cos chain, measured A/B (git stash plus identical build and measure commands) on spinning_shapes (3 objects rotating on all 3 axes every frame): 1,294,320 → 1,234,341 cycles/frame, ~4.6% less, from removing one class of soft-float call.

A follow-up went further. cglm builds the rotation matrix with up to three generic 4x4 matrix multiplies, one per nonzero Euler axis, each a 64 multiply-add matmul although most entries of a pure-axis rotation matrix are 0 or 1. Instead, the 9 nonzero 3x3 entries of the combined Rz·Ry·Rx product are expanded by hand from the three angles' sin/cos (still from the LUT) and folded into the output with one glm_mat4_mul:

/* out = Rz * Ry * Rx, 9 nonzero entries expanded by hand instead of
 * three generic 4x4 matmuls. */
out[0][0] = cy * cz;
out[0][1] = cy * sz;
out[0][2] = -sy;
out[1][0] = sx * sy * cz - cx * sz;
out[1][1] = sx * sy * sz + cx * cz;
out[1][2] = sx * cy;
out[2][0] = cx * sy * cz + sx * sz;
out[2][1] = cx * sy * sz - sx * cz;
out[2][2] = cx * cy;

I verified it against the three-matmul path with NumPy differential tests over all 8 zero/nonzero axis combinations and thousands of random angle triples (max absolute error ~1e-16) before it touched the renderer. The gain was only 1,309,080 → 1,306,224 cycles/frame, ~0.22%, far less than the operation count suggests, because the optimizer already folds away most of the zero and one multiplies when each Rz/Ry/Rx factor starts from an identity-seeded matrix. Hand-expanding math only pays once you have checked what the compiler already does. A branch-free variant that skipped the matrix multiply altogether (translation column copy plus per-column scale) measured worse in every iteration than this one-matmul version, and I discarded it.

Technique 3: Quake III's fast inverse square root

Triangle shading needs each surviving triangle's world-space normal, normalized, once per shaded triangle per frame, in the renderer's hottest loop. cglm's glm_vec3_normalize() calls sqrtf() and divides by the result: two soft-float calls per triangle.

The fix is the well-known bit-hack:

static float sw3d_fast_inv_sqrt(float number)
{
    union { float f; uint32_t i; } conv = { .f = number };

    conv.i = 0x5f3759df - (conv.i >> 1);
    conv.f *= 1.5f - (0.5f * number * conv.f * conv.f); /* one Newton-Raphson step */
    return conv.f;
}

A magic-constant shift gives a rough inverse square root straight from the float's IEEE bit pattern, with no sqrt call, and one Newton-Raphson step sharpens it until it is visually indistinguishable for lighting. It replaces both the sqrt and the divide at every site in the GBA backend that called glm_vec3_normalize() (face-normal lighting in the renderer and the rasterizer's triangle-normal centroid computation), removing two soft-float calls per triangle.

Technique 4: an ECS dispatch early-out

Not every win is in the renderer. framer_world_progress(), the ECS scheduler's per-frame loop, walked every registered system's full entity range every frame, including systems whose query needs a component type that no entity in the scene has ever had. simple_cube registers collider/velocity/rigidbody systems on every platform (component import is unconditional), so most systems scanned entities every frame to match none.

The fix tracks a sticky OR of every component bit ever set in the world and skips a system's scan entirely, in O(1) with no entity walk, when its query's required mask includes a bit outside that set, since it can never match:

/* s_any_mask: sticky OR of every component bit ever set across the
 * world's lifetime. A query whose mask requires a bit outside this set
 * can never match any entity — skip the per-entity scan entirely. */
if ((q->mask & world->s_any_mask) != q->mask)
    continue;

This is the largest single win in the project: simple_cube 308650 → 288629 cycles/frame, spinning_shapes 757806 → 744701 cycles/frame (steady-state averages over frames 101-150). It is a scheduler fix, not a renderer trick, found the same way: measure where the cycles go.

Technique 5: making divides Bresenham-shaped

The scanline rasterizer (sw3d_fill_triangle()/sw3d_fill_quad()) originally tested every pixel in each triangle's bounding box against all three edge functions. The replacement computes each row's [lo, hi] x-span per edge, incrementally. That is Bresenham's line algorithm applied to "x as a function of y" along a triangle edge:

/* Incrementally tracks bound(y) = floor((b0 + (y - y0) * d) / a) for a
 * fixed positive `a`, one row at a time, with zero divisions after
 * init. The GBA's ARM7TDMI has no hardware divider, so trading one
 * division per edge (at init) for what used to be a same-sign test on
 * every bounding-box pixel is the whole point. */
struct row_bound {
    long val, step, rem, err, a;
};

"One division-equivalent test per candidate pixel" becomes "one division per triangle edge plus an integer add per row", a large change on a CPU with no divider.

It also produced an unusual micro-optimization. The one division still needed per edge (floordiv_pos()) is built on a / b and a % b in C, which GCC should fuse into a single __aeabi_idivmod call. Disassembly showed the fusion on one branch (a > 0) but not the other (a < 0, which negates both operands first): a redundant __aeabi_idiv next to the __aeabi_idivmod for the same division. Restructuring the C source gave byte-identical codegen, so I could not fix it from the C side. The fix is to call the library function directly and unpack its packed 64-bit r0:r1 quotient/remainder by hand, so the compiler cannot make the wrong fusion choice:

extern long long __aeabi_idivmod(long numerator, long denominator);

static long floordiv_pos(long a, long b)
{
    long long qr = __aeabi_idivmod(a, b);
    long q = (long)(uint32_t)qr;
    long r = (long)(qr >> 32);

    if (r != 0 && a < 0)
        q--; /* C truncates toward zero; floor() needs a -1 correction */
    return q;
}

This saved roughly 25,000-30,000 cycles/frame on spinning_shapes, by removing one redundant library call on one branch that no compiler flag could remove.

Technique 6: let the hardware scale a smaller image

The GBA has no hardware polygon fill, so every covered pixel is a CPU read-modify-write into VRAM. That is the floor under every other optimization here: once every avoidable divide and float operation is gone, you are still bound by how many pixels the CPU touches.

The way around it is not a CPU optimization. Mode 4's BG2 layer supports affine transforms even though it is a flat bitmap, the trick behind GBA games that faked SNES Mode 7 scaling. The renderer draws only a 120x80 corner of the framebuffer (a quarter of the 240x160 screen's pixels) and BG2's affine matrix stretches it across the screen at scanout, in hardware, at no CPU cost:

#if GBA_RENDER_SCALE == 1
static inline void gba_clear_buffer(vu16 *base) { /* full-res clear */ }
#else
static inline void gba_clear_buffer(vu16 *base)
{
    /* only clear the GBA_RENDER_WIDTH x GBA_RENDER_HEIGHT corner that's
     * actually sampled by BG2's affine matrix — the rest of the page is
     * never displayed, so clearing it is wasted work. */
}
#endif

On spinning_shapes this took the steady-state cost from ~1.55M to ~1.28M cycles/frame, roughly 10.8fps → 13.1fps, ~17% less, at the price of visibly blockier 2x nearest-neighbor edges. It is opt-in (-Dgba_half_res), because unlike every other technique here it is a visible quality trade, not a free win.

How the measurements work

None of the numbers above are estimates. scripts/debug/gba/cycle_probe.py drives headless mGBA, breaks on the engine's vblank-wait call (the one point every frame passes through exactly once), and reads the emulator's cycle counter on every hit. mGBA's CPU core is deterministic, unlike real silicon, so the same ROM, breakpoint and warm-up discard give bit-identical cycle counts on every run. "Did this help?" becomes a yes/no question: rebuild, re-run the probe, diff the number.

That is also what caught the two optimizations that were obviously correct and were not.

Failed attempt 1: caching screen-space half-extents that never change

The camera's screen-space half-width and half-height, once converted to fixed-point, do not change from frame to frame unless the projection changes. Hoisting the conversion out of the per-vertex projection loop and caching it looked like a free win: same values, computed once instead of once per vertex.

It measured as a regression.

The likely cause, from inspecting the generated assembly: the project builds with link-time optimization (LTO) and -Doptimization=3, and LTO's inlining heuristics are sensitive to function and loop size in ways the C source does not show. A cache check, even a cheap one, in an already-hot inlined loop changed the cost/benefit the inliner used elsewhere in the same translation unit, and the loss of more valuable inlining outweighed the arithmetic saved. The hoist was right about the math and wrong about the measured outcome.

Failed attempt 2: skipping integration work for a zero velocity

The same thing happened independently in velocity_integration_system(). Most entities in simple_cube have a Velocity that is exactly zero every frame. A zero-vector early-out before the glm_vec3_scale/glm_vec3_add calls is mathematically a no-op, so it looked like free cycles for every entity that was not moving:

/* tempting, and wrong on this build */
if (glm_vec3_isvalid(v->linear) && glm_vec3_norm2(v->linear) == 0.0f &&
    glm_vec3_norm2(v->angular) == 0.0f)
        continue;

Measured: +112 cycles/frame on simple_cube, +312 on spinning_shapes, a regression from a change with no behavior difference. Same root cause: the early-out added code size and a branch to a hot loop, LTO's inlining shifted, and the inlining lost elsewhere cost more than the skip saved. I reverted it in the same session.

The takeaway is not that loop-invariant hoisting or early-outs are untrustworthy; both are standard and usually right. Once a build relies on LTO and aggressive optimization levels, the compiler's decisions are part of the system you are optimizing, and they do not always move the way your model of the code predicts. The only check is the one used for every win here: change one thing, measure, keep it only if the number goes down.

Failed attempt 3: quantizing colors in the wrong number system

The GBA backend's shaded sprites and triangles go through gba_palette_index(), a linear scan (up to 256 entries, with a nearest-color distance calculation once full) that maps a computed RGB555 value onto BG_PALETTE, since Mode 4 is 8bpp paletted. A one-entry cache handles two consecutive calls requesting the same value. But continuous per-frame lighting (examples/lighting, added since this article's original techniques, orbits two colored point lights around a static sphere and cube) makes each shaded triangle request a slightly different value almost every frame. That defeats the cache and saturates the palette within seconds.

The obvious fix was to round each lit color channel to a coarser step, 16 buckets instead of RGB555's 32, before the palette lookup. A gradually changing light then collapses onto fewer, more repeated values: more cache hits and a palette that stays truer to intent for longer. The first implementation worked in float space, where the lit color already was a float:

/* looked free, measured otherwise */
static inline float gba_quantize_channel(float v)
{
    const float steps = 16.0f;
    return (float)(int)(v * steps + 0.5f) * (1.0f / steps);
}

Measured with cycle_probe.py on the same orbiting-lights demo, 150 frames, steady-state average over frames 101-150: 646528 cycles/frame with no quantization versus 658869 with it, a ~1.9% regression.

This is still the FPU-less ARM7TDMI. v * steps + 0.5f, the cast and * (1.0f / steps) are three more soft-float calls on each of the three channels, for every shaded triangle every frame, which cost more than the palette-scan time saved. An optimization aimed at this hardware's constraint had ignored that constraint.

The fix rounds in integer space, using a value the code already computes. f_to_5bit(), the existing float→RGB555-channel helper, produces a clamped 0-31 integer, and masking off its low bit gives the same 16 buckets for one AND:

static inline u16 f_to_5bit_quantized(float v)
{
    return (u16)(f_to_5bit(v) & ~1u);
}

Same demo, same measurement: 642455 cycles/frame, a modest ~0.63% improvement over the no-quantization baseline. The idea was sound; it had to be expressed in a number system this CPU handles cheaply.

Where this leaves things

examples/simple_cube now sits at 288074 cycles/frame. 16777216 / 288074, the same ratio cycle_probe.py reports for every measurement here, gives ~58.24fps, against a 60fps budget of 280896 cycles (~59.73fps). That is about 2.5% over budget, down from roughly 7-8% before this round of work. spinning_shapes, three shaded objects rotating on all three axes, a heavier scene by design, sits at 741378 cycles/frame, ~22.63fps. These are ceilings for these demo scenes on cycle-accurate emulation, not estimates; more triangles or lights move the frame cost accordingly.

Closing the rest of the gap on simple_cube means riskier changes: caching ECS query results across frames (beyond the existence check from Technique 4), or pre-converting mesh vertex data to fixed-point ahead of time instead of per-vertex at raster time. The latter is complicated because the same mesh struct is also filled through the engine's public float-only custom-mesh API, so caching would mean changing that API or building a cache-on-first-use scheme. Both are bigger than "swap a divide for a multiply", and I am stopping here rather than chasing harder-to-verify returns from more soft-float removal.

What's next

The GBA backend was the first test of whether framer-engine's approach (an ECS with 3D, software-rendered, no FPU) holds up on constrained hardware. The next targets step up in capability: 32-bit-era consoles like the PlayStation 1, and handhelds with 3D acceleration (PSP, Nintendo DS, 3DS). The goal is an engine that is pleasant for small demos and indie games on real retro hardware, GBA included.

At least one of them, probably the PSP with its conventional FPU-plus-GPU setup, is there for another reason. Every technique here exists because the GBA has no FPU and no hardware rasterizer. On a platform with both, those tricks do not apply, and the question becomes how far the engine and hardware go together. That work is closer to traditional real-time 3D budgeting (draw calls, vertex throughput, fill rate) than to soft-float avoidance, and needs the same measurement discipline aimed at a different bottleneck. Whether any of these tricks carry over is for later articles.