I know what you're thinking... on Man-You

プログラマ道 (puroguramā-dō)

Wed, 06 May 2026 22:30:00 +0200

プログラマ道¹.

Companion piece to the 100x post. That one was the framing argument: AI tooling is +0.99 with a wielding bonus, not a multiplier. This is the ground-level version. How I actually drive the thing day to day, what I will not delegate, and why none of this is new.

I drive Claude every day. Across Woosmap services in Python, a music app in Go and SwiftUI on the weekends (Tunes), a SNES toolchain in Python (a816, xdds), and a fair amount of 65c816 ROM hacking. The tool is real. It does not flip the table on the discipline an engineer needs. It raises the floor a notch and rewards the wielder. What the wielder is actually doing, when it works, is the part nobody writes down.

What blindness looks like

Drop Claude into a fresh codebase and watch it grep. It reads filenames, opens files, follows imports by string match, guesses at module boundaries. On a small repo this is fine. On anything real it degrades into expensive guessing. Python projects make it worse: the source of truth for a symbol’s type is often the installed package, not the repo, and the model cannot see installed packages. So it produces plausible code against an imagined API.

The fix is not a better prompt. The fix is to give the model the same thing a human gets: a language server. Once basedpyright is wired up and Claude can ask “what is this symbol, where is it defined, what type does it return”, the questions get sharper and the answers stop being invented. The model does not need to be smarter. It needs to stop being blind.

Our internal stack runs in containers, no local virtualenv, dependencies live inside the image. A model on the host sees nothing. The fix we use internally is a sidecar container that exposes a language server with access to the real installed packages, attached to the application container. Once that is in place, Claude stops hallucinating signatures. Same model, same prompts, dramatically less drift. The intelligence was never the bottleneck. The view was.

The testbed problem, named on a SNES

ROM hacking is notoriously trial-and-error². The CPU does not care about your intent. The PPU cares even less. A bug in HDMA timing is invisible until you stare at the right cycle on the right scanline, and the only way to know you fixed it is to watch the framebuffer change. There is no type system saving you. There is no test framework that ships in the box.

So I built one. kintsuki (yes, typo of kintsugi, name stuck) embeds ares (a SNES emulator) as a C library, exposes Python bindings on top, and gives me programmatic control of the emulator: step execution, trace, read memory, write memory, hook the per-frame interrupt, dump CPU and video memory, diff framebuffers. From there I write pytest cases that drive the ROM to a known state, assert on the bytes that should have changed, and fail loudly when they did not. Regression testing for ROM hacks. With kintsuki in the loop, Claude can iterate. It edits the asm, pytest runs the ROM, the snapshot comes back, the test says green or red.

This is the pattern in general. When the model gets stuck in a loop, almost always it is not a reasoning failure. It is a feedback failure. Build the testbed before you blame the prompt. A pytest case that reproduces the bug. A shell one-liner that exercises the endpoint. A snapshot test that goes red on the broken behavior. Whatever shape it takes in your domain. Once it exists, the model converges. Same iteration loop a competent human uses. Without it, you get hallucinated success.

For the SNES work I started with Mesen Lua scripts, which is the standard answer in the romhacking community. Useful for one-off probing, painful for regression testing. Lua is interpreted inside the emulator, the test harness lived outside the emulator, and the seam between them was where bugs hid. kintsuki replaced that whole arrangement with the emulator itself as a library called from pytest. One process, one language, one trace of execution. The Mesen scripts taught me what to want. Building kintsuki was admitting that the standard tool had hit its ceiling.

TDD plays well with the model for the same reason. The red test is the bar. “Done” is defined externally instead of by whatever sounds done.

Time to serve

The number I find useful for measuring whether the tool is moving anything is time to serve³. From the moment a problem is articulated to the moment the fix is in production, end to end. Not lines of code, not commits per day, not tokens consumed.

It captures the whole pipeline in one denominator. The parts the model speeds up (typing, boilerplate, first cuts) and the parts it does not (deciding what to build, the data model, the testbed, the diff review, CI, production). If the tool is genuinely net positive, the number drops. If the model is producing more code while the rest of the pipeline absorbs the cost, the number stays flat. The metric does not flatter the tool. It measures the wielder using the tool.

Worth pairing with the token economics piece from the public post. Time to serve is the team metric today. Cost per shipped feature is the same shape once tokens stop being subsidized.

What I do not delegate

This is the 道 part⁴. The places where I refuse to hand the wheel over, regardless of how confident the model sounds.

Architecture choices the model makes silently. I have shipped diffs where the model swapped a lifecycle (background task vs request-bound, lazy vs eager init, sync vs async at a boundary) without flagging the consequence. The change was not wrong-looking. It was just a different decision than the one I asked for, embedded in code that read as ordinary. That kind of thing slips past self-review. Human PR review by someone outside the loop with the model is still load-bearing.

Platform debugging. The general-purpose memory showed the rolling inventory animating correctly while the video memory stayed wrong: the data was right, the upload to the screen was not. The hypothesis (the per-scanline DMA was pointing at the wrong background layer during the only window each frame when the picture chip lets you write to it) was not in any prompt. The model coded the patch once I gave it the algorithm. It could not have formed the algorithm from a screenshot.

Data model and naming. The model defaults to plausible names that collide in app code, or to over-typing every parameter, or to stringly-typing everything. The boundary calls are mine. So is deciding when a function should not exist at all.

The model writes code. Choosing what should exist, what it is called, what invariants it preserves, that is still the job.

Getting oversmarted

The way the model wins against me, when it does, is not by being cleverer. It is by me losing track of what got built. Five tool calls deep, a refactor I did not quite ask for slipped in alongside the fix I did ask for, tests pass, diff looks reasonable. Sign off and the surprise lands a week later.

Countermove is the same rule I use on myself. Faut faire qui marche avant que c’est beau. Works for the model too. Let it go from A to B with the ugly first cut. Don’t pre-optimize the route, don’t stop it for a paint job halfway through. Then read the diff with eyes on. If on the road I see something starting to jeopardize the destination, that is the signal: I asked for something not clear enough. The drift is feedback on my prompt, not on the model.

The dog metaphor is the cleanest. Go fetch. The dog will fetch. If you said go fetch and pointed vaguely, the dog comes back with a stick when you wanted the ball. Not the dog’s fault. Throw better, or accept whatever comes back.

Discipline is two things. Keep the destination visible to yourself the whole time, so you notice when the path bends. And reread the diff. The model is fast enough that the only person who can lose track of what got built is you.

Ask Claude to review Claude

I regularly ask the model to look at its own work with fresh eyes. It catches dead branches, redundant guards, mocks that should be fixtures, signatures that drifted from the call sites.

No costume preamble, no “you are now a senior reviewer” framing. I am not a jeu de rôle grandeur nature guy. Played RPGs as video games, plenty, but I do not run my prompts like sessions at a tabletop. Why would I. In the same week I am a programmer, a CTO, a romhacker, a JS/TS dev, a reviewer, an ops guy. The hat changes with the task, no costume needed. Claude is the same. Ask it to look again, it does.

The act of generating and the act of judging are not the same operation, and forcing the second pass cheaply pulls out the obvious mistakes before they reach a human reviewer. Not a substitute for that reviewer. A filter that respects their time.

Same workflow as with a coworker

Working with the model is not that different from working with a teammate. I look for friction, usually starting with my own pain, and I build something to smooth it. My job has been building tools for developers for a long time, and it turns out Claude hits the same walls a human dev hits: bad imports, missing types, no testbed, no probe into the running system. Solve those for the human and the model gets the fix for free.

The LSP sidecar from earlier is the cleanest example. Built it because I was losing time to the model not seeing installed packages, but the same sidecar makes any human on the team more productive in the same repos. kintsuki is the same shape: deterministic snapshots so I could reason about HDMA, and the model uses them too.

You cannot build high on crappy foundations. That has been true with humans for decades and the model does not change it. If anything it makes the foundations more visible, because the model is brutal at exposing the soft spots in your dev loop.

Hard position to defend in rooms where AI is supposed to change the world. I see it as a tool. If typing on a keyboard were the programmer’s job we would already have been replaced by typists, who can produce an order of magnitude more words per minute than any of us. We were not. The keyboard was never the bottleneck. It is not the bottleneck now either.

The 道 has been written down for thirty years

The instincts in this post are not new. Two books in particular keep mapping onto what I do with the model.

Growing Object-Oriented Software, Guided by Tests⁵ is the testbed argument with the receipts. Freeman and Pryce wrote it for human teams trying to keep design honest as a system grows. Same instincts hold when the author is a model: start from a failing test, let the test shape the interface, refactor under green. Tests as the design surface, not just the safety net.

The Pragmatic Programmer⁶ is the other. Hunt and Thomas have a chapter called Programming by Coincidence that names the failure mode I push back on hardest with the model: code that works without the author understanding why. Their tracer bullets idea is the same instinct as faut faire qui marche avant que c’est beau, fire something end to end first, see where it lands, adjust aim.

Both books predate the model by decades and apply to it without modification. The 道 was written down a long time ago. The model is the newest student in the dojo, not the founder of a new school. If a tool reframes the discipline so completely that the canon stops applying, the canon was wrong. So far, the canon is fine.

What I tell people who ask

When the model and I disagree, I am right more often than not, and the times I am wrong it is because I delegated something I should have owned. That is the calibration. Not trust the model, not distrust the model. Trust your own ability to recognize when the output is wrong, and treat the model’s confidence as decoration.

プログラマ道¹ is the same job it has always been. The tools changed. The discipline did not.

Programmer plus 道, the way or path. Same suffix as 柔道 (jūdō, the gentle way), 剣道 (kendō, the way of the sword), 茶道 (sadō, the way of tea). Practice you keep refining, not a credential you finish. ↩︎ ↩︎
SNES vocabulary used in this section, glossed once: CPU is the 65c816 main processor. PPU is the picture processing unit, a fixed-function chip that composes the picture from sprites and tile layers (not a GPU in the modern sense, no shaders, no general compute). WRAM is general-purpose memory the CPU writes to. VRAM is the PPU’s separate memory, only writable through narrow windows each frame. HDMA is a per-scanline DMA mechanism that lets you change PPU registers during display. NMI is the non-maskable interrupt that fires once per frame, the standard window for VRAM updates. BG3 is one of the four background layers the PPU composes. The point of the article does not depend on the details, but the jargon is real and so are the bugs it produces. ↩︎
Not sure who coined “time to serve”. Closest canonical sibling is DORA’s lead time for changes (Forsgren, Humble, Kim, Accelerate). I have been using time to serve internally without a clean attribution. If you know the source, tell me and I will update this footnote. ↩︎
道 on its own. The way. What stays yours after the tooling moves under your feet. ↩︎
Steve Freeman and Nat Pryce, Growing Object-Oriented Software, Guided by Tests, Addison-Wesley 2009. The “guided by tests” half is the part that ages best. Tests as the design surface, not just the safety net. ↩︎
Andy Hunt and Dave Thomas, The Pragmatic Programmer, Addison-Wesley 1999, 20th anniversary edition 2019. Programming by Coincidence and Tracer Bullets are the two chapters that map cleanest onto working with an LLM. ↩︎

The 100x engineer, and the unit nobody defines

Mon, 04 May 2026 01:03:42 +0200

The 100x engineer is back in the discourse. This time the pitch is that an engineer with the right AI tooling can do the work of a hundred. Sometimes the number is 10x, sometimes it is 100x, occasionally someone goes wild and writes 1000x. The number is always round and the unit is always missing.

What are we counting?

If a 100x engineer using Claude can do 365 days of work in 3.65 days, whose 365 days? A staff engineer shipping platform infrastructure? A junior wiring up CRUD endpoints? My grandmother on a good day? The denominator is doing all the work in that sentence and nobody ever writes it down.

The RPG item problem

There is a cleaner way to frame this. The “100x engineer” pitch sells AI tooling as a multiplier, the way RPG loot sells itself as *100 to all stats. The math of a multiplier compounds the floor: if your stats are 0.01, a *100 item brings you to 1. If your stats are 8, the same item flings you to 800. The marketing wants you to imagine the second case. The buyer is usually the first.

What AI tooling actually behaves like is a flat-add item. Roughly +0.99 to all stats. The kid at 0.01 puts it on and lands exactly at 1.00. The senior at 8 puts it on and lands at 8.99. Same buff, applied to whoever wears it.

Except a flat-add understates one thing: the senior extracts more from the same item. They know what to ask for, recognise when the output is wrong, slot the working bits into a system the kid does not have yet. The base buff is the same; what the wearer does with it scales with their level. Closer to +0.99 base, with a wielding bonus. Still not a multiplier on stats. The buff is on judgment, not on raw capability.

When someone tells me a terminal and Claude Code made them 100x, the question is not snark. What was the floor? If the AI item moved you from “could not ship a working CRUD app in a week” to “can ship one in an afternoon,” that is the +0.99 doing exactly what +0.99 does. Not a 100x engineer. A +0.99 wearer who started at 0.01.

The 100x story flatters more than “I was very junior and my tools got better.” Same story. Tells you nothing about the ceiling, only about the floor the speaker was sitting on.

A multiplier without a base is a vibe, not a measurement.

A concrete counter-example

Tunes is the music app and backend I have been building on weekends, a Go service plus SwiftUI clients on iOS, macOS, and tvOS, replacing iTunes Match for my own library. AI has been in the loop on almost every commit. I am close to a year of weekend development and the thing is not shipped.

If 100x were real, Tunes should have been done sometime around last February. It was not. The reason is not that I was idle. Building a real product is mostly the parts AI does not compress: deciding the data model, debugging a SwiftUI animation that only misbehaves on tvOS, understanding why the Go service drops connections on a specific HLS edge case, designing the play queue semantics, redesigning them after I lived with the first version for a month.

The model writes the function. I still have to know which function to ask for. Otherwise I get a bubble sort hidden inside a request handler that takes 45 seconds for no good reason, and the model will defend it with a straight face until I push back.

AI maggots will read this and tell me it is a SKILL.md issue, that the model would have known better with the right rules file pinned to the context. Sure. It is a skill issue. Just not the kind they mean. The wielding bonus shows up here.

The token economics nobody is pricing in

The unit cost of all this is currently subsidised. Frontier model access is sold below what it costs to serve, on top of training spend that has not been amortised. Investors are paying so we get to play with cheap intelligence. That ends. It always ends.

When the price of a token starts to reflect what a token actually costs, the productivity story gets re-litigated against a less generous backdrop. The engineer who burned three thousand tokens flailing at the bubble sort and the engineer who spotted it in twenty seconds are doing very different work. Today they pay roughly the same. Tomorrow they will not. The boring metrics will show up: cost per shipped feature, cost per resolved bug, cost per regression caught before production. Management likes impressive marketing metrics. Field reality is usually a bit different.

The engineer in that scenario is in the same position as the buyer of a smart speaker whose vendor pivots, or the company shipping on top of an open source project that changes direction. Different floor, same shape: you built on something you do not own. The buyer floor and the vendor floor live in other posts. The engineer floor is here.

Real gain, yes. Not 100x, not 10x averaged across a real week. A +0.99 with a wielding bonus, applied to whoever wears the item. Sentence does not get retweeted, which is why it does not get shipped.

Anyone using the 100x number is selling something or measuring something silly. If they are selling, ask what the unit is and watch them squirm. If they are measuring, ask them to print the denominator on the next slide.

25 years to a complete machine

Mon, 04 May 2026 00:08:13 +0200

Since around the year 2000 I have been piling features into a French Final Fantasy IV translation patch. Variable-width fonts in dialog, in battle messages, in menu descriptions. Redrawn battle, load/save, main, and two-column magic menus. An expanded inventory of 0x48 slots rendered through a paged window instead of the static grid the original engine assumed. Most of these were features the SNES ROM hacking scene of the era said could not be done on real hardware. Other hackers eventually pulled off equivalents on other titles, fair enough; modifying a compiled ROM is more convoluted than editing source code, but not impossible. Twenty-five years on, “impossible” reads as a fair approximation of how hard, not of whether.

The bugs that come with hard features are also recent, introduced by my own patches. The field-menu item list eventually crashed: open inventory, swap items around, and after enough operations the stack pointer drifted until the next interrupt landed on a brk¹ instruction and the emulator stopped cold.

The cause was one wrong return mnemonic². swap_redraw_trampoline lives in bank $01³. Its real body, swap_redraw_hook_impl, lives in bank $21. The field menu calls the trampoline with jsr.l⁴, which pushes a 24-bit return. The bank-$21 body returns via rtl, which is correct. The trampoline itself ended in rts, which is not. One byte of the long-return address stayed on the stack every time an item was swapped. Run the swap path enough times and the underflow cascaded, the next NMI⁵ hit garbage, and the BRK trap I had wired into the vector table⁶ caught the offending PC.

The fix is one mnemonic. Getting to the point where I could see the wrong mnemonic took infrastructure.

Why a rolling inventory at all

The rolling inventory was not strictly necessary. Two columns of items were getting tight on screen as the inventory grew, and I knew FF6 had solved a similar layout problem more elegantly with a paged renderer. FF4 and FF6 share engine roots, which made porting the technique tractable instead of ground-up work. So I lifted the approach.

The memory side effect is real but secondary. The rolling render only keeps the visible slice in RAM and VRAM⁷, paging new ones in as the cursor scrolls. 0x48 slots at roughly 0x40 bytes of tile data each would have cost 0x48 * 0x40 = 0x1200 bytes per buffer in a static layout. The rolling layout drops that to one window’s worth. If saving memory had been the only goal there were cheaper fixes than this one.

The bug above was the price of pulling FF6 ergonomics into FF4: the trampoline that moves tile data between RAM and VRAM forgot to return with the right mnemonic, leaked one byte per swap, and the stack caught up with the player when they were having too much fun rearranging.

What made the bug reachable

For most of the 25 years I have been doing this, I would not have caught it.

The original ff4 source, like every long-running ROM hack, was a single textual .include chain compiled in one pass. Every label was global. There were no module boundaries. There was no way to ask the assembler “where does this symbol come from” except by grepping the file system. The trampoline-with-the-wrong-rts looked exactly like the trampoline-with-the-right-rtl two screens up. Lost in the noise.

The ff4 source recently finished its migration to a816’s .import system. Each module declares its own externs, compiles to its own .o, links through relocations. The LSP earns its keep on top of that. When you suspect swap_redraw_trampoline ends in rts where it should end in rtl, you do not grep for “swap_redraw” across hundreds of files. You click. The symbol resolves through the import graph, the trampoline opens in the right module, and the wrong mnemonic is on the screen.

Without modules the bug was unreachable. Without the BRK trap in the vector table the symptom was “crashes sometimes” instead of a PC value. The rest of the chain (LSP for navigation, kintsuki for deterministic reproduction) only mattered once those two existed.

Why these tools exist

I have been hacking on Final Fantasy IV, on and off, since around the year 2000. The earliest public trace is the ff4fr repo from 2011, itself a snapshot of older work. The reason the project keeps surviving long gaps is simple: it interests me. I come back, I poke at it for a few weekends, I drift, I come back. A hobby that happened to last.

None of the tools were built as long-game preparation for the rolling inventory bug.

a816 replaced x816.exe, the DOS assembler the FF4 scene had used for years. x816 is painful on Windows, dead on Linux and macOS. Building the ROM on GitHub Actions through dosbox was not going to happen. asar exists and is a fine alternative (and may not have been around when I started; I do not remember the timeline clearly). I went my own way because I wanted tight integration with the ff4 source: modules I controlled, a formatter I could shape, an LSP that understood bank annotations and docstring conventions, an object format that handled multi-region modules cleanly. Easier to build that end-to-end than to lobby someone else’s project for ergonomics specific to one ROM hack. The GitHub Actions side became its own win. Messing with workflows on a personal repo translated directly into knowledge I now apply at work.

kintsuki exists because I like automated tests. Running the ROM deterministically against pinned inputs needs primitives ares does not expose by default: mid-frame yield, PC override across scheduler entries, stp and wai flag preservation through external state-set. kintsuki adds the shim around the ares core and ships it as a Python wheel.

The honest reason underneath both: knowing enough to be dangerous has been the constant of my career. At fifteen, in 2000, I was dangerous with a hex editor and someone else’s assembler. That was already enough to start a translation patch. The version writing now is dangerous with a homemade assembler, an LSP, an emulator harness wrapped as a Python wheel, and a multi-region object format. The disposition has not moved. The capability has. Every year of doing software for a living added new ways of being dangerous, and the FF4 patch is the one personal project that stuck around long enough to absorb all of them. The tools and the patch grew up together. The features the fifteen-year-old wanted are now sitting on infrastructure he could not have built, but would absolutely have used.

The thing actually pacing the work right now is a self-imposed release date: December 26, 2026, for a packaged build of the patch. That deadline turned the loose interest into a backlog. The deadline is recent. The hobby is old.

Seven and a bit months until the release date. The patch will ship with bugs that came after it on top of an assembler and a harness that came before. The ROM hack will still have open issues, untranslated NPCs, and design decisions I will second-guess. None of that bothers me. It is the project I keep coming back to because it keeps being interesting, and right now the interesting thing is finishing it.

Not everything I write is meant to stick it up to the man. This one is not. Hobby code, on a console nobody ships for, against a deadline I made up because I felt like one. No thesis. A release on December 26.

brk is a one-byte instruction that triggers a software interrupt. The CPU pushes its state and jumps to a fixed handler address. Useful as a trap when that handler is wired to halt the emulator on an unexpected fall-through. ↩︎
A mnemonic is the short text name for a CPU instruction (rts, jsr, lda). The assembler turns each one into the matching machine-code byte. ↩︎
The 65C816 addresses 24-bit memory split into 256 banks of 64KB. $01:FF34 reads “address $FF34 in bank $01”. Code in different banks is reached with long-call instructions instead of short ones. ↩︎
jsr.l is a long jump to subroutine: it pushes a 24-bit return address (bank + 16-bit) before jumping. The matching return is rtl. The short pair is jsr + rts (16-bit return). Mismatching them leaks or eats bytes off the stack. ↩︎
Non-maskable interrupt. On the SNES it fires once per video frame, in the brief moment when the screen is between drawing two frames, and runs whatever handler the CPU is told to run for it. ↩︎
The vector table is a fixed block at $00FFE0-$00FFFF listing handler addresses for reset, NMI, BRK, and others. Wiring a BRK trap means setting the BRK handler to halt the emulator and record where the bad instruction came from, so an unexpected brk produces a debuggable crash instead of silent corruption. ↩︎
The SNES CPU’s main RAM is 128KB. VRAM is the graphics chip’s video RAM (64KB), holding the tile data and the screen layout. Anything visible has to be in VRAM. Tile data is usually staged in main RAM and copied across in bulk between two drawn frames. ↩︎

The complete machine, fading

Sun, 03 May 2026 23:14:38 +0200

My iPod 5.5g is old enough to drink alcohol in any country that has a drinking age. I bought it used, swapped the battery and the disk for an SD card mod, and it still does the one thing it was sold to do: play music. No firmware update has ever taken a feature away. No company has decided the device is too old to deserve syncing. Apple, to their credit, still ships iTunes and Music with iPod sync on macOS. But even the day they stop, the iPod will keep playing whatever is already on the disk. The original purpose has not eroded.

That is becoming rare.

Devices die from the network now

The hardware on my desk has not gotten worse. CPUs are faster, displays are sharper, batteries hold up longer than they used to. And yet the average lifespan of a “smart” thing I buy in 2026 feels shorter than what I was buying ten years ago. The thing breaks first from the network, not from the silicon.

The pattern is familiar by now:

The companion app is removed from the store, or rewritten in a way that drops support for older models.
The TLS certificate the device was pinned to expires, and the firmware that would let it accept a new one was never updated.
The cloud API the device talks to is versioned out of existence. A v1 endpoint becomes v2, and v1 is shut down because keeping it alive is unbillable maintenance.

A speaker, a thermostat, a camera, a “smart” anything. None of these failure modes have anything to do with whether the device can still do its job. They have to do with whether anyone, somewhere else, is still willing to keep the other end of the wire alive.

AI accelerated the pattern without being its cause. Every product launch in the last two years has the same slide: “Powered by AI”, which usually means an API call to a cloud LLM the vendor pays for, the user does not control, and the headline feature does not work without. Cut the connection and the feature is gone. Often the basic feature is gone too, because the product team built the whole flow assuming the cloud would always answer.

The same dynamic plays out at every floor. The vendor depends on the open source it ships on top of, and gets squeezed when upstream changes direction (the supply-chain story). The engineer using AI tools depends on a token economy that is currently subsidised, and will get squeezed when investors want their money back (the 100x story). The buyer of the smart speaker depends on the vendor staying alive long enough to keep the cloud running. Three floors, one shape. Dependency on something you do not own is the constant. The label on the box changes.

A worse mainframe, with prettier hardware

The easy version of this argument is “we are going back to the 70s mainframe.” It misses what got worse. The 70s deal was structurally better than what is shipping now in three specific places.

The 70s mainframe sat in your building. You owned it, or your employer did, and the cable to the terminal ran through the walls of a building you had keys to. Today’s mainframe is offshore, owned by a company whose name you did not choose, reachable only over a TLS pipe whose certificates you do not control. The “central operator” is not down the hall. It is in someone else’s jurisdiction.

The 70s terminal was honestly dumb. A keyboard, a screen, a serial line, no pretence. You knew what you had. Today’s terminal is shaped like a real computer and behaves like one until you cut its connection. The dishonesty of selling a thin-client as a personal device is the part that did not exist before.

The 70s mainframe came with a service contract you signed. SLAs, support, a vendor who could be sued. Today’s “service” is a TOS the vendor can change weekly, with a clause letting them brick a device they do not like, and you signed it without reading because the alternative was that the box you paid for did not turn on.

Three ways the deal got worse: location, honesty, accountability. The mainframe analogy understates the regression.

What “complete” used to mean

You thought I was going to channel the Woz. Trying to be a bit more original.

The Casio F-91W has been in continuous production since 1989. It tells the time, runs a stopwatch and an alarm, lights up when you press the button, and lasts about seven years on a battery. That is the entire feature set. There is no companion app. The watch will keep doing exactly what it was sold to do until the LCD fails or the case cracks, on a part nobody is going to deprecate, against a service nobody operates.

A complete machine, in the sense I care about. Not “open” in the GPL sense, not “user-modifiable” in the kit-computer sense. Honest about what it is. The design refused to add things. Forty years later the refusal looks like prescience.

The iPod 5.5g is the same kind of object on a shorter timeline. Music on the disk, codec in the firmware, click wheel and headphone jack with no server in the loop. If iTunes vanished tomorrow, syncing becomes annoying, but the device still plays. Communities are already keeping that path open: Rockbox, libimobiledevice, third-party sync tools. The world where you sync an iPod without Apple exists because the device is honest about what it does.

A streaming-only speaker is the inverse. The codecs may live locally, but the catalogue does not, the auth does not, the discovery protocol does not. None of those parts belong to the buyer. The day any one of them goes away, the speaker stops being a speaker.

The Casio and the iPod 5.5g are the floor the rest of the catalogue should be measured against. Most of what is shipping in 2026 fails the comparison and sits closer to the streaming speaker.

Linux is real, and not enough

“Use Linux” is a real answer for laptops, servers, single-board computers, and a handful of phones. It is not an answer for the smart oven, the robot vacuum, the doorbell, or the infotainment system in your car. Most appliances ship as the vendor decided or they do not ship at all. The escape hatch exists. The long tail of household objects is outside it.

Practical advice for non-technical buyers: pick devices whose core function does not depend on the cloud. Pick the dumb version when the dumb version still exists. “Smart” in 2026 mostly means “rented.”

What I actually do

I work on AI tooling and run cloud services for a living. The tension is real and I am inside it.

On the personal side I lean local-first. Higgins runs a 7B model on my laptop with a local SQLite. If the cloud disappears, the assistant still answers. Costs capability, gains independence. There is also a side benefit nobody prints on the box: running the thing yourself teaches you a thing or two about how it works. The cloud version abstracts away the parts you would otherwise be forced to understand. I value both more than I used to.

The iPod will keep playing. The list of products I could say the same about in 2045 is shrinking every year.

Higgins: a neurosymbolic menubar buddy

Mon, 20 Apr 2026 10:00:00 +0200

Higgins is a local assistant that lives in the menubar. A Qwen 2.5 7B model runs on-device via MLX, a SQLite triple store remembers things across sessions, and a small collection of tools — AppleScript runners, EventKit bridges, gh CLI wrappers — lets him actually do work on the Mac. No cloud, no subscription, no data leaving the machine.

The name is a nod to Magnum P.I.’s reserved English major-domo. The vibe was aspirational; most of the engineering effort went into making sure he doesn’t cheerfully hallucinate your dentist appointment into 2023.

Why not just a chat wrapper

The starting pitch was neuro-symbolic: “LLMs are great at vibes and terrible at strict logic; symbolic systems are the opposite; put a logic engine inside a neural one.” Nice in a blog post, vague as a project.

What it actually means in code: the neural side (Qwen) handles language, intent, paraphrasing. The symbolic side (SQLite facts table, typed tool schemas, EventKit queries) handles everything where being wrong is worse than being silent — dates, names, calendar events, PR numbers. The LLM never gets to compute a date from priors; it forwards your words to a parser. The LLM never claims to remember your dog’s name; the fact comes from a SQL row, or it doesn’t answer.

That split is the whole project. Everything else is plumbing.

The Qwen brain

Qwen 2.5 7B Instruct, 4-bit quantized, ~4.5 GB on disk. Loaded via mlx-swift-lm with progress reported through a NSKeyValueObservation on the download’s Progress object — since swift-huggingface fires the handler once then mutates in place, not on every byte.

Picking Qwen over Gemma was almost incidental. Gemma 3 4B worked but wouldn’t accept a system role, forcing a hacky primer-pair trick. Gemma 4 26B-a4b is MoE, which mlx-swift-lm’s Gemma4Model doesn’t implement yet. Qwen accepts system prompts cleanly and, crucially, was trained to emit native tool calls — which matters a lot more than benchmark scores.

The old path was prompt-engineered: TOOL_CALL toolname {...} in the output, parsed with regex. Fragile. Qwen wants to emit something like:

<tool_call>
{"name": "reminder_add", "arguments": {"text": "…", "datetime": "…"}}
</tool_call>

mlx-swift-lm surfaces this as .toolCall(ToolCall) events in the generation stream, if you pass structured tool specs into UserInput(chat:, tools:). Once wired up, the model stops inventing its own format and starts using the one it was RL’d on.

Generation loop excerpt

for await gen in try MLXLMCommon.generate(input: input, parameters: params, context: context) {
 switch gen {
 case .chunk(let s): output += s
 case .toolCall(let c): call = c
 case .info: break
 }
}

The text parser stuck around as a fallback for edge cases.

The symbolic side

A SQLite file at ~/Library/Application Support/Symbolic/memory.db with two tables:

CREATE TABLE episodes(
 id INTEGER PRIMARY KEY,
 user_msg TEXT, assistant_reply TEXT,
 signal INTEGER, ts REAL
);
CREATE TABLE facts(
 id INTEGER PRIMARY KEY,
 subject TEXT, predicate TEXT, object TEXT,
 confidence REAL, last_seen REAL,
 UNIQUE(subject, predicate, object)
);

Episodes are the raw log: every chat turn lands here. Facts are the distilled, queryable form — subject/predicate/object triples with a confidence score. The interesting work happens in the pipe between the two.

Sleep

The consolidation metaphor is cribbed directly from human memory: episodic hippocampal traces don’t become semantic knowledge until you sleep. Higgins does the same. After 15 minutes of chat silence, or when you tap the 🌙 button, he runs a pass:

NREM: the model reads the day’s recent episodes and emits one JSON object per line — extracted stable facts about you, your world, your preferences. Ephemeral conversational filler (greetings, acknowledgments, weather chat) is skipped.
Synaptic downscaling: every fact’s confidence is multiplied by 0.95. Unused memories decay. Recalled ones get re-bumped above 1.0 and capped.
Purge: facts below 0.05 confidence are dropped.
Dream log: a human-readable markdown summary lands in ~/Library/Application Support/Symbolic/dreams/YYYY-MM-DD.md.

Sleeper entrypoint

func nightCycle(episodeLimit: Int = 100) async throws -> Report {
 let episodes = try await store.recentEpisodes(limit: episodeLimit)
 let facts = await extractFacts(from: episodes)
 for f in facts {
 try await store.addFact(subject: f.subject, predicate: f.predicate, object: f.object)
 }
 try await store.decayFacts(multiplier: 0.95)
 let purged = try await store.purgeWeakFacts(threshold: 0.05)
 // ...
}

There’s no remember command to learn. Facts accumulate while you use the thing.

Per-turn recall: closing the loop

Sleep would be useless if the facts just piled up. Before each generation, the user’s query is keyword-matched against the facts table (top 5 hits, LIKE on subject/predicate/object). Matching facts are injected into that turn’s system context as a bracketed prefix:

[Known facts from memory:
- dog / name / Rex (confidence 0.95)
- manz / employer / Woosmap (confidence 0.87)
]
what's my dog's name?

History stays clean — the decoration only exists for the single inference call. No tool call needed for the model to answer “Rex”. The symbolic substrate grounds the neural output on the way past.

At low scale (hundreds of facts) LIKE matching is good enough. Embedding-backed similarity search is the upgrade when it isn’t.

Tools

The tool layer is where the project actually earns its keep. Three surfaces:

AppleScript library. Curated scripts live in ~/Library/Application Support/Symbolic/scripts/, each with frontmatter metadata (-- name:, -- description:, -- args:). Seed scripts ship with the app; the user can edit or add more. The model calls them by name — calendar_today, note_quick, etc. — as if they were first-class tools.

EventKit bridge. AppleScript against Calendar.app is infamously slow — iterating every event on every calendar through the bridge takes 30+ seconds for a busy calendar. EventKit queries the same data in milliseconds. The catch: it needs NSCalendarsFullAccessUsageDescription in a real Info.plist. The SPM executable embeds one via a linker trick:

Package.swift

linkerSettings: [
 .unsafeFlags([
 "-Xlinker", "-sectcreate",
 "-Xlinker", "__TEXT",
 "-Xlinker", "__info_plist",
 "-Xlinker", "Resources/Info.plist",
 ]),
]

-sectcreate __TEXT __info_plist injects the plist into a section TCC reads when deciding whether to prompt for permission. Works without wrapping the binary in a .app bundle.

gh CLI wrappers. gh_my_prs, gh_review_queue, gh_notifications, gh_recent_merged. Subprocess calls to the user’s existing gh session — no OAuth dance, no token storage. A ProcessRunner actor handles timeouts and respects Task.isCancelled so a stuck subprocess doesn’t freeze the UI when you hit the red stop button.

Tool results render as folded cards with a wrench icon, tool name, and a one-line summary. Click to expand. Errors auto-expand. #123 · PR title · owner/repo links are real anchors thanks to MarkdownUI on the assistant-side rendering.

Dates, or: why the model doesn’t do arithmetic

7B models are bad at date math. “Remind me tomorrow at 8:15” reliably produced ISO strings from 2023, 2024, anywhere but actually tomorrow.

The fix is to not let the model do arithmetic at all. Any tool argument named datetime, when, due_date, etc. gets intercepted in Swift before dispatch. A tiny table handles English and French relative words (tomorrow, today, demain, hier) — NSDataDetector isn’t locale-parameterizable and picks up the system locale, which fails on English keywords under a French macOS. Anything the table doesn’t catch falls through to NSDataDetector, which anchors relative phrasing against Date().

Critically, the user’s raw message is the source of truth, not the model’s output. If you typed “tomorrow”, “tomorrow” is what gets parsed — even if the model hallucinates an ISO. The model’s job is intent extraction, not date reasoning.

The AppleScript Run button

When the model writes code in an assistant reply, the code block renders with a language tag and a copy button. When the language is applescript, a Run button appears:

CodeBlock with conditional run action

MarkdownUI.Markdown(turn.text)
 .markdownBlockStyle(\.codeBlock) { config in
 CodeBlock(
 code: config.content,
 language: config.language ?? "",
 onRun: runAction(for: config.language ?? "", code: config.content)
 )
 }

The action shells out to osascript -e <script>, captures stdout, appends the result as a tool turn. The user stays in charge — nothing runs without the click. Safer than giving the model unrestricted automation tools, more flexible than the curated script library.

What grew out of it

The original plan was to fine-tune a custom voice — a “genius caveman” Grug, all short fragments. Hours of LoRA experiments on Gemma 2 2B, Gemma 3 4B, Gemma 4 E4B later, the honest conclusion: voice tuning on small models produces caricature or collapse, and nobody cares about voice when the tools don’t work. The pivot was embracing stock Qwen and pouring the effort into the symbolic substrate instead.

What’s there now:

On-device Qwen 2.5 7B with native tool calling, loading with proper progress feedback
Conversation persistence as JSONL (machine) + Markdown sidecar (human), auto-saved per turn, restored on launch if recent
Tool calls through Qwen’s native format, fallback text parser, per-tool timeouts, cancel button, pending spinner
Sleep/nap consolidation turning episodes into facts, confidence decay, dream logs
Per-turn recall grounding generation on stored facts without touching history
EventKit for calendar/reminders in milliseconds, gh CLI for GitHub, NSDataDetector for natural-language dates
Right-click menu: new conversation, sleep now, open dreams/conversations/memory in Finder
Markdown-rendered assistant replies via MarkdownUI, plain monospaced tool output
AppleScript Run button on any applescript code block in a reply

The fine-tune loop (REM-derived training candidates → periodic LoRA refresh of stable voice and high-confidence facts) is designed but deferred. Retrieval has a faster iteration loop; weights-baking earns its keep when retrieval stops being enough. Given Higgins has had exactly zero users for three days, that threshold is a while off.

Meanwhile he tells me what’s due tomorrow, doesn’t make up dates, remembers the cat’s name is Charlie, and opens PR review queues when I ask. Good enough for now.

Freddie part 2: from tiles to map

Tue, 24 Mar 2026 12:00:00 +0100

In part 1 I covered the hard problems: COG multi-resolution, SDF raster smoothing, and Barbapapa topology-preserving Laplacian smoothing. Freddie could generate good-looking 10m land cover vector tiles. This post is about what happened next: killing the algorithm I spent a week perfecting, solving a surprisingly dumb storage problem, and wiring the tiles all the way through to the map SDK.

Barbapapa is dead

I killed Barbapapa a few weeks after shipping it.

It stung a little. Seven revisions, the hole vertex bug that ate two days, the Jacobi-style simultaneous updates, the deterministic iteration fix. Elegant stuff. But after staring at enough tiles side by side, the conclusion was unavoidable: Barbapapa was lying.

Land cover at 10m resolution doesn’t have smooth boundaries. A forest edge is where a satellite sensor switched from “tree” to “grass” at some pixel boundary. It’s inherently jagged. Barbapapa was manufacturing gentle curves that were never in the data: aesthetically pleasing, but dishonest. The tiles looked better in the way a retouched photo looks better. Not more accurate, just smoother.

Meanwhile, the chain-based Douglas-Peucker simplifier I’d built alongside it was doing more useful work with less ceremony.

Edge DP

Edge DP operates on the same half-edge topology that Barbapapa used, but the philosophy is opposite. Instead of moving vertices closer to their neighbors to manufacture curves, it removes the vertices that don’t carry information. Less romantic, more honest.

The core idea: find the topologically significant vertices (junctions where three or more land cover classes meet) and treat everything between them as a chain. Each chain can be simplified independently without breaking the topology.

Five phases:

1. Find junctions. Walk every vertex and count how many distinct classes touch it. Three or more classes makes it a junction, an anchor in the topology skeleton. Tile boundary vertices are also pinned.

2. Extract chains. Walk each polygon ring, breaking at junctions and class transitions. Two adjacent faces sharing a boundary reference the same chain object. Simplify it once, both faces see the result. This is the same principle that made Barbapapa work (shared topology) but applied to vertex removal instead of vertex movement.

3. Douglas-Peucker per chain. The textbook recursive algorithm: find the vertex farthest from the line between endpoints, keep it if it exceeds the threshold, recurse on both sides. Nothing clever here. The cleverness is in applying it at chain granularity instead of per-polygon, where it would tear shared boundaries apart.

4. Reconcile. If any chain needs a vertex, it stays. Prevents holes at multi-chain junctions where one chain would remove a vertex another one depends on.

5. Rebuild twins. Remove the marked vertices from all rings and relink half-edge twin pointers so the topology stays consistent.

The result: aggressive vertex reduction with guaranteed topological consistency. No gaps, no overlaps, no invented curves. What you see is what the sensor saw, with fewer vertices saying it.

Z14Factor

One subtlety at high zoom: SDF upscaling leaves residual staircase artifacts. The Gaussian blur approximates class boundaries but doesn’t eliminate the pixel grid entirely. A Z14Factor of 0.6 multiplies the DP threshold at z14 and above, a less aggressive simplification that preserves more of the SDF-smoothed detail. Base threshold of 2.0 becomes 1.2 at z14, keeping vertices that would otherwise be culled. The SDF already did the heavy lifting on boundary shape. DP just needs to not undo that work. Small tweak, noticeable difference.

What got deleted

This was the satisfying part. With Edge DP handling everything alone, I got to delete a lot of code:

Barbapapa: gone (~6,800 lines including tests and docs)
Visvalingam-Whyatt: gone (experimental, Edge DP outperformed it everywhere)
SimplifyFaces: gone (subsumed by Edge DP’s chain extraction)
Ten per-class viewer layers: collapsed into one fill layer with zoom-interpolated colors
Duplicated config across freddie and freddie-server: unified into a single ProcessingConfig struct

The pipeline description shrank to one line: SDF (blur=6.0, minZ=10) + mode filter 7×7 + edge DP (threshold=2.0).

Painter’s algorithm

Collapsing ten per-class layers into one fill layer immediately broke rendering. Enclosed faces could paint over their parents. A lake surrounded by forest would disappear under green because the forest polygon rendered last. Obvious in hindsight, invisible until you actually try it.

The fix: topological sorting before MVT encoding. For each face, ray-cast to check whether it’s enclosed by another face, then depth-first sort so parents paint first. A lake inside a forest inside a continent renders in the right order. Combined with placing the landcover layer right after the map background in the style, one MapLibre fill layer handles everything correctly. Fewer layers, less complexity, better result.

The 200GB problem

Freddie generates beautiful tiles. But beautiful tiles sitting on my laptop don’t help anyone. They need to end up in the actual map, served alongside the base tiles: roads, buildings, POIs, all the stuff Planetiler generates.

The obvious approach: merge the two mbtiles files offline into one. Base map (~80GB) + landcover (~60GB) = one file, ship it. I built freddie-merge for exactly this. Iterated over every tile in both files, combined the MVT layers, wrote the result. Clean, simple, done.

The merged output was over 200GB. I stared at the number for a while.

The reason is embarrassing. Planetiler stores identical tiles once through content-addressed deduplication. Vast stretches of ocean at low zoom are the same tile: one copy, millions of references. Merging landcover into every tile makes each one unique, so deduplication collapses entirely. 80 + 60 doesn’t equal 200, but 80 (heavily deduplicated) + 60 (heavily deduplicated) merged together becomes 200 (nothing deduplicated). I’d undone one of Planetiler’s best optimizations without realizing it.

Runtime MVT concatenation

The fix is almost too simple to be satisfying.

MVT layers are top-level protobuf fields. Two valid MVT byte streams concatenated together produce a valid MVT byte stream. No parsing, no reencoding, no schema reconciliation, just append bytes. I had to read the protobuf spec twice to convince myself this was actually legal.

So: don’t merge at build time. Keep both tile pyramids as separate mbtiles files and concatenate at read time:

Fetch base tile from mbtiles A
Fetch landcover tile from mbtiles B
Decompress both (gzip)
Concatenate, landcover first, so it paints behind everything
Recompress, serve

When there’s no landcover configured, the merge path is never entered. Zero overhead for the existing setup. Both files keep their original deduplication intact. And freddie-merge (the tool I built, debugged, optimized with ATTACH JOIN, rewrote to iterate the smaller set) joins the growing pile of deleted code.

Layer ordering matters: landcover sits right after the map background, before roads and buildings and labels. At high zoom where ESA WorldCover runs out of detail, the existing OSM landcover_wood and landcover_grass layers take over with a minzoom of 12.

Teaching the SDK about fill colors

The last piece of the puzzle was the rendering SDK. It already had a multiclass styling system for POIs: one MapLibre layer that renders different icons and text labels depending on the feature class. I naively assumed landcover would slot right in. Same system, different data, right?

No. POIs and landcover need fundamentally different MapLibre expression shapes, and the difference is annoyingly subtle.

A POI layer uses case at the root to pick an icon, with interpolate nested inside for zoom-dependent sizing:

["case",
 ["==", ["get", "class"], "park"], "park-icon",
 ["==", ["get", "class"], "school"], "school-icon",
 "default-icon"]

A landcover fill layer needs the structure inverted: interpolate at the root for smooth color transitions across zoom, with case nested inside each zoom stop to pick the per-class color:

["interpolate", ["linear"], ["zoom"],
 0, ["case",
 ["==", ["get", "class"], "wood"], "#1a3d1a",
 ["==", ["get", "class"], "grass"], "#2d5a1e",
 "transparent"],
 14, ["case",
 ["==", ["get", "class"], "wood"], "#2d6b2d",
 ["==", ["get", "class"], "grass"], "#4a8c3f",
 "transparent"]]

The inversion matters because MapLibre can’t interpolate between case results. You have to case-select within each interpolation stop. I spent an embarrassing amount of time trying the other way before reading the spec carefully enough to understand why.

A new ClassMetadata type carries zoom color stops instead of flat color strings, and the expression builder constructs the full interpolate+case tree from the metadata. The POI path stays untouched, just extracted into its own methods so the two flows don’t tangle. One of those changes where the diff is medium-sized but the head-scratching that preceded it was substantial.

Four repos, one pipeline

Stepping back, the whole story spans four repositories and a surprising amount of deleted code:

freddie generates 10m land cover vector tiles: SDF smoothing, Edge DP simplification, painter’s algorithm sorting, mbtiles output
data-pipeline spins up a beefy ARM instance, downloads ~100GB of ESA WorldCover source TIFFs from S3, runs freddie, uploads the result, self-destructs
maps serves tiles with runtime MVT concatenation: no offline merge, no deduplication loss, zero overhead when landcover isn’t configured
maps-js renders them with zoom-interpolated fill colors through the multiclass style system

From satellite raster to styled vector on screen. GeoTIFF in, mbtiles out, MVT over HTTP, MapLibre expressions in the style JSON. Each piece does one thing.

The part that surprised me most wasn’t any individual algorithm. It was how cleanly everything composed once each piece had well-defined inputs and outputs. Freddie doesn’t know about the map SDK. The map server doesn’t know about SDF smoothing. The SDK doesn’t know the tiles came from satellite imagery. They just agree on protobuf bytes and the rest takes care of itself.

Also: I wrote and then deleted an entire smoothing algorithm, an entire merge tool, and three simplification strategies. The codebase got smaller as the feature got bigger. That might be the most satisfying part of this whole project.

The most useless way to port a macOS app

Tue, 24 Mar 2026 09:30:00 +0100

I grew up fascinated by projects like GNUStep, Haiku, Etoile, Wine, and ReactOS. Engineering feats, all of them. They reverse-engineer or reimplement entire operating system APIs so that software written for one platform can run on another. And they almost always end up in the same place: impressive technically, starved for contributors, forever chasing a moving target they can never quite catch.

I never liked the state of the Linux desktop either. Not because it’s bad per se, but because it’s fragmented. A KDE app on GNOME looks alien. Firefox rolls its own everything. GTK and Qt will never agree on anything. Every toolkit draws its own widgets, manages its own text rendering, handles its own accessibility story. The result is a desktop that feels like a coalition of independent projects rather than a coherent system.

This project is not going to fix any of that. But it’s an interesting story about how a combination of prior work, Apple open-sourcing key components, and an AI pair programmer led me down a rabbit hole I didn’t plan to enter.

The prior art that made this possible

I’ve been working with wgpu (Rust’s WebGPU implementation) for a while now, building a map renderer for Woosmap. That project taught me the fundamentals: how to manage GPU pipelines, how to do instanced rendering, how to deal with text atlases and glyph rasterization, how to bridge Rust and Swift through UniFFI.

On the Swift side, I had two apps: Tunes, a music player, and Leela, an internal management tool for Woosmap services. Both are SwiftUI apps, both depend on Apple’s frameworks in the usual way.

So one evening, half-curious and half-joking, I fed Claude the source of those projects alongside some context about GNUStep and friends, and typed:

“What would it take to make Tunes build and run on Linux?”

And then things got out of control.

What Claude came back with

The answer was, predictably, “a lot.” But the interesting part was the breakdown. SwiftUI is the main dependency, and SwiftUI is closed-source. But Swift itself is open-source. Swift Foundation is open-source. The Swift Package Manager works on Linux. So the gap is really:

A SwiftUI implementation (the view hierarchy, state management, layout engine, modifiers)
Some AppKit shims (NSColor, NSAppearance, the bits that SwiftUI still leans on)
A rendering backend that isn’t Core Animation or Metal
A compositor to manage windows, menus, and the desktop chrome

Instead of stopping at “that’s insane, don’t do it,” I kept going. Claude kept going. We started building.

Architecture

The project (codenamed Clone) splits pretty naturally along a language boundary.

Rust does what Rust is good at: GPU work. A wgpu renderer with pipelines for rectangles, rounded rects (SDF-based, because I can’t stop using SDFs apparently), shadows, text via a glyph atlas, and wallpapers. It also runs the window compositor through winit.

Swift does what Swift is good at: UI. A from-scratch SwiftUI implementation (about 50 View types, @State/@Binding/@Environment, ViewBuilder, layout, modifiers), the whole declarative stack. Plus enough AppKit shims to keep real apps happy, a SwiftData reimplementation backed by SQLite, and an IPC protocol over Unix sockets.

UniFFI bridges the two. Each frame, Rust asks Swift for render commands, Swift resolves the view tree into a flat list of positioned primitives, and Rust batches them into instanced GPU draws. It’s the same bridge I use in the map renderer, so at least that part wasn’t new territory.

graph LR
A["App.body"] --> B["ViewBuilder"] --> C["_resolve()"] --> D["Layout"]
D --> E["CommandFlattener"] --> F["IPC
CGFloat → Float"] --> G["Rust batcher"] --> H["wgpu draws"]
style A fill:#c4a7e7,stroke:#6e6a86,color:#191724
style B fill:#c4a7e7,stroke:#6e6a86,color:#191724
style C fill:#c4a7e7,stroke:#6e6a86,color:#191724
style D fill:#c4a7e7,stroke:#6e6a86,color:#191724
style E fill:#f6c177,stroke:#6e6a86,color:#191724
style F fill:#f6c177,stroke:#6e6a86,color:#191724
style G fill:#9ccfd8,stroke:#6e6a86,color:#191724
style H fill:#9ccfd8,stroke:#6e6a86,color:#191724

First signs of life

The moment I’ll remember is the grey rectangle. “Clone Desktop” in the center, a dock at the bottom with colored squares. I’d been at it for hours, wrestling with UniFFI bindings and layout math, and suddenly there it was: pixels on screen, drawn by Swift code, pushed through a Rust GPU backend, on something that was decidedly not macOS.

Day one: a grey box, a label, and a dock. It's not much, but it compiles and renders.

Settings and Finder

Once the basic views worked (stacks, text, lists, navigation), I needed something real to throw at them. Settings was a natural first target: sidebars, forms, toggles, text fields, the kind of layout variety that breaks things fast. Finder came next because browsing a directory with List and ForEach is such a fundamental SwiftUI pattern that if it didn’t work, nothing would.

Settings showing Wi-Fi preferences alongside Finder browsing the home directory. Both are SwiftUI apps running through Clone's stack.

Dark mode just kind of happened. Once I shimmed NSAppearance and implemented the semantic color system (Color.primary, .secondary, the system grays), flipping between light and dark was a toggle. A small thing, but unreasonably satisfying: it made the whole experiment feel like a real desktop for the first time.

The same Settings app in light mode. Appearance switching works through the reimplemented NSAppearance and Color system.

Tunes

This was the whole point, remember? “What would it take to make Tunes build and run on Linux?” Well, it builds. The login sheet renders inside the app window rather than as a separate NSPanel (a compromise, but not a terrible one), and the SwiftUI code is essentially unchanged. TextField, SecureField, Toggle, Button, Link: they all resolve and render. Same source, different universe.

Tunes running its login flow through Clone. The sheet renders in-window rather than as a separate panel, one of many compromises.

Leela

Leela is a management dashboard for Woosmap services: tabs, lists, nested navigation, version selectors, deploy queues. Most of the UI is driven by API responses, so it hammers ForEach with dynamic data, @StateObject, and conditional rendering. If Settings was a stroll through the park, Leela was a stress test.

Leela's services view running through Clone. Tabs, lists, version badges, sidebar navigation, all SwiftUI, all reimplemented.

I won’t pretend it was smooth. Getting here meant implementing a surprising amount of SwiftUI’s surface area. The state management system alone (StateGraph, scoped identity for ForEach, call-index disambiguation) went through several iterations where everything would render once and then silently stop updating. The kind of bug where you stare at a diff for an hour before realizing a closure captured a copy instead of a reference.

Under the hood

Text rendering

Text goes through cosmic-text for shaping, then gets rasterized into a 4096x4096 glyph atlas (single-channel, R8). Each glyph is cached and rendered as an instanced GPU quad. Fonts are bundled: Inter for UI text, Phosphor for icons.

This is almost certainly not how Apple does it. A real system would share GPU textures between the compositor and app, or pull from a system font cache. But it works, and there’s a special joy in watching a glyph atlas fill up character by character as the UI renders for the first time.

Layout

I reverse-engineered SwiftUI’s layout by reading objc.io’s thinking-in-swiftui and a lot of trial and error. The model: parents propose a size to children, children report back what they need, parents position them. Sounds simple until ZStack enters the picture: nil-sized views from .background() modifiers would expand to fill constraints and eat space from real siblings. That one took a while.

ScrollView was another head-scratcher: it fills its proposed size but lays out content unbounded in the scroll axis. Getting that inversion right, where the container constrains in one direction and is infinite in the other, broke my mental model twice before clicking.

IPC

Each app is a separate process. The compositor (CloneDesktop) runs the Rust event loop and manages surfaces. Apps connect over a Unix socket at /tmp/clone-compositor.sock and exchange length-prefixed JSON messages. There’s an annoying CGFloat-to-Float conversion at the wire boundary: Swift thinks in 64-bit coordinates, the GPU thinks in f32, and someone has to reconcile that at the border.

The `ycodebuild` trick

This is probably my favorite hack in the project. To compile an existing macOS app against Clone instead of Apple’s frameworks, ycodebuild generates a shadow SPM package that maps import SwiftUI to Clone’s SwiftUI, import AppKit to Clone’s shims, and so on. The app source code doesn’t change at all: you just build against a different package graph, and the compiled binary talks to the compositor over the socket. The same .swift file, two completely different platforms.

Honest assessment

I should be upfront about the gap between “it renders” and “it works.”

Animations are mostly stubbed. Accessibility is nonexistent. The compositor redraws every surface every frame like it’s 1997. There are // TODO: implement scattered across the codebase where AppKit APIs return no-ops and hope nobody notices. Sheets render in-window because I never built the panel system. The glyph atlas will fall over the moment someone opens a CJK document.

And (this is the awkward part) it doesn’t actually run on Linux yet. The whole premise was “what would it take,” and the answer turned out to include some Apple framework dependencies I haven’t replaced with their open-source equivalents. It’s doable. It’s just not done.

But here’s the thing that caught me off guard. The time from that initial Claude prompt to a state where Tunes and Leela compile and render recognizable UI was hours, not weeks. Not autonomous AI magic (plenty of manual fixes and architectural decisions along the way) but Claude carried an enormous amount of boilerplate: generating 50 View type stubs, wiring up modifier chains, implementing the state graph, setting up IPC. The kind of work that would’ve taken me days of tedious typing, compressed into a conversation.

Why this matters (a little)

Apple open-sourcing Swift and Foundation was a bigger deal than most people realize. Not because anyone’s going to ship a SwiftUI app on Linux tomorrow, but because it lowered the floor for experiments like this from “completely impossible” to “merely impractical.”

The projects I admired growing up (GNUStep, Haiku, Wine) were built by small teams reverse-engineering closed systems over years. The combination of open-source language infrastructure and AI assistance compresses that timeline dramatically. Not to a point where it’s practical or production-ready, but to a point where a single person can explore the shape of the problem in a weekend.

That’s the real takeaway. Not “I ported macOS to Linux.” I didn’t. But I went from a throwaway prompt to colored boxes on screen running real SwiftUI app code, and that felt like something worth writing about.

What’s next

Honestly? Probably nothing. The project scratched a twenty-year itch. But I know myself, and I know there’s a Kawase blur pipeline sitting unused in the renderer that’s going to call my name at 11pm some Tuesday. If I do keep going:

Per-window offscreen textures: the compositor shouldn’t redraw everything every frame, that’s embarrassing
Glassmorphism / backdrop blur: because what’s the point of reimplementing macOS if you can’t have the frosted glass
Actually running on Linux: the whole original premise, still unfinished
Shared GPU textures: how a real compositor would work, instead of copying pixels through a socket like an animal

Or maybe it stays as a weekend experiment and a blog post. Either way, the kid who thought GNUStep was the coolest thing ever is pretty happy right now.

When "mostly aligned" stops being enough

Sat, 07 Mar 2026 22:41:00 +0100

Every product starts by depending on open source it doesn’t fully understand. That’s fine. It’s the whole point. You stand on the shoulders of people who solved problems you haven’t even encountered yet, and you ship. The dependency is invisible because it works.

Then the product grows, and you start noticing the edges. The upstream project makes choices you wouldn’t have made. The schema carries fields you don’t use. The routing engine takes a turn you can’t explain. The update process becomes a gamble: bump to the latest version and hope nothing changed in a way that breaks your use case. You’re accountable to your customers for software you treat as a black box.

This is the story of how that played out for us across map tiles, routing, and rendering, and why the hardest part isn’t the engineering.

Not all dependencies are equal

We depend on dozens of open source projects. FastAPI, Django, Pydantic, httpx, pytest, the list goes on. If FastAPI disappeared tomorrow, we’d rewrite some decorators and route definitions. It would be work, but the product would still be the product. These are plumbing. They shape the surface (how requests arrive, how responses leave) but they don’t define what the product is.

Map tiles and routing are different. Without tiles, the maps SDK renders nothing. Without a routing engine, the distance API returns nothing. These aren’t tools we use to build the product. They are the product, or at least the foundation it stands on. An empty shell without them.

The dependency risk isn’t about how many lines of code use a library. It’s about how much of the product’s core value lives in upstream code you don’t control. When that answer is “most of it,” the relationship with that upstream project stops being a convenience and starts being an existential question.

The illusion

Here’s the part nobody talks about.

When you ship an MVP built on third-party foundations, the customer doesn’t know. They don’t know the tiles come from MapTiler. They don’t know the routing runs on Valhalla. They see a Woosmap maps API and a Woosmap distance API. They think they bought a product.

And they did, from their perspective. The API works. The tiles render. The routes compute. The fact that most of the core value is someone else’s software, running in your infrastructure, with your branding on top, is an implementation detail they never see.

So when you sell the MVP, you’re implicitly selling the product three years from now. The customer is buying what they think exists. Leadership is reporting on what they think the team built. And the engineering team knows the truth: that the product, as everyone imagines it, doesn’t exist yet. What exists is a thin layer on top of open source that happens to work.

This creates a brutal dynamic. When you invest in actually building the foundations (replacing vendor tiles with your own schema, understanding the routing engine you depend on, building a style compiler) it looks like you’re doing nothing. No new features. No visible progress. The maps still render. The routes still compute. From the outside, nothing changed.

Worse, the transition introduces blips. You’re replacing running systems with new ones you wrote, and you’re human, so mistakes happen. A regression in tile rendering. A routing edge case the old system handled that yours doesn’t yet. For a brief window, you’ve made things slightly worse in pursuit of making them fundamentally better. And nobody outside the team has the context to understand why.

You’re in a fight against everyone’s perception: customers, leadership, sometimes other engineering teams. Living up to what people think the product already does while quietly building the thing they assumed was there all along.

The pattern

It starts the same way every time.

Phase 1: it works. You adopt the open source project. It does 90% of what you need. The remaining 10% you work around. The trade-off is obvious and good: building from scratch would take months, this takes days. You ship.

Phase 2: it mostly works. The product grows. The 10% gap becomes visible to customers. You can’t change it because it’s upstream. You file issues, maybe contribute patches, but the project’s priorities aren’t your priorities. They’re building for a community. You’re building for a product.

Phase 3: you’re accountable. A customer reports a bug that lives in the upstream code. You can’t just say “that’s an open source issue.” It’s your product. You need to fix it, but fixing it means understanding a codebase that wasn’t written for you, by people who had different goals. The gap between “using” and “maintaining” becomes real.

Phase 4: the fork in the road. You have three options. Contribute upstream and hope the community accepts your direction. Fork and maintain your own version. Or rebuild the parts you need from scratch.

Map tiles: from vendor to vertical integration

Buying tiles

The first version of Woosmap Maps didn’t generate tiles at all. We licensed vector tiles from MapTiler, a flat yearly fee for an MBTiles pyramid. Predictable, simple. We focused entirely on the SDK: the Mapbox GL fork, the Google Maps compatibility layer, the service clients. The tiles were someone else’s problem.

This was the right call. We were building a maps product, not a geodata pipeline. MapTiler tiles worked, the OpenMapTiles schema was well-documented, and we could ship without learning anything about OSM data processing.

By the time we started thinking about generating tiles ourselves, the pricing model had changed. I don’t remember the exact structure anymore, but it was enough of a signal: the terms we’d signed up for weren’t the terms going forward. Beyond pricing, tiles carried 16 layers with attributes we never read, the update cadence wasn’t ours, and the OpenMapTiles CC-BY 4.0 license required visible attribution, which was friction in every enterprise sales conversation.

Running OpenMapTiles ourselves

We took the OpenMapTiles stack in-house. PostgreSQL-based: import OSM data with imposm3 into PostGIS, run generated SQL scripts to transform it layer by layer, then generate vector tiles through per-tile PostgreSQL queries. Same schema, our infrastructure.

It worked, but planet generation took days and the database needed hundreds of gigabytes. And we still had the same schema, the same unused attributes, the same attribution requirements. We’d replaced a vendor bill with an infrastructure bill while keeping every constraint.

Planetiler

Planetiler (originally Flatmap) changed the equation. Java, streaming processing, no database. A full planet in hours on a single machine instead of days on a database cluster. We ran it with the planetiler-openmaptiles profile: same output, fraction of the cost.

But we still had the schema problem. Faster generation of tiles you don’t control is still tiles you don’t control.

Clean-room schema

The real break was writing our own Planetiler profile from scratch. Not a fork of OpenMapTiles. A clean-room reimplementation with our own layer names, our own attributes, and nothing we don’t use.

transportation became roads. transportation_name became road_labels. housenumber got dropped entirely. Almost every layer is written in Kotlin because the logic is too specific for declarative YAML: transit network propagation from OSM relations, building height arithmetic, multi-script labels across 85 languages, roads pre-split by bridge/tunnel structure.

The style compiler

Owning the tile schema without owning the style is half the job. A production Mapbox GL style is thousands of lines of JSON. Roads alone need six variants each across eight road classes.

Elzar, our style compiler, generates that JSON from Python code. The tile schema and the style are designed in lockstep. When the Kotlin tile generator emits a hide_3d attribute on buildings, the style compiler references it in the same PR. Schema and style can’t drift because they live in the same repo.

The spec question

Once you own both the tiles and the style generation, a new question surfaces: does the Mapbox GL style spec itself actually fit what you’re building?

The spec was designed for a general-purpose renderer serving a general-purpose schema. It has no concept of road structure, so a single road class needs six style layers (tunnel casing, tunnel fill, road casing, road fill, bridge casing, bridge fill) because brunnel is just another filter attribute. Eight road classes times six variants is 48 layers just for roads, before labels and shields. It’s like using assembly to build a web server: technically possible, but is it reasonable? Complex logic gets encoded as JSON expression trees (nested arrays of ["case", ["all", ["has", "name:nonlatin"], ...]]) that are technically correct and practically unreadable.

Elzar makes this manageable. Define a road once, get six variants. Use ~Q(hide_3d__exists=True) instead of raw filter arrays. Pre-splitting roads into roads_tunnel, roads, and roads_bridge at the tile level was our way of optimizing around the spec, moving work from render time to tile generation time because the style format couldn’t express it efficiently.

Elzar would stay regardless of the output format. Even if we designed a custom style spec, we’d still want to define styles in Python (something executable, debuggable, testable) rather than editing a declarative format by hand. The compiler’s value isn’t papering over the Mapbox spec. It’s that Python is a better authoring language for styles than any JSON dialect ever will be. The compiler stays. Only its target changes.

The renderer

That’s where Nimbus, the GPU map renderer experiment in Rust, enters the picture. If you control the renderer, you control the style format. The Mapbox spec stops being a constraint and becomes a choice. You can support it for compatibility while designing something better for the pipeline you actually have.

A style format designed for our schema wouldn’t need six layers per road class. It would know about brunnel structure because the tiles already express it. Tiles that carry exactly the data the style describes. A style format that maps naturally to the tile schema. A renderer that understands both natively. And Elzar at the center, generating whatever output the renderer needs from the same Python definitions we already have.

Routing: the black box problem

OSRM

The distance API started on OSRM. It’s fast and well-proven. But it loads the entire road graph into RAM, and for a world-scale service, that’s a hard constraint. The memory footprint made it impractical for the coverage we needed.

Valhalla

Valhalla solved the memory problem. Tile-based graph storage means you can serve the world without loading it all into RAM. We migrated, it worked, and for a while the “mostly aligned” phase was comfortable.

Then we started hitting the edges.

Valhalla is a massive C++ codebase maintained by a community with its own priorities. When a customer reports a routing bug (a wrong turn cost, a route through a service road that should be avoided) we can’t just fix it. We’d need to understand why the cost model works the way it does, trace through code that wasn’t written for our use case, and either patch it locally or convince the upstream community that our fix is the right one.

The tile update process is where the black box problem gets painful. When we want to regenerate Valhalla tiles with fresh OSM data, we typically also bump Valhalla to the latest version, because the version that generates tiles and the version that serves them sometimes need to match. But a version bump means inheriting every change the community made since our last update. New cost model tweaks we didn’t ask for. Behavioral changes we don’t understand. Route quality regressions we discover from customer complaints, not from changelogs.

We’re accountable for this software. Customers don’t care that Valhalla is open source. They see a Woosmap API returning a bad route and they file a ticket with us. And we’re stuck between “we don’t understand why it does that” and “we can’t change it without forking a C++ project we don’t deeply know.”

The C++ ecosystem makes this harder. Build systems, dependency management, debugging tooling: none of it is as accessible as the JVM or Rust’s cargo. Taking deeper ownership of Valhalla isn’t just a knowledge problem, it’s an ergonomics problem.

Calculon

Calculon is the “what if we built it ourselves” experiment: a routing engine in Rust. Bidirectional A*, many-to-many distance matrices, driving/cycling/pedestrian profiles, Valhalla-compatible API. It reuses Valhalla’s tile format (designing a graph format from scratch would be its own project) but everything else is new code.

It’s not a Valhalla replacement yet. But when a cost model produces a weird route, we can read the code. When we want to change how turn penalties work, we change them. When we bump a dependency, it’s a Rust crate with a changelog we understand, not a C++ monolith where any subsystem might have shifted.

There’s a middle path too: taking deeper ownership of Valhalla itself, contributing changes that serve our use case. But that requires investing heavily in a C++ codebase whose architecture and community may or may not align with where we need to go.

When replacing a black box makes things better

The pattern isn’t always painful. Sometimes replacing a dependency you don’t control with something you wrote yourself turns out to be a straight upgrade.

The Woosmap Store Locator used Mapnik (a C++ rendering library) to generate map tiles showing store locations. Mapnik is a serious piece of software, battle-tested for raster map rendering. But it wasn’t designed for our use case: rendering MVT vector tiles with point data. It had become a dependency we couldn’t easily update, couldn’t debug when it misbehaved, and couldn’t adapt to our evolving needs.

We replaced it with a Python renderer built on Cairo. The new renderer was simpler: written in a language the whole team knows, doing exactly what we need and nothing more. And it turned out to be more performant. Less load on the database. Enough of an improvement that we could scale down the RDS instance.

A full-featured C++ rendering engine, replaced by a focused Python implementation, and the result was faster and cheaper. Not because Python is faster than C++. Obviously it isn’t. But because a tool built for your specific problem, that you understand completely, can be optimized in ways a general-purpose tool never will be. You know which queries to run, which data to skip, which shortcuts are safe. The black box can’t know any of that.

The ones still early in the cycle

Not every dependency has reached the accountability phase yet. Maparazzo, our static maps renderer, wraps Mapbox GL Native’s C++ core in a Python library for headless map image generation. Right now it’s in phase 1: it works. The C++ renderer produces correct images, the Python wrapper makes it callable from our FastAPI services, and static maps ship to customers.

But the signs are already there. The C++ renderer leaks memory: OpenGL contexts hold state after objects are destroyed. We’ve worked around it by pooling Map instances and reusing GL contexts, but the leak is in code we inherited, not code we wrote. When it misbehaves, debugging means diving into a massive C++ codebase with its own conventions, its own build system, its own history of decisions we weren’t part of.

It’s comfortable enough today. The problems will come. And when they do, Nimbus (which can already render our tiles headlessly from Rust) is the path out. Same pattern, different timeline.

The open source dependency lifecycle

The uncomfortable truth is that depending on open source unknowingly makes you a maintainer. Not in the “you have commit access” sense. In the “you’re responsible for its behavior in your product” sense.

For an MVP, “mostly aligned” is enough. The open source project does 90% of what you need, the community is active, the abstractions hold. You’d be foolish to build from scratch. And the community’s goals don’t need to perfectly match yours. They just need to overlap enough that you can ship.

But products have customers. Customers have expectations. When the open source project’s goals diverge from yours, even slightly, you feel it. Not as a single breaking change, but as accumulated friction. Features you can’t add because the schema doesn’t support them. Bugs you can’t fix because the codebase is impenetrable. Updates you can’t skip because the format requires version parity.

And ownership doesn’t stop at the obvious dependencies. Once you own the data and the generation, you discover that the formats and specs are dependencies too. The Mapbox style spec. Valhalla’s tile format. These are constraints you inherited from the ecosystem, and they shape what’s easy and what’s hard in ways that aren’t visible until you try to push past them.

The question isn’t whether to take ownership. It’s when, of what, and how deep. Do the changes you need benefit the community? Would they accept a PR, or is your direction orthogonal to theirs? Is the ecosystem accessible enough to invest in, or is the cost of understanding it higher than the cost of rebuilding?

For tiles, we kept Planetiler as the engine and own everything else: schema, style, and the compiler that connects them. For routing, the answer is still forming. For the style format and the renderer, we can work around the constraints today, but we can see the path to not having to.

“We have no vision”

I’ve heard this from engineers on the team. That we’re only reactive. That we just respond to customer requests and there’s no plan. And I understand why it looks that way: if you think phase 1 is the end, then everything after looks like firefighting.

But if you lay out the phases, the vision is obvious. We have years of deliberate work ahead: owning the tile schema, closing the style spec gap, making the renderer a first-class citizen, taking control of routing. Each phase builds on the last. Each one makes the product more ours and less a wrapper around someone else’s decisions. That’s not reactive. That’s a roadmap.

The problem is that this roadmap is invisible to anyone who thinks the product was finished when the MVP shipped. And that’s the majority of people: customers, leadership, and yes, most engineers. They see the API, it works, so the hard part must be done. Everything after is maintenance.

This is exactly the same problem as technical debt, just at a different scale. Everyone will sign up for taking on debt if they think it ships something faster. Add a dependency you don’t understand. Skip the tests. Use the upstream schema as-is. Ship it. Move on to the next thing, take on more debt there too. And when someone proposes paying it down (replacing Mapnik with our own renderer, writing a clean-room tile schema, building a style compiler) the response is “why? it works.”

It works until it doesn’t. And by then the debt has compounded. The codebase is half-rotten, everyone complains about it, but nobody wants to stop adding features long enough to fix it. The boy scout rule (leave the code better than you found it) gets dismissed as slowing things down. So the rot spreads.

Supply chain debt works the same way. Every dependency you don’t understand is a loan against your future ability to move fast. The interest is paid in debugging time, in workarounds, in version bumps that break things you can’t explain, in customers waiting for fixes that live in upstream code you can’t change. And just like financial debt, the longer you ignore it, the more it costs.

The vision isn’t a feature roadmap. It’s the systematic transfer of ownership from upstream to us, one layer at a time, while the product keeps running. It’s unglamorous, often invisible, and it’s the only way to build something that lasts.

My role as CTO is to grow the technological asset of the company. Features are what the product does. The asset is what the product is. The PO grows the former. I grow the latter. When the asset is a thin wrapper around upstream dependencies, it’s fragile. Anyone can replicate it. Every phase of ownership makes it thicker, harder to reproduce, more valuable. That’s not maintenance. That’s building the company’s core value.

And that means making these phases crystal clear to everyone. If the team doesn’t see the multi-phase roadmap, they’ll assume there isn’t one. If leadership doesn’t understand why replacing a working renderer with a new one matters, they’ll see wasted effort. If engineers think phase 1 is the destination, they’ll resist every investment in phase 2 through 5 as unnecessary complexity.

And here’s where I haven’t done a good enough job. When the foundational work isn’t visible (or worse, when I end up doing it on my own because it’s hard to justify in sprint planning) it creates the illusion that there’s only one stream of work: the product owner’s backlog. Feature requests, customer issues, integration tickets. That becomes the only work that feels real, the only work that gets discussed in standups, the only work that counts.

Engineers naturally gravitate toward pleasing the PO. The PO has the backlog, the priorities, the stakeholder pressure. The CTO has… a vague sense that the foundations need work. If I haven’t made the supply chain roadmap as concrete and visible as the feature backlog, I can’t blame anyone for treating it as optional.

But it’s a dangerous dynamic. A team that only serves the PO’s backlog is a team that only adds features to an MVP while the foundations quietly rot. The PO’s job is to maximize customer value now. The CTO’s job is to make sure the product can still deliver that value in two years. Those aren’t opposing goals, but they operate on different timescales, and if the long-term one isn’t articulated clearly enough, it loses every single sprint planning meeting.

The vision exists. It’s in the progression from bought tiles to clean-room schema. It’s in Calculon sitting next to Valhalla. It’s in Elzar generating styles that match a tile schema we designed. The work speaks for itself, but only if someone tells the story clearly enough that it becomes a roadmap, not a side project. And that someone is me.

There’s a harder question underneath all of this: is growing the technological asset even possible when the company is purely ARR-driven?

ARR optimizes for now. Ship the feature that closes the deal. Fix the bug that stops the churn. Hit the number this quarter. Every sprint is a negotiation between what moves the metric today and what makes the product real. And the metric always wins, because it’s concrete: a number on a dashboard, a target in a board deck.

The technological asset doesn’t have a dashboard. There’s no metric for “we now own our tile schema” or “we can debug a routing issue in hours instead of weeks.” The value is real but it compounds invisibly: faster iteration, fewer black-box surprises, the ability to say yes to product requests that would have been impossible before. None of that shows up in ARR until months or years later, and by then nobody connects the cause to the effect.

A company that’s purely ARR-driven will always deprioritize foundational work. Not out of malice. Out of measurement. If the only thing that counts is what moves the number this quarter, then building the asset that makes next year’s numbers possible will always lose. And you end up in a trap: a product that looks successful from the revenue side while its foundations quietly thin out, making it harder and more expensive to deliver on the promises that drive that very revenue.

The uncomfortable truth is that ARR and the technological asset need each other. ARR without the asset is a product that gets harder to maintain every quarter. The asset without ARR is an engineering project with no customers. The CTO’s job is to make the case that both need investment, and that starving one to feed the other is a debt that always comes due.

You start by depending. You end by being accountable. What happens in between is the difference between an MVP and a product.

Multi-arch Docker builds in GitHub Actions

Thu, 19 Feb 2026 10:00:00 +0200

We needed ARM64 containers. Our Python services run on mixed infrastructure: amd64 in CI and some production clusters, arm64 on newer nodes. Building on one arch and emulating the other with QEMU was painfully slow and broke native extensions. So we added proper multi-arch builds to our GitHub Actions CI.

It took a week to get right. Then five more fixes over two weeks as each failure mode revealed itself in production. This is what went wrong and how we fixed it.

The setup

We use reusable workflows in a shared Woosmap/.github repo. Individual service repos call these. A build-helper repo (checked out at runtime) contains the actual logic as a single Node.js action that routes to TypeScript modules based on an action input:

build-helper/src/main.ts

switch (action) {
 case 'build': await build(await get_build_info()); break
 case 'publish': await publish(await get_build_info()); break
 case 'release': await release(await get_build_info()); break
 case 'create_manifest':
 await createManifest(await get_build_info()); break
 case 'create_release_manifest':
 await createReleaseManifests(await get_build_info()); break
}

The multi-arch matrix is generated dynamically in the workflow:

.github/workflows/sonar_python_bender_build.yml

strategy:
 matrix:
 include: ${{ fromJson(inputs.multi_arch &&
 '[{"runner":"ubuntu-24.04","arch":"amd64"},
 {"runner":"ubuntu-24.04-arm","arch":"arm64"}]'
 || '[{"runner":"ubuntu-24.04","arch":"amd64"}]') }}
runs-on: ${{ matrix.runner }}

When multi_arch is true, two jobs run in parallel on native runners. No QEMU. An ARCH_SUFFIX environment variable (-amd64 or -arm64) drives platform selection downstream:

build-helper/src/build.ts

export function getPlatformForArch(): string {
 const archSuffix = getArchSuffix()
 return archSuffix === '-arm64' ? 'linux/arm64' : 'linux/amd64'
}

ARM64 jobs skip tests, coverage, and SonarCloud. Only amd64 does those. Build + push only for arm64.

Problem 1: buildx attestations

Buildx 0.10+ defaults to pushing attestation manifests (SBOM, provenance) alongside your image. When you docker push myimage:tag, it doesn’t push a plain image, it pushes a manifest list containing the image plus attestation layers.

This is fine if you just pull and run. But we need to stitch arch-specific images into a multi-arch manifest:

docker manifest create myimage:pr_10 \
 --amend myimage:sha-amd64 \
 --amend myimage:sha-arm64

docker manifest create expects plain images as sources, not manifest lists. When sha-amd64 is itself a manifest list (because of attestations), the stitching fails.

The fix: disable attestations with BUILDX_NO_DEFAULT_ATTESTATIONS=1.

Our first attempt was conditional:

# Broken: empty string "" fails strconv.ParseBool
BUILDX_NO_DEFAULT_ATTESTATIONS: ${{ inputs.multi_arch && '1' || '' }}

When multi_arch is false, this evaluates to "". Buildx tries to parse it as a boolean via Go’s strconv.ParseBool, which fails on empty string. This broke every non-multi-arch build across the organization.

The fix was trivial, always set it to 1:

BUILDX_NO_DEFAULT_ATTESTATIONS: 1

Attestations aren’t needed regardless of arch mode. An env var being set to empty is different from not being set at all: a subtle but organization-wide footgun.

Problem 2: PR number resolution

Each arch job publishes its image with a tag. We need a consistent tag so the manifest step can find both. The original design used pr_X-amd64 / pr_X-arm64, where the PR number came from an API call:

build-helper/src/utils.ts (before)

export async function get_pr(): Promise<number> {
 const {data: commit_prs} =
 await oktokit.rest.repos.listPullRequestsAssociatedWithCommit({
 owner: repo[0], repo: repo[1], commit_sha: sha
 })
 if (commit_prs[0] === undefined) {
 throw new Error('Commit has no PR')
 }
 return commit_prs[0].number
}

The listPullRequestsAssociatedWithCommit API sometimes returned empty results: race conditions, merge commits, event timing. When it failed, get_publish_tag() silently fell back to the raw SHA:

build-helper/src/utils.ts (silent fallback)

export async function get_publish_tag(): Promise<string> {
 try {
 return `pr_${await get_pr()}`
 } catch (e) {
 return process.env['GITHUB_SHA'] || '' // No warning!
 }
}

The amd64 job would tag as pr_42-amd64 (API succeeded), the arm64 job as abc123def (API failed). The manifest step looked for pr_42-arm64, which didn’t exist.

Two fixes:

Read PR number from the event payload, always available, zero API calls:

build-helper/src/utils.ts (after)

export async function get_pr(): Promise<number> {
 const con = new Context()
 if (con.payload.pull_request) {
 return con.payload.pull_request.number
 }
 // Fallback for push events only
 const {data: commit_prs} = await oktokit.rest.repos
 .listPullRequestsAssociatedWithCommit({...})
 // ...
}

Use commit SHA for arch-suffixed images, the SHA is always the same across parallel jobs:

build-helper/src/publish.ts

export async function docker(image: string, name: string): Promise<void> {
 const archSuffix = getArchSuffix()
 const sha = process.env['GITHUB_SHA'] || ''
 const tag = archSuffix ? `${sha}${archSuffix}` : await get_publish_tag()
 // ...
}

The manifest step bridges them:

build-helper/src/manifest.ts

const manifestTag = await get_publish_tag() // "pr_X"
const sha = process.env['GITHUB_SHA'] || ''
const archTags = ARCHITECTURES.map(arch => `${sha}-${arch}`)
await createAndPushManifest(name, manifestTag, archTags)

So the flow is: {sha}-amd64 + {sha}-arm64 → manifest pr_X. No API race, no silent fallback.

Problem 3: deploy ordering

The Leela admin service deploys on every PR build. It was wired inline in the build job:

- name: Deploy
 if: matrix.arch == 'amd64'
 uses: ./helper/.github/actions/deploy

Problem: in multi-arch mode, the deploy step ran before the manifest existed. It tried to pull pr_X, which only gets created in a separate create_manifest job that depends on both arch builds completing.

The fix adds a separate downstream job for multi-arch deploys:

.github/workflows/sonar_python_bender_build.yml

# Inline deploy only for single-arch
- name: Deploy
 if: matrix.arch == 'amd64' && !inputs.multi_arch
 uses: ./helper/.github/actions/deploy

# Multi-arch: deploy after manifest is ready
pr_deploy:
 if: inputs.multi_arch
 needs: create_manifest
 runs-on: ubuntu-24.04
 steps:
 - name: Deploy
 uses: ./helper/.github/actions/deploy

Same pattern for the release workflow: deploy only after create_release_manifest completes.

Problem 4: release version calculation

The createReleaseManifests function creates the multi-arch manifest for release tags. It originally called get_split_version_tag() to determine the version:

build-helper/src/release.ts (before)

const tag = await get_split_version_tag() // Fetches latest release, bumps version
const version = `${tag[0]}.${tag[1]}.${tag[2]}`

get_split_version_tag() fetches the latest GitHub release and bumps the version based on PR labels. But createReleaseManifests runs after the release jobs complete. The release already happened. So get_latest_tag() now returns v5.5.3 (the new release), and bumping it gives v5.5.4, which doesn’t exist as a tag.

The fix: just use get_latest_tag() directly.

build-helper/src/release.ts (after)

const version = await get_latest_tag()
const parts = version.split('.')
const archTags = ARCHITECTURES.map(arch => `${version}-${arch}`)

await createAndPushManifest(name, version, archTags) // v1.2.3
await createAndPushManifest(name, `${parts[0]}.${parts[1]}`, archTags) // v1.2
await createAndPushManifest(name, parts[0], archTags) // v1

Three manifest tags per release. Semver consumers can pin to major, minor, or patch.

The full tagging flow

After all the fixes, the image lifecycle looks like this:

PR Build:
{sha}-amd64, {sha}-arm64 → manifest: pr_X
Release (per-arch):
pull pr_X (Docker resolves correct arch)
push v1.2.3-amd64, v1.2.3-arm64
Release Manifests:
v1.2.3-amd64 + v1.2.3-arm64 → manifests: v1.2.3, v1.2, v1

Only the amd64 job creates the GitHub release to avoid duplicates:

build-helper/src/release.ts

if (!archSuffix || archSuffix === '-amd64') {
 await create_release(version)
} else {
 core.info(`Skipping GitHub release creation on ${archSuffix} build`)
}

Timeline

Feb 4: Initial multi-arch support. Matrix strategy, arch suffix, manifest creation.
Feb 5: Deploy ordering fix. Leela deploying before manifests existed.
Feb 16: Same fix for the release/merge workflow.
Feb 17: BUILDX_NO_DEFAULT_ATTESTATIONS to fix attestation-poisoned manifests.
Feb 18: SHA-based arch tagging + PR number from event payload.
Feb 19: BUILDX_NO_DEFAULT_ATTESTATIONS empty string crash fix.
Feb 19: get_latest_tag() instead of get_split_version_tag() in release manifests.

Five fixes in two weeks after the initial rollout. Each one discovered in production by a different failure mode.

What I’d do differently

Test the non-multi-arch path. The empty string strconv.ParseBool crash only affected repos that hadn’t opted into multi-arch. We tested the new feature, not the old path.

Use SHA tags from the start. The PR-number-based approach was elegant but fragile. The SHA is always consistent across parallel jobs, no API calls, no race conditions.

Make the manifest step a required downstream job. The deploy-before-manifest problem was obvious in retrospect. Any step that depends on a multi-arch manifest should be in a job that needs: the manifest creation job. No exceptions, no “it works for single-arch” shortcuts.

Docker’s multi-arch story is good once it works. Getting there from a single-arch CI that evolved organically over years is where the pain lives. Every assumption about “one build produces one image” breaks in interesting ways.

Full-text search and lyrics in a Go music server

Wed, 18 Feb 2026 18:00:00 +0200

The Tunes server started as a way to stream my music library from a Raspberry Pi. It’s accumulated features since then. Two recent additions changed how I interact with it day-to-day: accent-insensitive full-text search across the entire library, and lyrics, both embedded from audio files and fetched from LRClib.

The search problem

Searching a music library is harder than it sounds. You want “bjork” to match “Björk”. You want “cafe” to match “Café”. You want “creme” to match “Crème Brûlée” in an album title. And you want it fast enough that results appear as you type.

The previous search was a LIKE query with wildcards. It worked, but it couldn’t handle diacritics and got slow on a 15,000-track library with multi-column matches.

FTS5 with unicode61

SQLite’s FTS5 extension does exactly what we need. The unicode61 tokenizer with remove_diacritics 2 strips all Unicode diacritical marks during indexing and querying:

tunes/search_fts.go: FTS5 table creation

func (s *SearchFTS) InitializeFTS() error {
 createSQL := `
 CREATE VIRTUAL TABLE IF NOT EXISTS tracks_fts USING fts5(
 track_id UNINDEXED,
 name,
 artist,
 album_artist,
 album,
 composer,
 genre,
 tokenize='unicode61 remove_diacritics 2'
 )`
 if err := s.db.Exec(createSQL).Error; err != nil {
 if strings.Contains(err.Error(), "no such module: fts5") {
 Sugar.Warn("FTS5 module not available - search will be disabled.")
 s.fts5Available = false
 return nil
 }
 return fmt.Errorf("failed to create FTS5 table: %w", err)
 }
 s.fts5Available = true
 return nil
}

track_id UNINDEXED stores the ID in the FTS table for joins but doesn’t tokenize it. If FTS5 isn’t compiled into the SQLite build, search degrades gracefully instead of crashing.

The index rebuilds atomically in a transaction: delete all FTS data, repopulate from the tracks table using COALESCE for NULL safety:

tunes/search_fts.go: index rebuild

func (s *SearchFTS) RebuildIndex(ctx context.Context) error {
 return s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
 if err := tx.Exec("DELETE FROM tracks_fts").Error; err != nil {
 return fmt.Errorf("failed to clear FTS table: %w", err)
 }
 populateSQL := `
 INSERT INTO tracks_fts (track_id, name, artist, album_artist,
 album, composer, genre)
 SELECT track_id, COALESCE(name, ''), COALESCE(artist, ''),
 COALESCE(album_artist, ''), COALESCE(album, ''),
 COALESCE(composer, ''), COALESCE(genre, '')
 FROM i_tunes_tracks WHERE deleted_at IS NULL`
 return tx.Exec(populateSQL).Error
 })
}

Query building

User queries are split into words, each gets a * suffix for prefix matching, and they’re joined with implicit AND. FTS5 special characters are stripped to prevent injection:

tunes/search_fts.go: query sanitization

func buildFTSQuery(query string) string {
 words := strings.Fields(strings.TrimSpace(query))
 var parts []string
 for _, word := range words {
 escaped := escapeFTSToken(word)
 if escaped != "" {
 parts = append(parts, escaped+"*")
 }
 }
 return strings.Join(parts, " ")
}

func escapeFTSToken(token string) string {
 var result strings.Builder
 for _, r := range token {
 switch r {
 case '"', '*', '^', '(', ')', '{', '}', '[', ']', ':', '-':
 continue
 default:
 result.WriteRune(r)
 }
 }
 return result.String()
}

Typing “aphex win” becomes aphex* win*, which matches “Aphex Twin” via prefix expansion. Typing “cafe del” matches “Café Del Mar” because the tokenizer already stripped the accent at index time.

Unified search endpoint

The /search?q= endpoint returns artists, albums, and tracks in a single response, each ranked independently by BM25:

tunes/search_handler.go

type UnifiedSearchResponse struct {
 Artists []ArtistInfo `json:"artists"`
 Albums []AlbumInfo `json:"albums"`
 Tracks []ITunesTrack `json:"tracks"`
}

func (ls *LibraryService) HandleSearch(w http.ResponseWriter, r *http.Request) {
 query := strings.TrimSpace(r.URL.Query().Get("q"))
 artists := ls.searchArtistsFTS(ctx, repo, query, 5)
 albums := ls.searchAlbumsFTS(ctx, repo, query, 10)
 tracks := ls.searchTracksFTS(ctx, repo, query, 30)
 // ...
}

Each section queries specific FTS5 columns. Artist search hits both artist and album_artist, then merges and aggregates. The BM25 scoring comes from FTS5’s built-in function:

SELECT track_id, bm25(tracks_fts) as rank
FROM tracks_fts
WHERE tracks_fts MATCH ?
ORDER BY rank
LIMIT ?

BM25 order is preserved through joins by building a track map and reordering after the GORM query:

tunes/search_handler.go: preserving BM25 order

trackMap := make(map[int]ITunesTrack, len(tracks))
for _, t := range tracks {
 if t.TrackID != nil {
 trackMap[*t.TrackID] = t
 }
}
ordered := make([]ITunesTrack, 0, len(trackIDs))
for _, id := range trackIDs {
 if t, ok := trackMap[id]; ok {
 ordered = append(ordered, t)
 }
}

Album search uses a two-stage approach: FTS returns track IDs, those map to album_key, then the album_cache table provides pre-aggregated album metadata. This avoids re-aggregating album info on every search.

Lyrics from audio files

The music scanner already walks the filesystem to index tracks using github.com/dhowden/tag. Adding lyrics extraction was a matter of probing the right metadata keys:

tunes/music_scanner.go: lyrics extraction

func (ms *MusicScanner) extractLyricsFromMetadata(rawMetadata any) *ExtractedLyrics {
 var lyrics ExtractedLyrics
 switch raw := rawMetadata.(type) {
 case map[string]any:
 for _, key := range []string{
 "lyrics", "USLT", "unsynchronised_lyrics",
 "unsynced_lyrics", "LYRICS", "\xa9lyr",
 } {
 if val, ok := raw[key]; ok {
 switch v := val.(type) {
 case string:
 if v != "" { lyrics.PlainLyrics = v }
 case []byte:
 if len(v) > 0 { lyrics.PlainLyrics = string(v) }
 }
 }
 }
 // Also check SYLT for synchronized lyrics
 for _, key := range []string{"SYLT", "synchronised_lyrics", "synced_lyrics"} {
 // ... same pattern
 }
 }
 return &lyrics
}

USLT is the ID3v2 unsynchronized lyrics frame, "\xa9lyr" is the M4A/AAC lyrics atom, SYLT is synchronized (timestamped) lyrics. The scanner handles both plain text and timestamped LRC format.

LRClib integration

For tracks without embedded lyrics, the server fetches from LRClib, a free, open lyrics database with synced timestamps.

The lyrics service uses golang.org/x/time/rate to respect API limits:

tunes/lyrics_service.go

const (
 lrcLibBaseURL = "https://lrclib.net/api"
 lrcLibUserAgent = "Tunes/1.0 (https://github.com/manz/tunes-server)"
 lrcLibRateLimit = 2
)

func NewLyricsService(db *gorm.DB) *LyricsService {
 return &LyricsService{
 repo: NewLyricsRepository(db),
 httpClient: &http.Client{Timeout: 15 * time.Second},
 rateLimiter: rate.NewLimiter(rate.Limit(lrcLibRateLimit), 1),
 }
}

When fetching, the service validates that the returned lyrics match the track’s duration within 5 seconds. This prevents mismatches: same song title by different artists, live vs studio versions:

tunes/lyrics_service.go: duration tolerance

if duration > 0 && lrcResponse.Duration > 0 {
 if math.Abs(float64(duration)-lrcResponse.Duration) > durationTolerance {
 return nil, ErrLyricsNotFound
 }
}

The handler implements fetch-through caching: check the database first, if miss then fetch from LRClib, save, and return. No external call on subsequent requests for the same track.

Lyrics are stored with GORM’s FirstOrCreate + Assign for clean upserts:

tunes/lyrics_repository.go

func (r *LyricsRepository) SaveLyrics(ctx context.Context, lyrics *TrackLyrics) error {
 return r.db.WithContext(ctx).
 Where(TrackLyrics{TrackID: lyrics.TrackID}).
 Assign(TrackLyrics{
 LRCLibID: lyrics.LRCLibID,
 Instrumental: lyrics.Instrumental,
 PlainLyrics: lyrics.PlainLyrics,
 SyncedLyrics: lyrics.SyncedLyrics,
 Source: lyrics.Source,
 }).
 FirstOrCreate(lyrics).Error
}

Background enrichment

A POST /lyrics/enrich endpoint triggers a goroutine that processes tracks without lyrics in batches of 50. It finds un-enriched tracks with a LEFT JOIN:

tunes/lyrics_repository.go: finding tracks without lyrics

err := r.db.Joins(
 "LEFT JOIN track_lyrics ON i_tunes_tracks.track_id = track_lyrics.track_id",
).Where("track_lyrics.id IS NULL").
 Where("i_tunes_tracks.deleted_at IS NULL").
 Limit(limit).
 Find(&tracks).Error

Rate-limited at 2 requests per second, a 15,000-track library takes a few hours to fully enrich. The mutex prevents concurrent enrichment runs:

tunes/lyrics_service.go: single-flight enrichment

func (s *LyricsService) StartBackgroundEnrichment(ctx context.Context, ...) {
 s.mu.Lock()
 if s.isEnriching {
 s.mu.Unlock()
 return
 }
 s.isEnriching = true
 s.mu.Unlock()

 go func() {
 defer func() { s.mu.Lock(); s.isEnriching = false; s.mu.Unlock() }()
 tracks, _ := s.repo.GetTracksWithoutLyrics(ctx, 50)
 // ... process batch, respect rate limiter
 }()
}

Admin via sudoers

The authorization model is Unix-inspired. Instead of a boolean is_admin on the User model, there’s a dedicated sudoers table with audit trail:

tunes/models.go

type Sudoer struct {
 ID int `gorm:"primaryKey;autoIncrement"`
 UserID int `gorm:"column:user_id;uniqueIndex;not null"`
 GrantedBy int `gorm:"column:granted_by;not null"`
 GrantedAt time.Time `gorm:"autoCreateTime"`
 ExpiresAt *time.Time `gorm:"column:expires_at"`
 Active bool `gorm:"column:active;default:true"`
}

Destructive operations (deleting tracks, managing users) additionally require a sudo token, obtained by re-authenticating via WebAuthn passkey. Two-step elevation: request a challenge, prove you’re you, get a time-limited token. This prevents a stolen session from doing real damage.

Routes use higher-order functions for auth gating:

cmd/tunes-server/main.go

func sudoerRoute(authMiddleware func(http.Handler) http.Handler,
 authService *AuthService) func(http.HandlerFunc) http.Handler {
 return func(handler http.HandlerFunc) http.Handler {
 return authMiddleware(requireSudoerMiddleware(authService)(handler))
 }
}

The first user must be created via CLI before the server starts: tunes-server users add <username> --sudoer. No web-based registration for the initial admin. You need shell access.

What changed

The server went from “stream my library” to “actually find what I want to play.” FTS5 with accent stripping makes search work the way you’d expect: type a rough approximation of the artist name, get the right results. Lyrics with synced timestamps let the iOS app show karaoke-style scrolling text. Background enrichment means I didn’t have to manually tag anything. It just filled in over a couple of evenings.

Still no replacement for Apple Music’s catalog. But for my 15,000 tracks, it works better than iTunes Match ever did.

Calculon part 2: costing profiles, performance, and maneuvers

Wed, 18 Feb 2026 10:00:00 +0200

Why can't I move on?

Part 1 covered the basics: six crates, Valhalla’s tile format, a driving cost model, bidirectional A*, many-to-many matrix, and an Axum HTTP server. Everything worked for the driving case on the Monaco dataset.

Since then, the engine gained bicycle and pedestrian costing, memory-mapped tiles, a comparison webapp, turn-by-turn directions in 34 languages, and enough performance work to make it usable on France-scale data. This is what changed.

Bicycle and pedestrian costing

Two new cost models implement the DynamicCost trait alongside the existing AutoCost.

Bicycle defaults to 18 kph (hybrid bike). The model factors in grade, road stress, surface quality, and turn costs, all lighter than auto because a cyclist doesn’t mind a left turn as much as a driver crossing oncoming traffic.

Grade uses a Tobler-style speed table. Flat terrain is 1.0x; steep downhill goes up to 2.2x; steep uphill drops to 0.3x. Hill avoidance is a separate preference (use_hills) that applies an additional penalty on grades regardless of the speed adjustment.

The interesting part is road stress, how comfortable a road is to cycle on:

crates/calculon-costing/src/bicycle_cost.rs

fn accommodation_factor(&self, edge: &DirectedEdge) -> f32 {
 let use_type = edge.use_type();
 if use_type == Use::Cycleway as u32 || use_type == Use::MountainBike as u32 {
 return 0.0;
 }
 match edge.cycle_lane() {
 CycleLane::Separated => 0.0,
 CycleLane::Dedicated => 0.1,
 CycleLane::Shared => 0.3,
 CycleLane::None => {
 if edge.bike_network() { 0.2 }
 else if edge.shoulder() { 0.5 }
 else { 1.0 }
 }
 }
}

A separated cycle lane costs nothing extra. A shared lane adds 0.3. No accommodation at all on a road that’s not part of any bike network costs 1.0, scaled by the use_roads preference. Combined with ROAD_CLASS_FACTOR that penalizes trunk roads (0.8) vs residential (0.0), the engine steers cyclists toward quieter streets with bike infrastructure.

Pedestrian defaults to 5.1 kph. It adds SAC hiking scale (7-tier trail difficulty), a step penalty for stairs, lighting preference, and use-type bonuses for footways and sidewalks:

crates/calculon-costing/src/pedestrian_cost.rs

const GRADE_SPEED_FACTOR: [f32; 16] = [
 1.5, 1.4, 1.3, 1.15, 1.05, 1.0, // downhill: slightly faster
 1.0, 1.0, // flat
 0.95, 0.85, 0.75, 0.65, 0.55, 0.45, 0.4, 0.35, // uphill: slower
];

Both profiles make a critical architectural decision: they disable hierarchy entirely. The A* hierarchy transitions work by “jumping up” to highways as the search moves away from endpoints. That makes sense for driving: a 200km route shouldn’t expand residential streets in the middle. But for cycling and walking, the shortcut edges that power hierarchy are built for driving and would produce wrong routes. So bicycle and pedestrian stay on level 2 where all edges are accessible, trading search speed for correctness.

Memory-mapped tiles

The tile reader now uses mmap(2) instead of heap-allocated reads:

crates/calculon-tiles/src/graph_tile.rs

enum TileData {
 Owned(Vec<u8>), // Tests
 Mapped(Mmap), // Production
}

pub fn from_mmap(mmap: Mmap) -> Result<Self, TileError> {
 let base_ll = Self::validate(&mmap)?;
 Ok(GraphTile { data: TileData::Mapped(mmap), base_ll })
}

In the graph reader:

crates/calculon-tiles/src/graph_reader.rs

let file = std::fs::File::open(&path).map_err(TileError::Io)?;
let mmap = unsafe { Mmap::map(&file) }.map_err(TileError::Io)?;
let tile = Arc::new(GraphTile::from_mmap(mmap)?);

No heap allocation, no copy. The OS page cache handles eviction. The old application-level LRU cache became unnecessary. Tiles are cheap Arc pointers to mapped pages.

Thread-local tile cache

Routing makes thousands of get_tile calls per search. Even a RwLock on a HashMap adds up. The solution is a three-level cache:

crates/calculon-tiles/src/graph_reader.rs

thread_local! {
 static TL_CACHE: RefCell<FxHashMap<u32, Arc<GraphTile>>> =
 RefCell::new(FxHashMap::default());
}

pub fn get_tile(&self, id: &GraphId) -> Result<Arc<GraphTile>, TileError> {
 let tile_key = id.tile_value();

 // Level 1: thread-local (no synchronization, ~20ns)
 let hit = TL_CACHE.with(|c| c.borrow().get(&tile_key).cloned());
 if let Some(tile) = hit { return Ok(tile); }

 // Level 2: shared cache (RwLock, parallel readers)
 // ... insert into thread-local on hit

 // Level 3: mmap from disk, populate both caches
 // ...
}

The hot path (thread-local hit) is a plain HashMap lookup in a RefCell. No lock, no contention, no Arc::clone from the shared cache. FxHash (from rustc_hash) replaces std::HashMap everywhere, optimized for integer keys that don’t need cryptographic hashing.

Distance approximator

Edge snapping calls distance computations in a tight loop, projecting a point onto every nearby road segment. Haversine uses sin, cos, asin, sqrt per call. For relative comparisons within a small area, that’s overkill.

The DistanceApproximator pre-computes cos(lat) once and then uses only multiplications:

crates/calculon-core/src/geo.rs

pub struct DistanceApproximator {
 lng_scale: f64,
 meters_per_lng_degree: f64,
}

impl DistanceApproximator {
 pub fn new(ll: &PointLL) -> Self {
 let lng_scale = Self::lng_scale_per_lat(ll.lat);
 Self {
 lng_scale,
 meters_per_lng_degree: lng_scale * METERS_PER_DEGREE_LAT,
 }
 }

 pub fn distance_squared(&self, ll: &PointLL) -> f64 {
 let lat_m = (ll.lat - self.center_lat) * METERS_PER_DEGREE_LAT;
 let lng_m = (ll.lng - self.center_lng) * self.meters_per_lng_degree;
 lat_m * lat_m + lng_m * lng_m
 }
}

Construct one for the search point, reuse it across all segment projections. 26% faster in edge snapping benchmarks. The accuracy loss is negligible within a few kilometers, well within the range of any snap search.

Distance-adaptive hierarchy

Short urban routes were sometimes jumping to highways unnecessarily. A 2km trip through a city shouldn’t consider the nearby motorway just because the hierarchy said “expand to level 0 after 5,000 meters.”

The fix scales hierarchy limits proportionally to route distance:

crates/calculon-route/src/bidirectional_astar.rs

fn adjust_limits_for_distance(limits: &mut [HierarchyLimits], distance: f32) {
 // 5km route: expand_within_dist = 2500m, max_up_transitions = 11
 // 1km route: expand_within_dist = 2000m, max_up_transitions = 5
 // 20km+ route: no adjustment
}

A 5km route caps level-2 expansion at 2,500m and allows only 11 upward transitions (from 100 default). A 1km route drops to 5 transitions with a 2,000m cap. Routes above 15km keep the defaults. The search still finds the optimal path, it just doesn’t waste time on irrelevant highway segments for short trips.

A* heuristics in the cost matrix

The matrix previously used pure Dijkstra for each source/target pair: expand uniformly in all directions. Now each gets an A* heuristic aimed at its closest counterpart:

crates/calculon-matrix/src/cost_matrix.rs

let closest = targets.iter()
 .min_by(|a, b| {
 let da = src.distance(a);
 let db = src.distance(b);
 da.partial_cmp(&db).unwrap_or(std::cmp::Ordering::Equal)
 })
 .unwrap_or(&targets[0]);
h.init(closest, cost_factor);
self.heuristics_forward.push(h);

This steers each search toward likely connection points rather than expanding uniformly. Combined with Valhalla-style hierarchy and a path_dist_threshold set to 2x the max source-target distance, the matrix handles 9x9 urban grids without exploring the entire region.

Turn-by-turn maneuvers

The first version returned routes with “Start” and “Arrive” stubs. Now there’s a proper maneuver builder that groups consecutive edges into instructions.

The builder triggers a new maneuver on name changes, significant turns, road type transitions (roundabouts, ramps, ferries):

crates/calculon/src/maneuver_builder.rs

fn should_start_new_maneuver(&self, in_roundabout: bool) -> bool {
 self.entering_roundabout
 || self.exiting_roundabout
 || self.entering_ramp
 || self.exiting_ramp
 || self.entering_ferry
 || self.exiting_ferry
 || self.turn_type == TurnType::Reverse
 || (self.name_changed && !in_roundabout)
 || (self.turn_type != TurnType::Straight && self.name_changed)
}

Narrative generation uses rust-i18n with 34 locale files. Each maneuver type maps to a template:

locales/en-US.yml

start.5: "Drive %{cardinal_direction} on %{street_names}."
turn.1: "Turn %{relative_direction} onto %{street_names}."
enter_roundabout.2: "Enter the roundabout and take the %{ordinal_value} exit onto %{roundabout_exit_street_names}."
destination.0: "You have arrived at your destination."

Roundabouts track exit counts. Cardinal directions derive from heading. Ordinals support up to the 10th exit. The locale files include French, German, Spanish, Italian, and (yes) pirate English.

Edge snapping improvements

Snapping a coordinate to the road network was one of the weaker points. Two fixes:

Multi-tile search. When a point falls near a tile boundary, the search now checks neighboring tiles in all 8 directions:

crates/calculon-route/src/bidirectional_astar.rs

let near_south = point.lat - bbox.min_pt.lat < TILE_BOUNDARY_MARGIN;
let near_north = bbox.max_pt.lat - point.lat < TILE_BOUNDARY_MARGIN;
// ... check all 8 directions
for (i, &(dlng, dlat)) in offsets.iter().enumerate() {
 if !flags[i] { continue; }
 let neighbor_id = TileHierarchy::get_graph_id(&neighbor_pt, level);
 if let Ok(tile) = reader.get_tile(&neighbor_id) {
 let candidates = tile.find_closest_edges(point, access_mask, ...);
 all.extend(candidates);
 }
}

Smarter candidate pruning. Post-collection, candidates beyond min_distance * 3.0 (minimum 25m threshold) are pruned, then select_nth_unstable_by does an O(N) partition instead of a full sort. Only the top K candidates get sorted.

All My Circuits

A Lit + TypeScript + MapLibre GL webapp for side-by-side comparison with Valhalla. Route and matrix modes, all three costing profiles, Valhalla comparison with percentage diffs on time and distance, resizable sidebar, URL state persistence, clickable matrix routes in verbose mode, and links to Google Maps and HERE for sanity checking.

The debug mode loads per-edge data (speed, density, road class, costs) so you can see exactly why the engine chose one road over another. The inspect endpoint visualizes tile-level node and edge data for when things really go wrong.

Criterion benchmarks

Benchmarks run against France tiles across four crates:

Tiles: cache hit (~20ns thread-local), cold load (mmap), edge traversal
Costing: single edge cost, transition cost, access check
Routing: 5km urban, 55km regional, 240km long-distance
Matrix: 1x1, 4x4 coastal, 9x9 urban

A separate bench runner does HTTP-level comparison against Valhalla, reporting p50/p95/p99 latencies and route cost/distance diffs. Useful for regression testing and for validating that the cost model matches Valhalla’s output within acceptable margins.

Where it stands

The engine handles France-scale data with three costing profiles. Routes match Valhalla on the scenarios I’ve tested. Turn-by-turn directions work in 34 languages. The matrix scales to 9x9 with A* heuristics. Performance is good enough for a single-server deployment.

What’s new since part 1:

Bicycle and pedestrian cost models
Memory-mapped tiles with thread-local cache
FxHash everywhere, DistanceApproximator for snapping (26% faster)
Distance-adaptive hierarchy limits
A* heuristics in the cost matrix
Turn-by-turn maneuvers with localized narrative in 34 languages
Multi-tile edge snapping
Criterion benchmarks and a Valhalla comparison webapp

What’s still ahead:

Isochrones: reachability polygons from a point within a time/distance budget
Contraction hierarchies: the hierarchy transitions help but true CH preprocessing would unlock cross-Europe routes at acceptable latency
Via points: multi-leg routes with intermediate stops

Building a routing engine in Rust

Sat, 14 Feb 2026 10:00:00 +0200

Why can't I move on?

A routing engine in Rust. Bidirectional A*, many-to-many distance matrices, Valhalla-compatible API. Six crates, about 8,000 lines.

Why

We use Valhalla for routing and distance matrices at Woosmap. It works most of the time. When it doesn’t, we’re stuck. We treat it as a black box: file an issue, upgrade, hope the next release fixes things. When the matrix returns wrong costs for certain edge cases or the engine routes you through someone’s driveway, there’s no realistic way for us to dig in. It’s a massive C++ codebase that assumes you’ve been living in it for years.

The goal is to replace Valhalla entirely for our use case: routing, distance matrices, and isochrones, with driving, cycling, and walking profiles. Not fork it, not wrap it. Rewrite the parts we need in Rust so we can actually own the thing.

Architecture

Six crates in a Cargo workspace:

calculon/
├── calculon-core # Foundation types: coordinates, graph IDs, constants
├── calculon-tiles # Binary tile reader: nodes, edges, transitions
├── calculon-costing # Cost models: auto/driving with turn penalties
├── calculon-route # Bidirectional A* pathfinding
├── calculon-matrix # Many-to-many distance matrix
└── calculon # HTTP server: Axum handlers, Valhalla-compatible API

The HTTP layer at the top is thin: parse JSON, call the algorithm, format the response. Everything interesting happens in the middle crates.

Tile-based graph

The road network lives in binary tiles, same format Valhalla uses. Nodes are intersections, directed edges are road segments, edge info holds geometry and names, node transitions connect tiles to each other.

crates/calculon-tiles/src/graph_tile.rs

pub fn from_bytes(data: Vec<u8>) -> Result<Self, TileError> {
 if data.len() < size_of::<GraphTileHeader>() {
 return Err(TileError::TooSmall {
 size: data.len(),
 min: size_of::<GraphTileHeader>(),
 });
 }

 let header = unsafe { &*(data.as_ptr() as *const GraphTileHeader) };

 if header.end_offset() as usize != data.len() {
 return Err(TileError::SizeMismatch {
 header_size: header.end_offset() as usize,
 actual_size: data.len(),
 });
 }

 let base_ll = header.base_ll();
 Ok(GraphTile { data, base_ll })
}

/// Get the node at a given index.
pub fn node(&self, index: u32) -> Option<&NodeInfo> {
 if index >= self.header().nodecount() {
 return None;
 }
 let offset = size_of::<GraphTileHeader>() + index as usize * size_of::<NodeInfo>();
 Some(unsafe { &*(self.data[offset..].as_ptr() as *const NodeInfo) })
}

No deserialization, just pointer casting into the raw bytes. The tile is a flat buffer with a header followed by fixed-size node and edge records at known offsets. Fast, but the crate refuses to compile on big-endian targets because the byte layout has to match.

GraphReader sits on top of an LRU cache. Algorithm needs an edge in some tile? Check the cache, load the file if it’s not there, hand back an Arc<GraphTile>. Default cache holds 200 tiles, which covers most regional queries without touching disk twice.

Reusing Valhalla’s tile format was a big shortcut. Designing a graph format from scratch would have been a project in itself.

Cost model

This is where routing gets interesting, and where most bugs hide. The cost model decides how expensive each road segment is to traverse. Not just distance. Time, comfort, how annoying a particular turn is. The values here are derived from Valhalla’s own cost model. No point reinventing what they’ve already calibrated.

crates/calculon-costing/src/auto_cost.rs

const RIGHT_SIDE_TURN_COSTS: [f32; 8] = [
 0.5, // Straight
 0.75, // SlightRight
 1.0, // Right (favorable)
 1.5, // SharpRight (favorable-sharp)
 9.5, // Reverse
 3.5, // SharpLeft (unfavorable-sharp)
 2.5, // Left (unfavorable, crosses oncoming traffic)
 0.75, // SlightLeft
];

const LEFT_SIDE_TURN_COSTS: [f32; 8] = [
 0.5, // Straight
 0.75, // SlightRight
 2.5, // Right (unfavorable)
 3.5, // SharpRight (unfavorable-sharp)
 9.5, // Reverse
 1.5, // SharpLeft (favorable-sharp)
 1.0, // Left (favorable)
 0.75, // SlightLeft
];

Two tables: one for right-side driving countries (France, US), one for left-side (UK, Japan). In France, a left turn crosses oncoming traffic so it costs more. In the UK, it’s the right turn. Get this wrong and the engine starts suggesting weird U-turn loops to avoid a simple left.

The rest of the cost model:

Speed factors: pre-computed 3.6 / speed_kph lookup table, seconds per meter at each speed
Highway preference: a slider from -1.0 to 1.0, push it high and the engine prefers motorways
Surface penalties: gravel costs more than asphalt
Intersection density: dense urban intersections add delay, factors range from 1.0 up to 3.5
Toll booths: flat time penalty per crossing
Traffic signals: extra delay at signalized intersections

All of this is pre-computed into lookup tables at startup. During search, computing the cost of an edge is just array indexing.

The algorithm finds a shortest path. The cost model decides whether that path makes any sense to a human driver. Same constants Valhalla uses, but now in code we can read and step through when something looks off.

Bidirectional A*

Two search trees grow at the same time: one forward from the origin, one backward from the destination. They meet somewhere in the middle.

crates/calculon-route/src/bidirectional_astar.rs

pub struct BidirectionalAStar {
 edgelabels_forward: Vec<BDEdgeLabel>,
 edgelabels_reverse: Vec<BDEdgeLabel>,

 adjacencylist_forward: DoubleBucketQueue,
 adjacencylist_reverse: DoubleBucketQueue,

 edgestatus_forward: EdgeStatus,
 edgestatus_reverse: EdgeStatus,

 heuristic_forward: AStarHeuristic,
 heuristic_reverse: AStarHeuristic,

 cost_diff: f32,
 cost_threshold: f32,
 best_connections: Vec<CandidateConnection>,
 // ...
}

Everything is doubled: edge labels, priority queues, edge status, heuristics. The heuristic is geographic distance to the opposite endpoint. Simple, admissible, good enough.

The priority queue is a double-bucket structure: coarse outer buckets, fine inner buckets. Turns out a binary heap isn’t great for routing because the cost values cluster in narrow ranges. Buckets handle that better.

The search stops when the best connection found so far, plus a 420-second buffer, exceeds the cheapest item in either queue. That buffer lets it keep looking for slightly better paths after the first connection without searching forever.

Hierarchy transitions help a lot. Near the endpoints, the search considers local roads. As it moves away, it “jumps up” to highways and trunk roads. In the middle of a long route, there’s no point expanding residential streets. This cuts the number of edges explored dramatically.

Many-to-many matrix

A 50x50 distance matrix done naively (individual route for each pair) means 2,500 searches. That’s not going to work.

The trick: expand all sources forward simultaneously, all targets backward simultaneously, and watch for collisions. When source i’s forward search reaches an edge that target j’s reverse search already touched, that’s a connection.

crates/calculon-matrix/src/cost_matrix.rs

pub struct CostMatrix {
 edgelabels_forward: Vec<Vec<BDEdgeLabel>>,
 queues_forward: Vec<DoubleBucketQueue>,
 edgestatus_forward: Vec<EdgeStatus>,

 edgelabels_reverse: Vec<Vec<BDEdgeLabel>>,
 // ...
}

Each source and target gets its own labels, queue, and status set, but they all walk the same graph. A ReachedMap (basically a HashMap<edge_id, Vec<location_index>>) tracks who’s been where. When a forward expansion hits an edge in the reverse map, we record the connection cost.

Result is an N*M grid of time/distance values. Some cells might be unreachable. The whole thing caps at 2 million iterations so it doesn’t run forever on disconnected graphs.

HTTP server

Three endpoints on Axum, matching Valhalla’s JSON format so existing clients can point at Calculon without code changes.

crates/calculon/src/server.rs

pub fn build_router(state: Arc<AppState>) -> Router {
 Router::new()
 .route("/route", post(handler_route::handle_route))
 .route("/sources_to_targets", post(handler_matrix::handle_matrix))
 .route("/status", get(handler_status::handle_status))
 .layer(CorsLayer::permissive())
 .with_state(state)
}

Route and matrix searches are CPU-bound: they can block for hundreds of milliseconds on a big query. They run on Tokio’s blocking thread pool via spawn_blocking so they don’t starve the async runtime.

Each request gets a CancelGuard backed by an AtomicBool. Client disconnects? The guard drops, sets the flag, and the algorithm checks it periodically and bails out. Seemed like overkill at first, but it matters when someone fires off a matrix request and immediately refreshes the page.

A route request:

{
 "locations": [
 {"lat": 43.731, "lon": 7.419},
 {"lat": 43.696, "lon": 7.271}
 ],
 "costing": "auto",
 "costing_options": {"auto": {"use_highways": 0.8}},
 "units": "kilometers"
}

Response comes back with a polyline6-encoded shape, total time, distance, bounding box, and some basic maneuvers (though the maneuvers are stubs for now, just “Start” and “Arrive”).

Where it stands

Driving works. Routes match Valhalla’s output on the Monaco dataset. The matrix handles concurrent requests with cancellation. The server drops in behind the same API.

What’s there:

Driving cost model with turns, highways, surface, density, tolls
Bidirectional A* with hierarchy transitions
Many-to-many matrix via simultaneous bidirectional Dijkstra
Valhalla-compatible REST API
LRU tile caching with memory-mapped I/O
Cooperative request cancellation
Tests against Monaco dataset

What’s left to reach parity:

Cycling and walking profiles: the cost model is structured for it, but only driving is implemented so far.
Isochrones: reachability polygons from a point within a time/distance budget. The search infrastructure is there, the polygon generation isn’t.
Turn-by-turn directions: maneuvers are stubs. Real maneuver generation needs street names, instruction templates, sign data.
Contraction hierarchies: the A* is fine for regional routes but CH would be needed for cross-country at acceptable latency.
Multi-leg routes: only origin to destination, no via points yet.

About 8,000 lines across six crates. The driving case works end to end. Cycling, walking, and isochrones are what’s between this and actually replacing Valhalla.

Scaling Freddie to 10m landcover

Tue, 10 Feb 2026 12:00:00 +0200

Serving vector tiles from 10-meter resolution land cover data. What breaks when you 100x the input resolution, and how topology-preserving smoothing fixes the ugly parts.

Background

Freddie is our tile server for land cover data. It reads GeoTIFF raster files (pixels classified as wood, grass, built-up, water, etc.) and serves them as vector tiles or raster tiles over HTTP. Nothing fancy. Read pixels, make shapes, encode, serve.

The original setup used ESA WorldCover at ~100m resolution. Native resolution sits around z8 in Web Mercator, so serving z0–z8 tiles was straightforward. Sample pixels, run-length encode, merge adjacent faces of the same class, encode to MVT. Done. It just worked, and I didn’t think much about it.

Then we switched to the 10m dataset. Native resolution jumps to ~z13–z14. The source TIFFs balloon to 36,000 x 36,000 pixels each. Everything that worked at 100m either broke or looked terrible. This is the story of fixing all of it.

What breaks

Three problems, in order of how quickly you notice them.

Staircase boundaries

At zoom levels above native resolution (z15, z16), each source pixel stretches across 4–16 tile pixels. The vector boundaries follow pixel edges exactly: 90-degree staircase patterns everywhere. At 100m this was never visible because nobody zoomed past z8. At 10m you can zoom to z16 and the pixel grid is right there, staring back at you. Minecraft terrain, basically.

Performance at low zoom

A z0 tile covers the entire world. With a 36,000 x 36,000 source TIFF, generating that tile means sampling millions of pixels. At 100m the source was small enough that brute force worked. At 10m, a single z0 request could take seconds. I watched the first one come back and thought the server had hung.

Topology seams

The naive fix for staircases (smooth each polygon independently) creates gaps and overlaps between adjacent regions. Two polygons that share a boundary get smoothed in different directions. You end up with visible seams, z-fighting, or missing pixels at class boundaries. Fix one problem, create another. Classic.

COG multi-resolution

The performance problem, at least, has a clean solution: Cloud Optimized GeoTIFF.

A COG file stores the full-resolution image plus downsampled overviews, typically at 2x, 4x, 8x, 16x, 32x reductions. Instead of one 36,000 x 36,000 image, you get a pyramid:

Level	Size	Use for
0	36,000 x 36,000	z10+
1	18,000 x 18,000	z8–z9
2	9,000 x 9,000	z6–z7
3	4,500 x 4,500	z4–z5
4	2,250 x 2,250	z2–z3
5	1,125 x 1,125	z0–z1

The heuristic is simple: levelIndex = (maxZoom - zoom) / 2. A z0 tile now samples from a 1,125 x 1,125 overview instead of the full image. About 1,000x less data to read.

Resolution level selection

func (tc *TiffCollection) selectLevel(zoom int) int {
 levelIndex := (tc.maxZoom - zoom) / 2
 if levelIndex >= len(tc.levels) {
 levelIndex = len(tc.levels) - 1
 }
 return levelIndex
}

The TiffCollection manages multiple TIFF files with spatial indexing: a 3 x 3 degree grid for fast lookups. At tile boundaries where multiple TIFFs overlap, deterministic ordering ensures consistent output.

z0 tiles went from seconds to milliseconds. Problem one down, two to go.

The smoothing problem

With COG handling performance, the real challenge is visual quality. Run-length encoding produces pixel-aligned polygons. At native resolution that’s fine: the boundaries are detailed enough. But zoom in past native and you see the grid. It’s ugly, and no amount of squinting makes it acceptable.

Two complementary approaches: smooth the raster before vectorizing, or smooth the vectors after. I ended up doing both.

SDF raster filtering

Signed Distance Fields are typically used in font rendering. Valve popularized them for GPU text back in 2007. Turns out the same idea applies beautifully to classified rasters.

For each class in the tile:

Compute a distance transform (Meijster algorithm, O(n) per dimension): every pixel gets its Euclidean distance to the nearest boundary
Sign it: positive inside the class region, negative outside
Gaussian blur the SDF
Reclassify pixels based on the blurred SDF values

Blurring an SDF instead of blurring raw pixel values is the key insight. Raw blur smears class boundaries into garbage. SDF blur produces smooth, continuous boundaries that still snap cleanly to class assignments.

The blur radius scales with the upscale factor:

z14 (native, 1x): blur = 6.0 source pixels
z15 (2x upscale): blur = 3.0 source pixels
z16 (4x upscale): blur = 1.5 source pixels
z17+ (8x+): clamped to 1.0

A 5 x 5 mode filter can run before SDF to remove isolated single-pixel noise, common in land cover data where a lone “built-up” pixel appears in a forest.

Barbapapa: topology-preserving vector smoothing

SDF smoothing handles the raster level. But even with pre-smoothed rasters, the vectorized output still has pixel-aligned vertices. Near native resolution, where SDF barely does anything, the staircases are still there. We need vector-level smoothing too.

The fundamental problem: polygons that share a boundary must be smoothed identically along that boundary. Smooth them independently and you get gaps and overlaps. I tried independent smoothing first. It looked like the polygons were trying to escape from each other.

The solution uses the half-edge data structure (DCEL) that Freddie already maintains for face merging. Adjacent polygons share vertex references through twin half-edges. Move a shared vertex and it moves for both polygons simultaneously. The topology enforces consistency.

I named the algorithm Barbapapa, after the cartoon character that can reshape into anything while staying the same blob. The algorithm has five stages:

1. Vertex deduplication. After face merging, some vertices at identical coordinates are separate objects. Canonicalize them so shared boundaries reference the same vertex instance.

Vertex deduplication

canonical := make(map[[2]int]*vertex)
for _, v := range allVertices {
 key := [2]int{v.X, v.Y}
 if existing, ok := canonical[key]; ok {
 // Repoint all half-edges from v to existing
 replaceVertex(v, existing)
 } else {
 canonical[key] = v
 }
}

2. Edge subdivision. Laplacian smoothing can only move existing vertices: it can’t add curvature where there are no points. Subdivide edges by inserting midpoints, spaced at ~3.5 source pixels apart. Both sides of a shared edge (the half-edge and its twin) get subdivided identically.

3. Identify immovable vertices. Tile boundary vertices must not move (adjacent tiles need matching edges). Vertices that exist only on hole boundaries are also pinned (I’ll get to why that caveat matters in a moment).

4. Laplacian smoothing. For each movable interior vertex, compute the centroid of its neighbors and blend:

new_position = (1 - alpha) * current + alpha * centroid

Jacobi-style: compute all new positions first, then apply all updates simultaneously. This avoids order-dependent results. Three to four iterations with alpha = 0.5–0.6 produces good results.

5. Collision prevention. Ensure no vertex collapses onto another. Sorted vertex iteration guarantees deterministic output.

The algorithm overhead is under 5% of total tile generation time. Most time is still spent on TIFF reading and face merging. Good: the smoothing is essentially free.

The hole vertex problem

Version 1 of Barbapapa had a subtle bug that killed 93% of its effectiveness. I shipped it thinking it was working. It was barely doing anything.

Here’s the setup: small classified regions (a pond in a forest, a building in a field) appear both as a face (the pond) and as a hole in the surrounding face (the forest). The vertices on these boundaries are shared between the face’s outer ring and the surrounding face’s hole ring.

My original code marked all hole-boundary vertices as immovable. The reasoning seemed sound: holes have specific shapes that shouldn’t be distorted.

The problem: a vertex on a small face’s outer boundary is also on the surrounding face’s hole boundary. So marking all hole vertices as immovable pinned almost every interior vertex. In test datasets, 110 out of 118 interior faces had zero smoothing applied. The algorithm was running, doing math, producing output, and changing almost nothing. I only caught it when I added per-face metrics and saw the numbers.

The fix: only pin vertices that appear exclusively on hole boundaries and never on any face’s outer boundary. After the fix, unsmoothed interior faces dropped from 110 to 8, the remaining 8 being sub-pixel features too small to matter. Suddenly Barbapapa actually worked.

Zoom-adaptive parameters

Smoothing needs vary wildly across zoom levels. At z14 (native resolution) the pixels are small, so light smoothing is fine. At z16 (4x upscale) the pixel grid is screaming at you, so you need to be aggressive. Below native, the data is already downsampled and naturally smooth.

Rather than hardcoding per-zoom parameters, everything scales with sourcePixelSize: how many tile pixels each source pixel occupies:

Adaptive smoothing parameters

func GetEffectiveParams(sourcePixelSize float64) (iterations int, alpha float64) {
 if sourcePixelSize < 1.0 {
 // Below native: scale down proportionally
 scale := sourcePixelSize
 return int(3.0 * scale), 0.5 * scale
 }
 // Above native: boost with log curve, cap at reasonable limits
 iterBoost := math.Log2(sourcePixelSize) * 0.75
 alphaBoost := math.Log2(sourcePixelSize) * 0.05
 return min(int(3.0+iterBoost), 8), min(0.5+alphaBoost, 0.8)
}

Using pixel size directly instead of zoom level means the same code works regardless of source resolution. Swap in a different dataset and the smoothing adapts automatically. One less thing to tune by hand.

Pipeline system

With multiple raster filters and vector simplifiers, the number of possible combinations grew fast. I was adding flags to the server every other day (--sdf, --mode-filter, --barbapapa-iterations, --sdf-blur-radius) and it was getting out of hand. So I built a pipeline system instead.

A pipeline is a named sequence of processing steps:

TIFF Data → [Raster Filters] → Vectorization → [Vector Simplifiers] → MVT

Built-in pipelines cover common configurations:

Pipeline	Raster filters	Vector simplifiers
raw	none	SimplifyFaces
default	none	SimplifyFaces, Barbapapa
smooth-sdf	SDF	SimplifyFaces, Barbapapa
smooth-mode5x5	Mode 5x5	SimplifyFaces, Barbapapa
smooth-combined	Mode 5x5, SDF	SimplifyFaces, Barbapapa
aggressive	Mode 5x5, SDF (8px)	SimplifyFaces, Barbapapa (8 iter), Douglas-Peucker

Each step collects metrics (timing, face counts, vertex counts) so you can see exactly where time is spent. This turned out to be invaluable for debugging: when a tile looks wrong, you can trace exactly which step mangled it.

Running with a pipeline

./freddie-server --pipeline smooth-combined

One flag instead of six. Custom pipelines can still be composed from individual filters if you want to experiment.

Determinism

This one bit me late. Go maps have non-deterministic iteration order. It’s by design, to prevent people from depending on it. When iterating over faces to apply smoothing, different runs could produce different vertex orderings, leading to different (though equally valid) output. Visually identical, but different bytes.

For tile caching that’s a disaster. You want the same input to always produce the exact same bytes, otherwise your cache never hits. The fix: an OrderedFaceMap that sorts keys once at construction and iterates in sorted order. Combined with Jacobi-style (simultaneous) updates in Barbapapa, the output is now bit-for-bit reproducible. Run it twice, get the same file. Should have been obvious from the start, but Go’s map randomization is the kind of thing you forget about until it ruins your afternoon.

Performance

Numbers matter, so here they are. Benchmarks on M1 Max, single tile at z8:

Operation	Time	Memory	Allocations
GetWebMercatorTileMVT	345ms	152MB	599k
GetBitmapTileWithMargins	259ms	117MB	10k
GetWebMercatorTilePNG	247ms	113MB	10k

With 10 workers: ~35ms per tile, ~286 tiles/second. A full z0–z8 pyramid (21,845 tiles) generates in about 90 seconds.

I built a benchmark harness that runs multiple pipelines across multiple regions and zoom levels, spitting out per-tile metrics and SVG/PNG outputs for visual comparison. Change a value, regenerate, compare. Without this I would have been guessing at parameter values forever. The difference between “looks right” and “is right” is hard to eyeball across thousands of tiles.

The algorithm’s evolution

Barbapapa went through seven revisions. Each one was driven by a specific visual artifact I found while staring at tiles, the kind of bug where you zoom into some random patch of Indonesia and go “wait, what is that.”

v1.0: Basic Laplacian smoothing. Worked, but barely visible. I thought the whole approach was a dead end.
v1.1: Edge subdivision. Suddenly there were actual curves instead of straight lines. The moment it clicked.
v1.2: Vertex deduplication. Enabled cross-face consistency. Shared boundaries finally moved together.
v1.3: Linked twin subdivision. Topological correctness for shared edges, fixing the cases where v1.2 missed a link.
v1.4: Zero-length edge prevention. Eliminated degenerate geometry that made the MVT encoder choke.
v2.0: Hole-only vertex identification. The 93% fix. The algorithm finally did what it was supposed to do all along.
v2.1: Deterministic iteration. Reproducible output for caching, after Go’s map randomization burned me.

The benchmark harness made this iteration loop possible. Every revision got validated against the full test suite before shipping: no visual regressions, no degenerate geometry, no byte-level surprises.

Known limitations

Two artifacts I decided to live with.

Fish-tail spikes at z14+. Where two smoothing forces compete at narrow polygon tips, V-shaped spikes can appear. They’re small, they’re rare, and fixing them would mean adding a post-smoothing vertex cleanup pass that I don’t want to maintain.

Integer coordinate quantization. MVT uses integer coordinates. Smoothed positions get rounded, which at very high zoom can produce slightly uneven curves. Floating-point vertices would fix it, but that’s a change that ripples through the entire encoding pipeline. Not worth it for an artifact you have to zoom to z17 to notice.

What changed architecturally

Looking back, the 10m upgrade touched four areas of the codebase, and the interesting thing is how little of the original code it replaced:

Multi-resolution TIFF handling: COG support with automatic level selection. The TiffCollection manages spatial indexing across multiple files. This was the most “engineering” change: straightforward but essential.
Half-edge topology operations: Safe edge chaining, twin linking, subdivision. Error handling for topology invariant violations. This is the foundation that makes Barbapapa possible, and the part I’m most proud of. Getting the DCEL right was hard, but once it worked, everything built on top of it was clean.
Pipeline system: Composable raster and vector processing steps with metrics. Named pipelines for one-command configuration. Born out of flag fatigue.
Adaptive parameters: Everything scales with sourcePixelSize instead of zoom level. Swap the dataset and the system adapts. The kind of design decision that pays for itself the first time someone asks “can we try a different source?”

The core tile serving path (HTTP handler, TIFF sampling, face merging, MVT encoding) stayed the same. The new code wraps around it rather than replacing it. I like that.

Where it stands

Freddie serves 10m land cover data from z0 to z16. Low zoom tiles are fast thanks to COG overviews. High zoom tiles look smooth thanks to Barbapapa and optional SDF filtering. The pipeline system makes it easy to experiment with different filter combinations without touching server code.

The 10m dataset covers the full globe at a resolution where you can see individual fields and city blocks. At 100m everything was green-or-gray blobs. At 10m you can tell a park from a garden.

Tree: a browser that remembers where you came from

Tue, 10 Feb 2026 10:00:00 +0200

Every browser has tabs. Linear, flat, disposable. Open thirty of them researching one topic and they sit in a row with no indication that twelve of them came from the same Wikipedia article. Close the wrong one and the context is gone.

Tree is a macOS browser that replaces tabs with a tree. Every page knows its parent. Cmd+Click a link and it becomes a child node of the current page. The sidebar shows the full hierarchy: research paths branch naturally, and you can always trace back to where you started.

The data model

The core is a SwiftData model with a self-referential parent-child relationship:

Models/TreeNode.swift

@Model
final class TreeNode {
 #Unique<TreeNode>([\.nodeId])

 var nodeId: UUID
 var url: URL
 var title: String
 var faviconData: Data?
 var timestamp: Date
 var lastUsedAt: Date = Date()
 var weight: Int
 var isPinned: Bool
 var sortOrder: Int = 0

 @Relationship(inverse: \TreeNode.children)
 var parent: TreeNode?

 @Relationship(deleteRule: .cascade)
 var children: [TreeNode]
}

The cascade delete rule means pruning a node takes its entire subtree with it. No orphans. The parent/children relationship is bidirectional. SwiftData maintains both sides automatically through the inverse parameter.

Each node tracks a weight that increments on every visit, and an isPinned flag. These feed the search ranking:

var searchRank: Int {
 (weight * 2) + (isPinned ? 100 : 0)
}

Pinned nodes get a massive boost. Frequently visited pages float to the top. It’s a simple heuristic but it works: the pages you care about surface first.

Branching

The key interaction is branching: Cmd+Click a link and the browser creates a child node instead of navigating away. The parent shifts to the secondary panel, and the child becomes the primary:

State/NavigationState.swift

func selectFromBranch(_ child: TreeNode) {
 secondaryNode = primaryNode
 primaryNode = child

 child.incrementWeight()
 updateWebViewVisibility()
 saveLastSession()
}

This is where the tree structure pays off. You’re reading documentation, Cmd+Click a linked API reference, and now you see both side by side: the original page on the right, the reference on the left. The sidebar shows them as parent and child. Navigate back up the tree anytime with keyboard shortcuts.

Pinned nodes take this further. When a node is pinned, all navigation from it creates children, not just Cmd+Click. Pin your project’s docs landing page and every link you follow branches automatically, building a research tree without any extra effort.

WebView management

A browser can’t keep unlimited WebViews in memory. Tree uses an LRU cache that caps active WebViews at a configurable maximum (default 4):

WebKit/WebViewManager.swift

private func evictIfNeeded() {
 while activeWebViews.count >= maxActive {
 if let toEvict = accessOrder.first(where: { !visibleNodes.contains($0) }) {
 deallocate(toEvict)
 } else if let toEvict = accessOrder.first {
 deallocate(toEvict)
 } else {
 break
 }
 }
}

The eviction prefers invisible nodes. If both panels are showing WebViews, those are protected: the cache evicts the least recently accessed node that isn’t currently on screen. Only when everything is visible does it fall back to strict LRU order.

Before eviction, the WebView’s full browsing state gets persisted to disk.

State persistence

WebKit exposes interactionState, an opaque object that captures a WebView’s entire browsing state: history stack, scroll position, form data. Tree archives this to Application Support whenever a WebView is evicted, the app backgrounds, or the user closes a panel:

WebKit/WebViewStateManager.swift

func saveState(of webView: WKWebView, for nodeId: UUID) {
 guard let state = webView.interactionState else { return }

 let data = try NSKeyedArchiver.archivedData(
 withRootObject: state,
 requiringSecureCoding: false
 )
 let fileURL = stateFileURL(for: nodeId)
 try data.write(to: fileURL, options: .atomic)
}

When a node is revisited and needs a fresh WebView, the manager restores the archived state instead of loading the URL from scratch. The page appears exactly where the user left it: same scroll position, same back/forward history. The user never notices the WebView was recycled.

A cleanup pass on launch removes orphaned state files for nodes that no longer exist in the database.

The sidebar is a recursive SwiftUI tree. Session roots (nodes with no parent) appear at the top level. Each node with children renders as a DisclosureGroup with nested TreeNodeRow views:

Views/SidebarView.swift

@Query(filter: #Predicate<TreeNode> { $0.parent == nil })
private var sessionRoots: [TreeNode]

private var displayedNodes: [TreeNode] {
 let nodes = navigationState.contextualRoot?.children ?? sessionRoots
 return nodes.sorted { a, b in
 if a.isPinned != b.isPinned {
 return a.isPinned
 }
 return a.sortOrder < b.sortOrder
 }
}

Pinned nodes sort to the top. Below that, manual sort order (set via drag-and-drop reordering) controls the sequence.

Drag-and-drop also handles reparenting: drag a node onto another node and it becomes a child. The drop handler checks for cycles (you can’t drop a parent onto its own descendant) by walking up the target’s ancestry chain:

private func isDescendant(of ancestorId: UUID) -> Bool {
 var current: TreeNode? = node.parent
 while let parent = current {
 if parent.nodeId == ancestorId { return true }
 current = parent.parent
 }
 return false
}

Focus mode

A deep tree gets unwieldy. Focus mode lets you temporarily treat any node as the root: the sidebar shows only its children, and a breadcrumb bar at the top shows the path from the true root:

func focusBreadcrumbs() -> [TreeNode] {
 guard let root = contextualRoot else { return [] }
 var path: [TreeNode] = []
 var current: TreeNode? = root
 while let node = current {
 path.insert(node, at: 0)
 current = node.parent
 }
 return path
}

Click any breadcrumb to re-focus at that level, or click the home icon to unfocus entirely. The focus state persists across restarts via UserDefaults.

Ad blocking

Rather than building content blocking from scratch, Tree loads the full Ghostery browser extension via WebKit’s WKWebExtensionController API. Every WebView gets the extension controller attached to its configuration:

configuration.webExtensionController = WebExtensionManager.shared.controller

This gives you enterprise-grade tracker and ad blocking with zero maintenance. Ghostery handles its own filter list updates, UI (accessible via a toolbar popover), and persistent storage. The browser just loads the extension at startup and grants permissions.

Tree navigation is keyboard-driven. The tree structure maps to directional shortcuts:

Shortcut	Action
`Cmd+Shift+Up`	Go to parent node
`Cmd+Shift+Down`	Go to first child
`Cmd+Shift+Left`	Previous sibling
`Cmd+Shift+Right`	Next sibling
`Cmd+Shift+J`	Quick search overlay
`Cmd+T`	New session
`Cmd+W`	Close panel
`Cmd+Shift+W`	Prune node and subtree

The spatial metaphor works because the sidebar is a tree. Up means parent. Down means child. Left and right move between siblings. Once the mapping clicks, you rarely need the mouse for navigation.

Session restoration

On launch, the app restores the last active primary and secondary nodes, plus the focus state, from UserDefaults:

func restoreLastSession(from nodes: [TreeNode]) {
 let defaults = UserDefaults.standard
 let primaryIdString = defaults.string(forKey: Self.lastPrimaryKey)
 let secondaryIdString = defaults.string(forKey: Self.lastSecondaryKey)
 let contextualRootIdString = defaults.string(forKey: Self.contextualRootKey)

 // Match UUIDs to loaded TreeNodes and restore state...
}

Combined with WebView state persistence, a restart puts you back exactly where you were: same pages, same scroll positions, same split view layout, same focused subtree.

What it replaces

The thing it replaces isn’t another browser. It replaces the workflow of opening a dozen tabs, losing track of which ones are related, and eventually closing them all because you can’t remember what you were doing.

With Tree, the context is the structure. A research session about some API has the docs page as the root, the getting started guide as one branch, the API reference as another, and Stack Overflow answers as leaves. Close the app, come back tomorrow, and the tree is still there, with every page exactly where you left it.

Rate limiting with a single Lua script

Tue, 10 Feb 2026 09:49:00 +0200

We needed rate limiting across all our services: FastAPI, Django, you name it. The usual approach is to grab a library, wire it up per-framework, and accept the slight differences in behavior between them. We went a different way: one Lua script that runs atomically in Redis, with everything else being thin wrappers around it.

This is how the rate limiting system in application-kit works, including per-project overrides, element-based counting, and a monitor mode for gradual rollouts.

The problem with two rate limiters

Early on, we had two Lua scripts: one for basic path-based rate limiting and another for per-project limits with override support. They did almost the same thing but diverged just enough to be annoying. Bug fixes had to land in two places. Behavior wasn’t always consistent. The classic duplication trap.

The fix was to collapse both into a single script (PROJECT_RATE_LIMITER_LUA) with optional parameters that fall back to sensible defaults. No override key? It behaves like a simple limiter. Pass one? It checks per-project overrides atomically.

One Lua script to rule them all

The core idea: every rate limit check is a single atomic Redis operation. No round-trips, no race conditions between reading the count and incrementing it.

The unified Lua script (simplified)

local rate_limit_key = KEYS[1]
local override_key = KEYS[2]
local default_max_requests = tonumber(ARGV[1])
local default_expiry = tonumber(ARGV[2])
local increment_amount = tonumber(ARGV[3]) or 1

-- Check for per-project override
local max_requests = default_max_requests
local expiry = default_expiry

if override_key ~= "" then
 local override = redis.call('HGETALL', override_key)
 if #override > 0 then
 -- Parse hash fields into max_requests and expiry
 for i = 1, #override, 2 do
 if override[i] == "max_requests" then
 max_requests = tonumber(override[i + 1])
 elseif override[i] == "expiry" then
 expiry = tonumber(override[i + 1])
 end
 end
 end
end

-- Get current count
local count = tonumber(redis.call('GET', rate_limit_key)) or -1
local is_over_limit = 0

if count == -1 then
 -- First request in this window
 redis.call('SET', rate_limit_key, increment_amount)
 redis.call('EXPIRE', rate_limit_key, expiry)
 count = increment_amount
 if count > max_requests then
 is_over_limit = 1
 end
elseif count + increment_amount > max_requests then
 is_over_limit = 1
else
 count = redis.call('INCRBY', rate_limit_key, increment_amount)
end

local ttl = redis.call('TTL', rate_limit_key)
return {is_over_limit, count, ttl, max_requests}

The script takes two keys and three arguments:

KEYS[1]: the counter key (e.g. ratelimit:/api/search:1:42)
KEYS[2]: the override key (empty string means “no overrides”)
ARGV[1..3]: default max requests, expiry in seconds, and increment amount

Everything after that is Redis doing its thing. One EVALSHA, one atomic operation, four values back.

Redis key layout

The key format puts the endpoint first and IDs at the end:

Counter: ratelimit:{endpoint}:{org_id}:{project_id}
Override: ratelimit_override:{endpoint}:{org_id}:{project_id}

This ordering matters. Endpoint names can contain colons (like api:v1:search), so we parse keys from the right: the last two segments are always org_id:project_id. The parse_override_key() function handles this reliably, stripping the known prefix and extracting integers from the tail.

Counters auto-expire via Redis TTL. When the window ends, the key vanishes and the next request starts fresh.

Per-project overrides

The override system uses Redis hashes. An override key like ratelimit_override:/api/search:1:42 stores:

max_requests = "500"
expiry = "120"

Because the Lua script checks for overrides inside the same atomic call, there’s no window where a request could slip through with stale limits. If an override exists, it replaces the defaults. If it doesn’t, the defaults apply. Zero ambiguity.

Overrides have their own optional TTL (independent of the rate limit window), so you can set a temporary elevated limit that auto-expires:

Setting a temporary override

await set_rate_limit_override(
 redis, org_id=1, project_id=42,
 endpoint="/api/search",
 max_requests=500, expiry=120,
 override_ttl=86400 # Override expires in 24h, limit window is still 120s
)

Management functions (set, get, delete, list, clear) are provided but designed for admin/provisioning services. Not every consumer needs them.

Element-based counting

Not all requests are equal. A distance matrix call with 10 origins and 5 destinations should count as 50 elements, not 1 request.

The increment_amount parameter in the Lua script makes this trivial. Instead of INCR, we use INCRBY:

Element-based rate limiting

# Request: 10 origins x 5 destinations = 50 elements
rows, cols = len(request.origins), len(request.destinations)

await apply_element_rate_limit(
 request, response,
 max_requests=1000, # 1000 elements per minute
 expiry=60,
 increment_amount=rows * cols, # This request costs 50
 redis_client=redis,
)

Element counters use a key_suffix (default: /elements) to maintain separate counters from regular request counting. Same endpoint, two independent limits:

ratelimit:/api/matrix:1:42 → request counter
ratelimit:/api/matrix:/elements:1:42 → element counter

Monitor mode

Rolling out rate limits on existing APIs is nerve-wracking. You want to know who would be affected before actually blocking anyone.

Monitor mode (RATE_LIMIT_MODE=monitor) runs the full rate limit logic (counting, checking, setting headers) but never returns a 429. Instead, it tags the Datadog root span:

Monitor mode behavior

if result.is_over_limit and mode == RateLimitMode.monitor:
 span = tracer.current_root_span()
 if span:
 span.set_tag("ratelimit.over_limit", endpoint_path)

Clients still see RateLimit-Remaining: 0 in response headers, so they can self-throttle if they’re polite. But the server won’t enforce it.

Three modes, set via the Bender manifest:

Mode	Counts	Headers	Blocks	Datadog tag
`on`	Yes	Yes	Yes (429)	No
`off`	No	No	No	No
`monitor`	Yes	Yes	No	Yes

The setting is read dynamically on every request. No restart needed to switch modes.

Framework wrappers

FastAPI: three entry points

FastAPI gets the most complete support with three ways to rate limit:

1. ProjectRateLimiter: the recommended default. Uses dependency injection, extracts project identity from the authenticated request, supports overrides and monitor mode:

FastAPI ProjectRateLimiter

@router.get(
 "/search",
 dependencies=[Depends(ProjectRateLimiter(max_requests=100, expiry=60))],
)
async def search():
 ...

It follows the template method pattern: make_key() and make_override_key() can be overridden without touching the rate limit logic itself:

Custom rate limiter via subclassing

class GeoRateLimiter(ProjectRateLimiter):
 def make_key(self, request: Request) -> str | None:
 region = request.headers.get("X-Region", "default")
 base = super().make_key(request)
 return f"{base}:{region}" if base else None

2. PathRateLimiter: simpler, no project isolation. Useful for public or webhook endpoints where there’s no authenticated project:

FastAPI PathRateLimiter

@router.post(
 "/webhook",
 dependencies=[Depends(PathRateLimiter(max_requests=10, expiry=60))],
)
async def webhook():
 ...

3. Programmatic API: apply_rate_limit() and apply_element_rate_limit() for when limits depend on request content:

Runtime-determined limits

async def process_batch(request: Request, response: Response):
 batch_size = len(request.json()["items"])
 await apply_element_rate_limit(
 request, response,
 max_requests=500, expiry=60,
 increment_amount=batch_size,
 redis_client=redis,
 )

Django: decorator-based

Django uses a @rate_limit decorator that auto-detects sync vs. async views:

Django rate limiting

@authenticate_key()
@rate_limit(max_requests=100, expiry=60)
def my_view(request):
 return JsonResponse({"status": "ok"})

The endpoint name is auto-detected from the URL pattern (e.g. /api/v1/datasets/<int:dataset_id>/search). Django middleware catches RateLimitExceeded exceptions and returns a 429 with the proper rate limit headers.

Both frameworks share the same Lua script, the same key generation functions, and the same result handling logic. The wrappers are genuinely thin.

The result object

Every rate limit check produces a RateLimitResult:

RateLimitResult

@dataclass(frozen=True)
class RateLimitResult:
 is_over_limit: bool
 request_count: int
 ttl: int
 max_requests: int

 @property
 def remaining(self) -> int:
 return max(0, self.max_requests - self.request_count)

 def to_headers(self) -> dict[str, str]:
 return {
 "RateLimit-Limit": str(self.max_requests),
 "RateLimit-Remaining": str(self.remaining),
 "RateLimit-Reset": str(self.ttl),
 }

Frozen dataclass, immutable, with a convenience method for standard rate limit headers. Every response (except when mode is off) gets these three headers so clients can implement their own backoff.

What made this work

A few design decisions that paid off:

Atomic override resolution. The Lua script checks overrides inside the same call that does the counting. No separate Redis roundtrip means no race condition between “check the override” and “apply the limit.”

Optional parameters with backwards-compatible defaults. override_key="" and increment_amount=1 mean the same script handles simple path limiting, per-project limiting, and element counting. No script duplication.

Dynamic configuration. get_rate_limit_mode() calls into Bender on every request. No cached settings, no restarts. Switch from monitor to on when you’re confident.

Separate counters via key suffix. Element counting doesn’t interfere with request counting. Same endpoint, independent limits, same Lua script.

The whole thing is tested with fakeredis[lua], which actually executes the Lua script, so the tests are as close to production behavior as you can get without a real Redis.

Building a GPU map renderer from scratch

Tue, 10 Feb 2026 09:00:00 +0200

Building a vector map renderer in Rust with wgpu. From “everything renders as points” to a full style-driven renderer running on desktop, iOS, and the browser.

Why

We built our maps stack on top of Mapbox GL: the JS SDK, then native SDKs on the C++ core.

For static maps we wrapped the C++ SDK in a Python library (maparazzo). It worked but the C++ renderer leaked memory. OpenGL contexts kept state around after objects were destroyed. We tried pooling Map instances, reusing GL contexts. It helped but never fully solved it.

I wanted to understand every layer of the stack. What if we built a renderer from scratch, something we fully own, no inherited complexity from a massive C++ codebase?

Rust + wgpu seemed right. wgpu abstracts over Metal/Vulkan/DX12/WebGPU so you get native + browser from one codebase. Rust gives you memory safety without a GC. No more mystery leaks.

First pixels

Started from the wgpu-triangle example: a bare-bones triangle on screen. Built the stem by hand: tile fetching, MVT decoding, basic projection. From there I heavily leveraged Claude Code to help push through the harder parts and get it to where it is now.

Everything in one file. main.rs at 1,500 lines: MVT decoder, protobuf parser, renderer, tile fetching, all of it.

The MVT decoder is hand-rolled. No prost, no generated code. Protobuf is simple enough at the wire level: varints, length-delimited fields. The tile format uses a local coordinate grid (4096x4096 per tile) with delta-encoded geometry commands.

First render: every feature drawn as points. Polygons, lines, everything, just dots on screen. But dots in the right places.

Making it a real map

Extracted modules one by one. mvt.rs for tile decoding, renderer.rs for GPU work, tile_loader.rs for fetching, tile_coords.rs for slippy map math.

Added earcutr for polygon triangulation: it converts arbitrary polygons with holes into triangles the GPU can draw. First tessellation attempt produced spikes everywhere. The MVT spec requires rings to be closed but some tiles had implicit closure. Two-pass decode: first pass collects keys/values/feature ranges, second pass builds features with proper ring closing.

Background tile loading with worker threads and mpsc channels. Scroll zoom, mouse pan. Viewport buffer zone to pre-load surrounding tiles so panning feels instant.

Tessellation cache keyed by (TileCoord, layer_index, feature_index): earcut is expensive, no need to redo it when the camera moves.

Moving projection to the GPU

The big architectural decision. Originally the CPU converted every vertex from lon/lat to screen pixels. Camera move = rebuild all vertices = slow.

Flipped it: vertices store world lon/lat, the GPU shader does Web Mercator projection. Camera moves only update a 64-byte uniform buffer. Vertex data stays untouched.

center_mercator, scale, rotation, viewport_scale, aspect, viewport_size

This means mark_camera_dirty() for pan/zoom (cheap uniform upload), mark_geometry_dirty() only when tiles arrive or leave. Night and day difference for interactivity.

Style-driven rendering

A map renderer without style support is just a polygon viewer. Built an expression engine and style evaluator.

The expression engine handles the Mapbox style spec: get, has, match, case, interpolate, step, let/var, coalesce, arithmetic, string ops, comparisons. let/var was critical: Woosmap styles use variable bindings for i18n name resolution.

Filter compilation: both legacy ["==", "key", val] and modern ["==", ["get","key"], val] syntax.

The key insight for z-ordering: iterate style layers, not MVT layers. The style defines the draw order. For each style layer, find matching features across all tiles.

Background color comes straight from the style: just set the wgpu clear color.

Paris at different zoom levels, same style, same renderer, z4 to z16:



z4, country level	z8, region level

z12, city level	z16, street level

Lines

Lines in a GPU renderer are not trivial. A “line” on screen is actually a quad (two triangles) for each segment, with normals perpendicular to the direction. Line joins at vertices need special treatment.

Implemented miter, bevel, and round joins. Miter joins can spike to infinity at sharp angles, so clamp with a miter limit and fall back to bevel. Round joins emit a fan of triangles.

Line normals computed in Mercator space, not screen space, because of the GPU projection design. The shader transforms them correctly.

Icons and sprites

Sprite atlas: load {url}.json (metadata) + {url}.png (texture). Each icon is a region in the atlas with UV coordinates.

SDF (Signed Distance Field) rendering in the fragment shader. One shader handles both SDF sprites and regular RGBA sprites: the is_sdf field on each vertex switches behavior. SDF gives you clean scaling and halos at any size from a single texture.

Icon quads: 4 vertices + 6 indices per point. Screen-aligned: the anchor position projects through the map transform but pixel offsets stay fixed.

@2x sprite support for Retina: load the high-res atlas, halve the pixel sizes.

Text

Text was the biggest single feature. PBF glyph format (same as Mapbox), shelf-packed texture atlas, SDF rendering reusing the icon shader.

GlyphAtlas dynamically grows (1024x1024 initial, doubles when full). SDF distance stored in alpha channel. The glyph shader is literally the icon shader: is_sdf = 2.0 + halo_buff triggers PBF glyph mode with the correct SDF thresholds.

Text shaping is 1:1 codepoint-to-glyph. No GSUB/GPOS, no complex script support yet. Works for Latin, CJK, Cyrillic. Arabic and Thai would need a real shaper like rustybuzz.

Word wrapping, text-anchor (9 positions), text-offset, text-variable-anchor with collision retry, text-radial-offset, text-justify.

Collision detection

Without collision detection, labels pile on top of each other. Built a grid-based system.

Labels grouped by feature: icon + text for the same feature are placed together (all-or-nothing). Groups sorted by (layer_index, symbol-sort-key). Grid cells are checked, if occupied the label gets rejected and fades out.

Variable anchor retry: if a label’s primary anchor collides, try alternatives. Pre-compute screen positions for each anchor variant, test them in order. Phase 1: collect decisions. Phase 2: apply vertex shifts. Two phases to avoid borrow conflicts.

A tour of European cities, all rendered with the same pipeline:



Paris	London	Barcelona

Amsterdam	Berlin	Rome

Vienna	Prague	Budapest

Stockholm	Istanbul	Athens

Lisbon	Copenhagen	Helsinki

Oslo

Performance

Event-driven loop with ControlFlow::Wait. No busy polling. EventLoopProxy waker: worker threads signal the event loop when tiles arrive.

Extract debounce: during active zoom/pan (100ms window), skip full geometry extraction, just update camera uniforms. Once input stops, run the deferred extract. New tile arrivals always trigger extract immediately: you want to see them.

Zoom style patches: instead of re-extracting everything when zoom changes (colors and sizes are zoom-dependent), patch the extracted vertex data in-place. Rewrites only the color/opacity/width fields.

Time-sliced extraction: spread heavy work across multiple frames so the renderer doesn’t stall.

Dependencies optimized in debug builds: [profile.dev.package."*"] opt-level = 2.

Multi-platform

macOS / iOS with UniFFI

UniFFI generates Swift bindings from Rust. MapEngine wraps wgpu in a Mutex, exposes methods like draw_frame(), set_center(), set_zoom().

The Metal surface is created from a raw CAMetalLayer pointer. MSAA disabled in the FFI path because MTKView’s MSAA resolve triggers a size mismatch with wgpu. sRGB correction handled by a uniform flag: fragment shaders apply sRGB-to-linear when rendering to MTKView’s sRGB surface.

HttpClientDelegate callback interface: Swift injects a URLSession-backed implementation so network requests go through the platform’s native stack.

xcframework build: aarch64-apple-darwin, x86_64-apple-darwin, aarch64-apple-ios, aarch64-apple-ios-sim.

Browser with WASM

wasm32-unknown-unknown target, wasm-bindgen for the JS interface. Tile loading uses browser fetch() via web_sys. No threads in WASM so tile extraction is sequential (10-30 tiles is fine without parallelism).

Style/sprite loading is async in the WASM path (browser fetch) vs sync on native (ureq). The renderer has platform-agnostic setters: apply_style_data(), set_sprite_atlas().

web-time crate replaces std::time::Instant everywhere: re-exports std on native, performance.now() on WASM.

Globe

Optional 3D sphere projection at low zoom (toggle with G key). The shader branches on a globe_mix uniform: above 0.5 it projects lon/lat onto a unit sphere, rotated by center latitude and map rotation.

Back-face discard in the fragment shader: globe_rz < 0.0 hides the back of the sphere. No depth buffer needed.

Polygon subdivision for the globe: edges longer than 5 degrees get split recursively. Conforming subdivision: all edges of a triangle get split (not just the longest) so adjacent triangles agree on shared edge vertices. No T-junction cracks.

Google Maps Styler

A declarative styling engine that applies Google Maps-style JSON rules to modify layer colors. HSL transforms: hue, saturation, lightness, invert, gamma. Delta-based: rules shift colors rather than setting absolute values.

Hierarchical feature type matching with a tree structure. "all" walks the entire tree, "road.highway" matches just highway layers.

POI expansion: the styler takes a single POI layer with class metadata and expands enabled classes into individual layers, each with its own filter and icon.

Where it stands

This is not done. It’s not a replacement for Mapbox GL Native, not even close.

233 tests. The basics work: fill, line, background, symbol layers with icons and text. Expressions, collision detection, variable anchor, text wrapping, text justification. Runs on macOS, iOS, and the browser. Globe projection at low zoom.

What’s missing or broken:

Text shaping is naive 1:1 codepoint-to-glyph. Arabic, Thai, Burmese, anything that needs ligatures or reordering doesn’t work. Need rustybuzz or equivalent.
Line rendering has no dashes, no gradients, no line-cap styles beyond basic round.
Symbol placement along lines is basic. No label curving along the road, no smooth rotation interpolation.
Collision detection is coarse. The grid works but it’s not as smart as Mapbox’s: no cross-tile collision, no viewport-edge handling for partially visible labels.
Raster layers not supported at all. No hillshade, no satellite imagery compositing.
Performance is acceptable for interactive use but extraction is still heavier than it should be. Mapbox GL Native has had years of profiling and optimization. We haven’t.
Fill extrusion (3D buildings) not implemented.
Rotation mostly works but some label placement breaks at non-zero bearing.
Expression coverage is incomplete, missing format, image, number-format, to-color and several type conversion operators.

About 15,000 lines of Rust. No C++ dependency. No OpenGL. The foundation is there, the details are not.

Migrating Python containers to Wolfi and uv

Tue, 10 Feb 2026 09:00:00 +0200

Our Python services ran on ubuntu:24.04 with pip-installed dependencies. It worked, but the images carried hundreds of packages we never used, Trivy scans were noisy with OS-level CVEs, and builds were slower than they needed to be. Over a couple of months we migrated to Chainguard’s Wolfi base image and uv for dependency management. This is how it went for the maps service, the one that renders static map tiles with a C++/Python hybrid stack.

Cleaning up dependency management with uv

Before touching the base image, we sorted out the dependency mess. The maps service had drifted. pyproject.toml declared one set of versions, uv.lock referenced another, and a private package (maparazzo) was pinned to a version that didn’t exist on the registry anymore.

The first PR synced the lock file with reality and bumped maparazzo to a published version. The second one removed version constraints from pyproject.toml entirely, keeping only the lock file as the source of truth:

pyproject.toml (before)

dependencies = [
 "application_kit[fastapi]",
 "fastapi>=0.115.0",
 "maparazzo==0.4.0",
 "aiosqlite>=0.20.0",
 "valkey-glide==2.2.5",
 "sniffio>=1.3.1",
 # Dev tools
 "types-redis>=4.6.0",
 "coverage[toml]>=7.6.0",
 "pytest>=8.3.0",
 "pytest-asyncio>=0.24.0",
 "pytest-cov>=6.0.0",
 "respx>=0.22.0",
 "ruff>=0.8.0",
 "mypy>=1.14.0",
]

pyproject.toml (after)

dependencies = [
 "application_kit[fastapi]",
 "maparazzo==2.0.0",
 "aiosqlite",
 "valkey-glide==2.2.5",
 "sniffio",
]

[dependency-groups]
dev = [
 "types-redis",
 "coverage[toml]",
 "pytest",
 "pytest-asyncio",
 "pytest-cov",
 "respx",
 "ruff",
 "mypy",
]

Two changes happened here. First, lower bounds like >=0.115.0 were dropped: the lock file already pins exact versions, so the constraints in pyproject.toml were just noise that could silently go stale. Second, dev tools moved to a proper [dependency-groups] section. This matters for the multi-stage build that comes next: uv sync --no-dev now skips pytest, ruff, mypy, and the rest. Previously they all shipped in the production image.

Running uv lock --upgrade after removing the constraints updated 23 packages in under 3 seconds. That’s the part of uv that makes the biggest day-to-day difference: resolving a 60-package graph is nearly instant.

The Ubuntu Dockerfile

Here’s what we had before:

Dockerfile (Ubuntu-based)

FROM ubuntu:24.04 AS base
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy
ENV UV_PYTHON_INSTALL_DIR=/python
ENV UV_PYTHON_PREFERENCE=only-managed

COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

RUN uv python install 3.12

WORKDIR /usr/src/app
ENV UV_PROJECT_ENVIRONMENT=/usr/src/venv

RUN --mount=type=cache,target=/root/.cache/uv \
 --mount=type=bind,source=uv.lock,target=uv.lock \
 --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
 uv sync --frozen --no-install-project --no-dev

ADD . /usr/src/app

RUN --mount=type=cache,target=/root/.cache/uv \
 uv sync --frozen --no-dev

FROM ubuntu:24.04

ARG DEBIAN_FRONTEND="noninteractive"
ENV PACKAGES="ca-certificates libopengl0 libegl1 curl"

RUN apt-get update && apt-get install --no-install-recommends -y \
 ${PACKAGES} && apt-get clean && apt-get autoclean && apt-get autoremove \
 && rm -rf /var/lib/apt/lists/*

COPY --from=base --chown=python:python /python /python
COPY --from=base --chown=app:app /usr/src/venv /usr/src/venv
COPY --from=base --chown=app:app /usr/src/app /usr/src/app

WORKDIR /usr/src/app
ENV PATH="/usr/src/venv/bin:$PATH"

RUN python ./minify_styles.py

EXPOSE 8000

This was already using uv and multi-stage builds, which was good. But the production stage started from a fresh ubuntu:24.04, which pulls in apt, dpkg, systemd fragments, locales, and hundreds of other packages that a Python web service never touches. The apt-get update && apt-get install dance also meant the image wasn’t reproducible: two builds a week apart could get different package versions.

The COPY --from=ghcr.io/astral-sh/uv:latest was another problem. latest means the uv version changes silently. If a new uv release changes lock file behavior or introduces a bug, your CI breaks with no obvious diff.

Switching to Wolfi

Wolfi is a Linux distribution built specifically for containers. No package manager bloat, no shell login infrastructure, minimal base. Chainguard maintains it and publishes daily CVE-patched images.

The new Dockerfile:

Dockerfile (Wolfi-based)

FROM cgr.dev/chainguard/wolfi-base AS base

RUN apk add --no-cache \
 mesa-egl \
 mesa-gl \
 mesa-gbm \
 libgcc \
 libstdc++ \
 bash \
 ca-certificates-bundle

ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy
ENV UV_PYTHON_INSTALL_DIR=/python
ENV UV_PYTHON_PREFERENCE=only-managed

COPY --from=ghcr.io/astral-sh/uv:0.9.18 /uv /uvx /bin/

RUN uv python install 3.12

WORKDIR /usr/src/app
ENV UV_PROJECT_ENVIRONMENT=/usr/src/venv

RUN --mount=type=cache,target=/root/.cache/uv,id=uv-base \
 --mount=type=bind,source=uv.lock,target=uv.lock \
 --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
 uv sync --frozen --no-install-project --no-dev

COPY . /usr/src/app

RUN --mount=type=cache,target=/root/.cache/uv,id=uv-base \
 uv sync --frozen --no-dev

RUN /usr/src/venv/bin/python ./minify_styles.py


FROM base AS build

RUN --mount=type=cache,target=/root/.cache/uv,id=uv-build \
 uv sync --frozen

ENV PATH="/usr/src/venv/bin:$PATH"
ENV RUFF_CACHE_DIR=/tmp/.ruff_cache
ENV MYPY_CACHE_DIR=/tmp/.mypy_cache
ENV COVERAGE_FILE=/tmp/.coverage


FROM base AS prod

ENV PATH="/usr/src/venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1

USER 65532

EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
 CMD ["/usr/src/app/healthcheck.sh"]

A few things to unpack.

Three stages instead of two

The old Dockerfile had base (build everything) and a final production stage that copied artifacts. The new one has three:

Stage	Purpose	Packages
`base`	Install runtime deps, sync production packages	Runtime only
`build`	Extends base, adds dev deps for testing in CI	Runtime + dev
`prod`	Extends base, sets non-root user and healthcheck	Runtime only

The build stage is what CI uses to run pytest, ruff, and mypy. The prod stage is what ships. Both extend base, so there’s no duplication of the dependency installation step.

The build stage redirects tool caches to /tmp because the app directory is read-only for the non-root user:

Tool cache redirection for CI

ENV RUFF_CACHE_DIR=/tmp/.ruff_cache
ENV MYPY_CACHE_DIR=/tmp/.mypy_cache
ENV COVERAGE_FILE=/tmp/.coverage

Pinned uv version

uv:latest became uv:0.9.18. One line, but it eliminates an entire class of “works on my machine” issues. When we want to upgrade uv, we do it explicitly in a PR.

Non-root production

The prod stage runs as UID 65532, matching the convention from distroless images. The base stage still runs as root (needed for apk add and uv python install), but the production image drops privileges before the entrypoint.

Minification at build time

The old Dockerfile ran python ./minify_styles.py in the production stage at container startup (well, at build time of the final stage, but from a fresh Ubuntu). The new one runs it in the base stage right after syncing dependencies. This means the minified assets are baked in and shared by both build and prod.

Build cache isolation

The --mount=type=cache directives got id=uv-base and id=uv-build suffixes. Without these, Docker can share cache mounts between stages that run concurrently in BuildKit, which causes lock contention.

The s5cmd surprise

After deploying the Wolfi image, the tile generation job broke. It ran a shell script that downloaded s5cmd (an S3-compatible file transfer tool) at runtime:

tile.sh (runtime download, broken)

if ! [ -x "$(command -v s5cmd)" ]; then
 curl -L \
 https://github.com/peak/s5cmd/releases/download/v2.2.2/s5cmd_2.2.2_Linux-64bit.tar.gz \
 > s5cmd.tar.gz
 tar -xzvf ./s5cmd.tar.gz --directory /usr/local/bin/
 rm ./s5cmd.tar.gz
fi

Two problems. First, curl wasn’t in the image anymore; we’d removed it during the Wolfi migration because the application code doesn’t need it. Second, even if curl was there, extracting to /usr/local/bin/ would fail because the container runs as non-root.

The fix was one line in the Dockerfile and deleting the download script:

Installing s5cmd via apk

RUN apk add --no-cache \
 mesa-egl \
 mesa-gl \
 mesa-gbm \
 libgcc \
 libstdc++ \
 bash \
 s5cmd \
 ca-certificates-bundle

Wolfi has s5cmd v2.3.0 in its package repository, newer than the v2.2.2 being downloaded. Installing it at build time is faster, more reliable, and gets security updates through the normal apk channel.

This is the kind of thing that falls out of a migration like this. Runtime downloads of binaries are a pattern that works on fat base images but breaks the moment you tighten things up. And they should break. Downloading unsigned tarballs into a running container is a supply chain risk hiding in plain sight.

Scan results

After the migration, Trivy against the production image:

Severity	Before (Ubuntu)	After (Wolfi)
CRITICAL	0	0
HIGH	3	0
MEDIUM	12	0
LOW	27	0

The Ubuntu image had 42 findings, all in OS packages we never used. The Wolfi image has zero because it only contains packages we explicitly installed. The Python dependency layer had 2 findings in both cases (a starlette version that needed bumping), but the OS layer went from noisy to clean.

Dockle (Dockerfile best-practices linter) reports one false positive: it flags settings.py as a potential credential file because of the filename. Suppressed with dockle -af settings.py.

What made it work

A few things that helped this go smoothly:

uv’s dependency groups. Separating dev tools from runtime dependencies in pyproject.toml is what makes --no-dev actually useful. Before, pytest and ruff shipped in production because they were in the same flat dependency list.

Wolfi’s package repository. Having s5cmd, Mesa GL libraries, and a recent bash available via apk meant we didn’t need to maintain custom installation scripts. The packages are rebuilt daily with CVE patches.

The three-stage pattern. base splitting into build and prod is now our standard for Python services. CI targets build, production targets prod, and both share the same dependency installation.

Pinning uv. Sounds trivial but it removed a real source of non-determinism. uv:latest in a Dockerfile is the same antipattern as pip install package without a version: it works until it doesn’t, and you won’t know why.

Building a maps SDK from a Mapbox GL fork

Sun, 08 Feb 2026 14:00:00 +0200

Rewriting the Woosmap Maps SDK on top of a Mapbox GL JS fork, with a Google Maps-compatible API surface, built with Bun, and an experiment in offscreen rendering.

Why

The Woosmap Maps SDK depended on Mapbox GL JS. Then Mapbox changed the license. Version 2.0 moved to the Business Source License, not open source anymore. The last truly open version was 1.13.1 (December 2020). No more upstream bug fixes, no WebGL2 improvements, no new features. We were stuck on a frozen codebase.

Staying on 1.13 forever wasn’t viable. Browser APIs move, WebGL evolves, security patches stop. Switching to Mapbox 2.x meant accepting the BSL and its usage restrictions. MapLibre forked from the same 1.13 base, but taking a dependency on another project’s roadmap puts you in the same position, just with different maintainers.

So we forked 1.13.1 ourselves, ported the whole thing to TypeScript, and built a Google Maps-compatible API wrapper on top. Our customers use the Google Maps API surface: new woosmap.map.Map(), woosmap.map.Marker, woosmap.map.event.addListener. Switching to a Mapbox-style API would break every integration. One codebase, two API surfaces: Mapbox GL rendering engine underneath, Google Maps API on top.

Architecture

Two layers, cleanly separated.

Mapbox GL fork (src/mapbox-gl/): the rendering engine. WebGL2 context management, vector tile decoding, style evaluation, symbol placement with collision detection, gesture handling. 57 GLSL shaders, vertex and fragment pairs for fill, line, symbol, heatmap, hillshade, raster, extrusion. The full Mapbox style spec expression engine: get, has, match, case, interpolate, step, let/var, coalesce, arithmetic, string ops, comparisons.

Woosmap layer (src/woosmap/): the Google Maps compatibility wrapper. Map, Marker, InfoWindow, OverlayView, Data layer, pane management, gesture handling overlay. Plus service clients: stores, distance matrix, localities autocomplete, datasets, transit, directions.

The Woosmap Map class wraps the Mapbox Map:

src/woosmap/map.ts

import { default as MapBoxMap } from "../mapbox-gl/ui/map";

class Map extends MVCObject {
 _mapboxMap: MapBoxMap;

 constructor(element: HTMLElement, options: MapOptions) {
 // Create the Mapbox map internally
 this._mapboxMap = new MapBoxMap({ container: element, style, ... });
 // Wire up Google Maps-style events, panes, controls
 }
}

Properties fire change events automatically through MVCObject. Set marker.position and position_changed fires. Same pattern Google Maps uses.

The style system

Woosmap customers use Google Maps-style JSON rules to customize maps. Hue shifts, saturation, lightness, gamma correction, invert, applied hierarchically by feature type.

src/woosmap/map-style/map-style.ts

type MapStyler = {
 color?: string; // hex color
 hue?: string; // extract hue, keep lightness/saturation
 saturation?: number; // shift saturation
 lightness?: number; // shift lightness
 gamma?: number; // gamma correction on lightness
 invert_lightness?: boolean;
 visibility?: string; // "on" | "off"
 weight?: number; // line/label weight
};

A LayerRegistry maps Google’s feature type hierarchy (road.highway, poi.park, water) to Mapbox GL layer IDs. Style rules walk the tree, match layers, apply HSL transforms. The result: customers keep their existing style configurations, the renderer is completely different underneath.

Styles load dynamically from the Woosmap API via StyleFetcher, not baked into the bundle. Change your style server-side, maps update without redeployment.

Three bundles

Not every integration needs a full map. Some just need geocoding or store search. Building one monolithic bundle wastes bandwidth.

Bun builds three separate bundles from three entry points:

scripts/builder.ts

// Full SDK: map + services + worker
await Bun.build({
 entrypoints: ["./src/maps.ts", "src/worker.ts", "src/painter.ts"],
 splitting: true, minify: true, sourcemap: "linked",
 loader: { ".glsl": "text" },
});

// Services only: API clients, no WebGL
await Bun.build({ entrypoints: ["src/services.ts"], ... });

// Localities: place autocomplete widget
await Bun.build({ entrypoints: ["src/localities.ts"], ... });

GLSL shaders load as text strings via Bun’s loader plugin. No webpack shader-loader, no build-time compilation. The shader source gets inlined in the bundle and compiled at runtime by WebGL.

Environment configuration (local, staging, production) injects API endpoints at build time through define:

retVal[`Bun.env.API_BASE_URL`] = JSON.stringify(environment.API_BASE_URL);
retVal[`Bun.env.ASSETS_BASE_URL`] = JSON.stringify(environment.ASSETS_BASE_URL);

Worker architecture

Tile processing is expensive: protobuf decoding, geometry clipping, feature indexing. Doing it on the main thread blocks interactions.

The SDK spins up web workers from a blob URL constructed at load time:

src/maps.ts

const scriptURL = import.meta.url.replace("maps.js", "worker.js");
const response = `import "${scriptURL}";`;
const blob = new Blob([response], { type: "application/javascript" });
const workerUrl = URL.createObjectURL(blob);

Workers handle tile deserialization and geometry processing. The main thread handles rendering, events, and camera updates. Communication is message-based. Tiles arrive asynchronously, trigger geometry extraction, and the painter picks them up on the next frame.

Offscreen rendering experiment

The maps-offscreen.ts entry point pushes further: move the entire WebGL renderer into a worker using OffscreenCanvas + comlink.

The OnscreenMap lives on the main thread: handles DOM events, gesture recognition, camera state. The OffscreenMap lives in a worker: owns the WebGL context, runs the painter, processes tiles. Camera updates flow from main thread to worker, rendered frames appear on the transferred canvas.

src/woosmap/onscreen-map.ts

export class OnscreenMap {
 offscreenMap: Remote<OffscreenMap>;
 handlers: HandlerManager;
 transform: Transform;

 constructor(element: HTMLElement, options?: object) {
 this.canvas = document.createElement("canvas");
 const offscreen = this.canvas.transferControlToOffscreen();
 // Transfer canvas to worker, all GL happens there
 }
}

The tricky part is camera update batching. Every pan/scroll event would trigger a worker message, overwhelming the channel. The solution: batch camera updates with a timer and requestAnimationFrame, send at most one update per frame interval. Immediate sends for significant zoom changes, deferred sends for continuous panning.

Still experimental. Safari’s OffscreenCanvas support has quirks, and synchronizing the transform state between threads without jank requires careful debouncing.

Symbol placement

Text labels on maps are deceptively hard. The Mapbox GL fork handles:

Glyph atlasing: PBF glyph format, dynamically growing texture atlas (1024x1024 initial, doubles when full), SDF rendering for clean scaling at any size
Collision detection: grid-based spatial index, labels grouped by feature (icon + text placed together, all-or-nothing), sorted by layer priority and symbol-sort-key
Cross-tile continuity: CrossTileSymbolIndex maintains label placement across tile boundaries so labels don’t pop in and out when panning
Variable anchor: if a label’s primary anchor collides, try alternatives. Pre-compute screen positions for each variant, test in order
CJK text: local ideograph font families bypass the glyph server for Chinese, Japanese, Korean characters. Properly sized using browser font metrics

The symbol size evaluation is complex. text-size gets evaluated at five different zoom levels per feature for layout, collision boxes, line placement, and shader interpolation.

One practical fix we added: a maximum label count per bucket to prevent repeated line labels from overflowing. Road names on long highways would generate hundreds of labels. Now they’re capped.

Services integration

The Woosmap layer isn’t just rendering. It includes API clients for the full Woosmap platform:

StoresService: store locator with search, autocomplete, radius queries
DistanceService: matrix routing, isochrones, multiple travel modes (driving, walking, cycling)
LocalitiesService: place autocomplete supporting 70+ languages
DatasetService: query and overlay custom geospatial datasets
TransitService: public transportation routing
DirectionsService: turn-by-turn directions with renderer

Each service handles its own error taxonomy (INVALID_REQUEST, REQUEST_DENIED, OVER_QUERY_LIMIT, MAX_ELEMENTS_EXCEEDED) and throws typed errors (BadRequestError, ForbiddenError, TooManyRequestsError).

Overlays (StoresOverlay, DatasetsOverlay, DirectionsRenderer) connect service responses directly to map layers. Fetch stores, get markers. Compute a route, get a polyline.

Testing

97 test files. Vitest with Playwright for browser testing. The WebGL context needs a real browser, not jsdom.

Handler tests cover the full gesture matrix: mouse drag pan, scroll zoom, keyboard navigation, touch zoom/rotate, double-click zoom, box zoom. Each handler type has its own test file, testing event sequences and resulting transform states.

Style-spec tests validate the expression engine: filter compilation, color space conversions (RGB, HSL, LAB), interpolation, type coercion. These are ported from the Mapbox GL test suite and adapted for the TypeScript rewrite.

Visual regression tests capture screenshots and diff against baselines. Useful for catching rendering changes across the 57 shaders.

bun run test:browser # Vitest + Playwright (Chromium)
npx biome check src # Lint

What’s there

The basics are solid. Fill, line, symbol, raster, background, heatmap, hillshade, fill-extrusion layers all render. The full style expression engine works. Collision detection, variable anchor, text wrapping, cross-tile symbol continuity. Markers with labels, info windows, data layer with per-feature styling. All five service clients.

536 TypeScript files, 57 GLSL shaders. The Google Maps API compatibility layer means existing Woosmap integrations can switch renderers without code changes.

What’s still rough:

Offscreen rendering works but the camera synchronization needs more tuning for smooth panning on slower devices
Complex text shaping is 1:1 codepoint-to-glyph. Arabic, Thai, and other complex scripts need a proper shaper
Bundle size could be smaller with better tree-shaking of the Mapbox GL fork
Line rendering has no dash patterns or gradients yet

About 5MB of TypeScript source. The output bundles are significantly smaller after minification and tree-shaking.

A map style compiler in Python

Sat, 07 Feb 2026 16:00:00 +0200

BAM!

Generating Mapbox GL GL style JSON from Python code instead of editing 3,000-line JSON files by hand. Hierarchical style resolution, a Django-style filter DSL, injection-based layer ordering, and a Go sprite generator called from Python via ctypes.

The problem with style JSON

A Mapbox GL GL style is a single JSON file that describes everything about how a map looks. Background color, road widths at every zoom level, label fonts, icon placement, landcover tints, building extrusion heights. All of it.

A real production style runs to thousands of lines. Ours had over 150 layers. Roads alone need six layer variants each (tunnel casing, tunnel fill, road casing, road fill, bridge casing, bridge fill), and there are eight road classes. That’s 48 road layers before you count labels, oneway arrows, or rail.

Editing this by hand is slow and error-prone. Change the motorway color and you need to find it in six places. Add a new road class and you need to insert layers at exactly the right positions between tunnels and bridges. Reorder a layer and you break z-ordering for everything above it.

Python instead of JSON

The style compiler replaces hand-edited JSON with Python code. You define layers, colors, filters, and style properties in Python. The compiler resolves everything and emits a Mapbox GL GL style JSON file.

python -m map_style --style=streets -o style/streets_classic.json

The output is a standard Mapbox GL GL style: the renderer doesn’t know or care that it was generated. But the source is modular, typed, and DRY.

Layer definitions

A road is defined once. ClassicRoadDataNode generates all six variants:

streets_classic/roads.py

ClassicRoadDataNode(
 "motorway",
 classes=["motorway"],
 feature_type="road.highway.motorway",
 extra_filter=~Q(ramp__eq=1),
 casing_feature_type="road.highway.motorway",
)

This produces:

tunnel_motorway_casing: tunnel stroke
tunnel_motorway: tunnel fill
road_motorway_casing: road stroke
road_motorway: road fill
bridge_motorway_casing: bridge stroke
bridge_motorway: bridge fill

Each variant gets the correct brunnel filter automatically. Tunnel layers filter on brunnel == "tunnel", bridge on brunnel == "bridge", normal roads on brunnel not in ["bridge", "tunnel"].

Rail works the same way: ClassicRailDataNode generates tunnel, road, and bridge variants with optional hatching layers for the cross-ties pattern.

Injection points

With 150+ layers, ordering is everything. Roads must render above landcover. Bridges must render above buildings. Labels above everything. Getting one layer out of place breaks the visual hierarchy.

Instead of maintaining an explicit ordered list, the system uses injection points: named slots that define the rendering order:

generator/data.py

class InjectionsNames(Enum):
 TUNNEL_ROAD_CASING = "tunnel_road_casing"
 TUNNEL_ROAD = "tunnel_road"
 ROAD_AREA = "road_area"
 ROAD_CASING = "road_casing"
 ROAD = "road"
 BUILDING = "_building_injection"
 BRIDGE_ROAD_CASING = "bridge_road_casing"
 BRIDGE_ROAD = "bridge_road"
 BUILDING_3D = "building_3d"
 LABEL = "labels"
 POI_SYMBOL = "poi_symbol"

Layers insert themselves before a named injection point:

stack.add_layer_before(InjectionsNames.ROAD, Layer("road_motorway", ...))
stack.add_layer_before(InjectionsNames.BRIDGE_ROAD, Layer("bridge_motorway", ...))

The Stack maintains the order. Injection points themselves are stripped from the final output. They’re scaffolding, not layers. Adding a new road class doesn’t require knowing the index of every other layer. You just say “this goes in the ROAD group” and the system handles placement.

Filter DSL

Mapbox GL filters are nested JSON arrays. Writing them by hand is unreadable:

["all", ["==", ["get", "class"], "motorway"], ["!", ["in", ["get", "brunnel"], ["literal", ["bridge", "tunnel"]]]]]

The compiler uses Django-style Q objects instead:

Q(class__eq="motorway") & ~Q(brunnel__in=["bridge", "tunnel"])

& means AND, | means OR, ~ means NOT. The ExpressionSet class compiles Q trees to Mapbox GL expressions:

generator/filters.py

OPERATORS = {
 "eq": "==", "neq": "!=",
 "gt": ">", "gte": ">=",
 "lt": "<", "lte": "<=",
}

# Q(class__in=["motorway"]) becomes:
# ["in", ["get", "class"], ["literal", ["motorway"]]]

# Q(brunnel__exists=False) becomes:
# ["!", ["has", "brunnel"]]

Filters compose naturally. The ramp filter for motorway links:

Q(class__in=["motorway"]) & Q(ramp__eq=1)

The non-ramp filter for main motorways:

Q(class__in=["motorway"]) & ~Q(ramp__eq=1)

The Case class builds conditional expressions for layers that vary by feature class, like different icon colors per POI type:

case = Case({"restaurant": Q(class__eq="restaurant"), "hospital": Q(class__eq="hospital")})
case.resolve({"restaurant": "#e74c3c", "hospital": "#3498db"}, default="#666")
# ["case", ["==", ["get", "class"], "restaurant"], "#e74c3c",
# ["==", ["get", "class"], "hospital"], "#3498db", "#666"]

Hierarchical style resolution

A StyleTree organizes paint and layout properties in a dot-separated hierarchy. Child selectors inherit from parents:

streets_classic/style.py

styles = {
 "road": StyleElement(paint=LinePaint(line_cap="round")),
 "road.highway": StyleElement(paint=LinePaint(line_color=ROAD_HIGHWAY_FILL_COLOR, line_width=4)),
 "road.highway.motorway": StyleElement(paint=LinePaint(line_width=6)),
}

Resolving road.highway.motorway walks the tree from root to leaf, collecting all StyleElement instances along the path. Properties merge: the motorway gets line_cap="round" from road, line_color="#fc8" from road.highway, and line_width=6 from its own node.

generator/styler.py

def resolve(self, selector_path: str) -> list[StyleElement]:
 nodes = self.get_node_path(selector_path)
 elements: list[StyleElement] = []
 for node in nodes:
 elements += node.elements
 return elements

Same pattern as CSS cascade, but explicit. You see exactly what inherits from where by reading the selector paths. No specificity wars.

Colors

The color module parses hex, RGB, RGBA, HSL, and HTML color names. More usefully, it does HSL-based transformations:

generator/color.py

Color("#fc8").darken(-0.3).to_rgba()
# Shifts lightness down by 30%, used for deriving label colors from fill colors

All colors live in one file per style:

streets_classic/colors.py

BACKGROUND_COLOR = "rgba(252, 247, 229, 1)"
WATER_COLOR = "rgba(134, 204, 250, 1)"
ROAD_HIGHWAY_FILL_COLOR = "#fc8"
ROAD_HIGHWAY_STROKE_COLOR = "#e9ac77"
ROAD_MINOR_FILL_COLOR = "#fff"
ROAD_MINOR_STROKE_COLOR = "hsl(35, 6%, 80%)"

Change a color constant, regenerate, and every layer using it updates. No grep through JSON.

Pydantic models for the style spec

Every Mapbox GL layer type has a corresponding Pydantic v2 model with strict validation:

generator/mapbox_style.py

class LinePaint(BaseModel):
 model_config = ConfigDict(populate_by_name=True)

 line_color: str | list[Any] | None = Field(None, alias="line-color")
 line_width: float | list[Any] | None = Field(None, alias="line-width")
 line_opacity: float | list[Any] | None = Field(None, alias="line-opacity")
 line_dasharray: list[float] | None = Field(None, alias="line-dasharray")
 line_cap: str | None = Field(None, alias="line-cap")
 # ...

The alias parameter handles the mismatch between Python identifiers (underscores) and Mapbox GL property names (hyphens). line_color in Python becomes line-color in JSON output.

Pydantic catches type errors at definition time, not when the renderer fails to parse the JSON. Pass a string where a number is expected and you get an immediate error.

Sprite generation

Map icons (POI markers, highway shields, oneway arrows) are packed into a sprite sheet: a single PNG with a JSON manifest mapping icon names to pixel regions.

The sprite generator is written in Go for performance. Python calls it through ctypes:

generator/sprite.py

lib = ctypes.CDLL(lib_path)
lib.GenerateSprite.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int]
lib.GenerateSprite.restype = ctypes.c_char_p

result = lib.GenerateSprite(
 icon_library_path.encode(),
 output_path.encode(),
 pixel_ratio, # 1 for sprite.png, 2 for sprite@2x.png
)

The Go library handles SVG rasterization, SDF (signed distance field) generation for runtime-colorable icons, bin-packing into a texture atlas, and both 1x and 2x output.

Template-based icon names resolve from filter context. When a layer uses icon-image: "{class}_15", the compiler extracts which class values the filter allows and registers each concrete icon (restaurant_15, hospital_15, etc.) for sprite generation.

POI metadata

POI layers carry metadata beyond what Mapbox GL needs. The Woosmap API uses it to support dynamic styling at runtime:

class POIMetadata:
 filter: list[Any]
 symbol_color: str
 text_color: str
 icon: str
 visible: bool

This metadata embeds in the layer’s metadata field (which Mapbox GL ignores but passes through). The API can query which POI categories exist, their colors, and their visibility, without parsing the style expressions.

Feature types follow a hierarchy: poi.school, poi.sports_complex, transit.station.rail. The LayerRegistry maps these to the concrete Mapbox GL layers they affect, enabling Google Maps-style feature type styling through the Woosmap SDK.

Multiple styles

The architecture supports multiple style variants from the same infrastructure. streets_classic is the production style. streets_satellite reuses the same layer definitions, road nodes, and filter DSL with a different color palette tuned for satellite imagery overlay: higher contrast labels, semi-transparent fills, no background color.

Adding a new style variant means writing a new color file and adjusting a few layer properties. The layer structure, injection points, and filter expressions stay the same.

What this gets you

The streets_classic style has ~150 renderable layers. The Python source is organized across about a dozen files: colors, roads, mapping, style tree, POI definitions. Each file is under 300 lines.

The equivalent hand-maintained JSON was one file, thousands of lines, with duplicated filter expressions and no inheritance. Every road color change required finding and editing six layer definitions. Every new road class required inserting layers at the correct positions relative to all other layers.

Now it’s: change a constant in colors.py, run the compiler, get a valid style JSON. Type-checked, with the layer ordering handled by injection points instead of manual index management.

Tunes

Tue, 01 Jul 2025 10:31:38 +0200

iTunes Match worked great for years: you uploaded your library, Apple matched what it could, and you had access to everything from any device. But Apple is clearly moving everyone toward Apple Music, and iTunes Match has been slowly rotting. The writing was on the wall.

I didn’t want to subscribe to Apple Music. I wanted to keep my library, my ratings, my play counts, my playlists, all the metadata accumulated over 20 years. So I built my own streaming setup: a Go backend that serves the library over HTTP, and SwiftUI apps that play it on iOS, macOS, and tvOS.

Exporting the library

Apple Music (formerly iTunes) can export the library as an XML plist file. It’s a flat dictionary of tracks keyed by ID, with every field you’d expect: name, artist, album, play count, rating, date added, file location, and about 60 more.

A track entry in the exported XML

<key>1234</key>
<dict>
 <key>Track ID</key><integer>1234</integer>
 <key>Name</key><string>Windowlicker</string>
 <key>Artist</key><string>Aphex Twin</string>
 <key>Album</key><string>Windowlicker</string>
 <key>Total Time</key><integer>381573</integer>
 <key>Location</key><string>file:///Music/Aphex%20Twin/Windowlicker/01%20Windowlicker.m4a</string>
</dict>

The Go struct that maps to this uses plist tags for XML parsing and gorm tags for database storage, with every optional field as a pointer for proper nil handling:

Track model (abbreviated)

type ITunesTrack struct {
 TrackID *int `plist:"Track ID" json:"track_id" gorm:"primaryKey"`
 Name *string `plist:"Name" json:"name" gorm:"index"`
 Artist *string `plist:"Artist" json:"artist" gorm:"index"`
 Album *string `plist:"Album" json:"album" gorm:"index"`
 TotalTime *int `plist:"Total Time" json:"total_time"`
 Location *string `plist:"Location" json:"location"`
 Rating *int `plist:"Rating" json:"rating" gorm:"index"`
 PlayCount *int `plist:"Play Count" json:"play_count"`
 DateAdded *time.Time `plist:"Date Added" json:"date_added" gorm:"index"`
 // ... 60+ more fields
}

The initial version parsed the XML on every startup using howett.net/plist. That worked fine for a 15,000-track library (about 2 seconds of parsing). But it eventually moved to SQLite with GORM for persistence, which also enabled FTS5 full-text search and pre-aggregated album caches.

The Go backend

The server is straightforward. A LibraryService holds the database, artwork cache, rate limiter, and various sub-services. Routes are plain net/http handlers with a JWT auth middleware:

Route setup (abbreviated)

// Auth routes (no middleware)
http.HandleFunc("/auth/login", authHandlers.HandleLogin)
http.HandleFunc("/auth/register", authHandlers.HandleRegister)

// Library routes (auth required)
http.Handle("/tracks", authMiddleware(http.HandlerFunc(service.HandleTracks)))
http.Handle("/artists", authMiddleware(http.HandlerFunc(service.HandleArtists)))
http.Handle("/albums", authMiddleware(http.HandlerFunc(service.HandleAlbums)))
http.Handle("/stream/", authMiddleware(http.HandlerFunc(service.HandleStream)))
http.Handle("/artwork/track/", authMiddleware(http.HandlerFunc(service.HandleTrackArtwork)))

The /stream/{id} endpoint serves audio files directly with http.ServeFile, which handles range requests and conditional caching automatically. No need to build a custom streaming protocol. HTTP does the job.

Authentication uses Argon2 for password hashing and session tokens stored in the database. It later grew WebAuthn support for passkey login, and a QR code flow for tvOS (where typing passwords with a remote is painful).

Album artwork

This was the most annoying part. The exported XML doesn’t include artwork. The actual cover images are either embedded in the audio files or live somewhere in Apple’s ecosystem that isn’t accessible after export.

The solution is a three-tier artwork system:

Embedded extraction: read the audio file metadata with github.com/dhowden/tag, pull out the cover image if present, cache it locally
iTunes Store lookup: search Apple’s public API for the album, download the highest resolution artwork available
Manual upload: for albums that don’t match either source

The iTunes Store lookup uses a scoring system to match results. Exact album + artist name match scores highest, partial matches score lower, and anything below a minimum threshold is rejected:

iTunes Store artwork search

func (ls *LibraryService) searchITunesStore(artist, album string, tracks []ITunesTrack) *ITunesSearchItem {
 searchTerm := fmt.Sprintf("%s %s", strings.TrimSpace(artist), strings.TrimSpace(album))

 params := url.Values{}
 params.Set("term", searchTerm)
 params.Set("entity", "album")
 params.Set("media", "music")
 params.Set("limit", "10")

 searchURL := fmt.Sprintf("https://itunes.apple.com/search?%s", params.Encode())
 // ... fetch, decode, find best match by score
}

Apple’s API returns 100x100 thumbnails by default. The URL pattern is predictable (replace 100x100 with 600x600), but not all resolutions exist for all albums. The server tries 600, 512, and 300 with HEAD requests before falling back to the original.

The real constraint is rate limiting. Apple’s Search API starts returning 403s if you hit it too hard. The server uses a channel-based rate limiter that caps requests to 2 per second by default, configurable via TOML config.

The gapless playback problem

This one took a while to figure out. Live albums, DJ mixes, classical recordings: any music that’s supposed to flow continuously between tracks gets a gap when played back on iOS.

The root cause: iOS AVFoundation ignores the iTunSMPB gapless metadata in MP3 files. This is the atom that tells the player how many encoder delay and padding samples to skip. Apple’s own Music app reads it. AVFoundation does not.

The fix is converting MP3s to AAC. The AAC container stores gapless information in a way that AVFoundation actually respects. The server has a built-in transcoder:

Startup conversion

transcoder := tunes.NewTranscoderService(service, tunes.TranscodeOptions{
 Concurrency: 4,
 DeleteSource: false, // Keep original MP3s
})
results, err := transcoder.ConvertAllMP3s(context.Background())

It uses ffmpeg under the hood, runs 4 workers in parallel, keeps the originals, and updates the database paths. A 15,000-track library takes a couple of hours. After that, gapless playback works perfectly.

SwiftUI clients

The app targets iOS, macOS, and tvOS from a single codebase, with platform-specific extensions for navigation and layout. iOS uses tab navigation with a mini-player overlay, macOS uses a split view with sidebar, and tvOS uses focus-based navigation for the remote.

The audio player is built on AVQueuePlayer for seamless track transitions. It pre-buffers the next track 10 seconds before the current one ends, maintaining up to 3 items in the queue at any time.

One hard-won lesson: @Published property updates in background cause iOS to terminate the app for excessive main-thread UI work. The fix is gating all published writes behind a foreground check:

Background-safe state management

struct PlaybackState: Equatable {
 var currentTime: TimeInterval = 0
 var duration: TimeInterval = 0
 var isPlaying = false
 var isLoading = false

 static func == (lhs: PlaybackState, rhs: PlaybackState) -> Bool {
 return Int(lhs.currentTime) == Int(rhs.currentTime) &&
 Int(lhs.duration) == Int(rhs.duration) &&
 lhs.isPlaying == rhs.isPlaying &&
 lhs.isLoading == rhs.isLoading
 }
}

Grouping related properties into structs (instead of individual @Published vars) and implementing coarse equality comparison reduced SwiftUI re-renders significantly. The player only triggers a view update when the second changes, not on every time observer tick.

The API layer uses Combine publishers with automatic 401 retry: if a request fails with an expired token, the service transparently refreshes it and replays the original request. Network monitoring via NWPathMonitor lets the app degrade gracefully when offline, falling back to Core Data for cached data.

What grew out of it

What started as “stream my library from a Raspberry Pi” has accumulated features over time:

Full-text search with SQLite FTS5 across all track metadata
Play history tracking: every play gets logged with duration and completion status, which feeds a yearly replay feature (like Spotify Wrapped)
Lyrics: embedded USLT frames from audio files, plus LRClib API integration for synced timestamps
Offline mode: Core Data persistence with background sync, download management for keeping albums locally
Discogs integration: artist biographies and metadata from a separate database
Vector similarity search: track embeddings for “genius playlists” based on audio features and metadata

It’s not a replacement for Apple Music. It doesn’t have a catalog of 100 million songs. But it plays my music, with my metadata, on all my devices, without a subscription. That was the goal.

I know what you're thinking... on Man-You

プログラマ道 (puroguramā-dō)

What blindness looks like

The testbed problem, named on a SNES

Time to serve

What I do not delegate

Getting oversmarted

Ask Claude to review Claude

Same workflow as with a coworker

The 道 has been written down for thirty years

What I tell people who ask

The 100x engineer, and the unit nobody defines

The RPG item problem

A concrete counter-example

The token economics nobody is pricing in

25 years to a complete machine

Why a rolling inventory at all

What made the bug reachable

Why these tools exist

The complete machine, fading

Devices die from the network now

A worse mainframe, with prettier hardware

What “complete” used to mean

Linux is real, and not enough

What I actually do

Higgins: a neurosymbolic menubar buddy

Why not just a chat wrapper

The Qwen brain

The symbolic side

Sleep

Per-turn recall: closing the loop

Tools

Dates, or: why the model doesn’t do arithmetic

The AppleScript Run button

What grew out of it

Freddie part 2: from tiles to map

Barbapapa is dead

Edge DP

Z14Factor

What got deleted

Painter’s algorithm

The 200GB problem

Runtime MVT concatenation

Teaching the SDK about fill colors

Four repos, one pipeline

The most useless way to port a macOS app

The prior art that made this possible

What Claude came back with

Architecture

First signs of life

Settings and Finder

Tunes

Leela

Under the hood

Text rendering

Layout

IPC

The ycodebuild trick

Honest assessment

Why this matters (a little)

What’s next

When "mostly aligned" stops being enough

Not all dependencies are equal

The illusion

The pattern

Map tiles: from vendor to vertical integration

Buying tiles

Running OpenMapTiles ourselves

Planetiler

Clean-room schema

The style compiler

The spec question

The renderer

Routing: the black box problem

OSRM

Valhalla

Calculon

When replacing a black box makes things better

The ones still early in the cycle

The open source dependency lifecycle

The `ycodebuild` trick