Building the Liquidation Tracker: What Broadcast Engineering Taught Me About Software That Actually Survives
Twenty-some years in broadcast engineering teaches you one thing above all else: systems fail quietly long before they fail loudly. A router that's one relay away from a bad switch. A playout server that's been silently dropping frames for a week before anyone notices the stutter. You learn to distrust anything that "seems to be working fine" and go looking for the failure that hasn't announced itself yet.
I didn't expect that instinct to be the most useful thing I brought to building a financial operations web app. But it was.
Where this started
AllTV runs on paper trails — invoices, cash advances, liquidation reports, recurring bills, check request forms, transmittals between departments. All of it real money, all of it needing approval chains, all of it historically living in spreadsheets and email threads and whoever remembered to follow up. Anyone who's worked adjacent to finance ops in a media company knows this story. It doesn't scale, and it fails exactly the way a badly-maintained transmission chain fails: not all at once, but one missed approval, one lost attachment, one "I thought you signed off on that" at a time.
So we built a web app for it. NestJS API, Postgres underneath, a Next.js dashboard on top. Invoices, cash advances, CRFs, liquidations, recurring bills, transmittals, expenses, reimbursements — the whole lifecycle, with role-based access control so a field employee sees only their own submissions and a supervisor sees the queue that's actually theirs to approve.
That part sounds clean in a paragraph. It was not clean to build.
The bug that was hiding in plain sight
Here's the moment that stuck with me. We'd added a read-only demo account — someone external needed to see the app without being able to touch anything in it. Reasonable ask, seemingly simple feature. We shipped a guard that was supposed to block any non-GET request from that account.
An audit later, going endpoint by endpoint the way you'd walk a transmission path checking every junction, turned up something uglier: that guard was registered globally, but the authentication guard that actually identifies who's
The read-only account could write. It had been able to write the entire time the feature had been "working."
That's the broadcast-engineering pattern showing up exactly where I didn't expect it. Nothing alarmed. No error in the logs. The feature looked deployed and functional from every angle you'd normally check. It took going in and tracing the actual signal path — not trusting that a green light meant a clean signal — to find it. Same audit turned up two more controllers with financial-mutation endpoints that had no role restriction at all, which is its own special kind of quiet failure: not a bug in the guard, just a guard nobody had put there.
We fixed all of it, verified it with a clean build, and — this part matters as much as the fix — didn't stop there.
From "fixed the bug" to "the system shouldn't allow this class of bug"
The honest lesson from that audit wasn't "we had a bug," it was "our permission model was hardcoded decorators scattered across twenty-some controllers, and nobody could look at one place and know what any role could actually do." That's not a bug, that's a design that makes bugs inevitable.
So the next real piece of work was ripping that out for a database-driven permission system — every mutating endpoint checks against a permissions table instead of a hardcoded list, and it fails closed on anything it doesn't recognize, which is the opposite of the default-allow behavior that caused the original problem. A Super Admin can now grant or revoke access per role from an actual admin page instead of needing a code deploy and a redeploy cycle. If I've learned anything from maintaining broadcast automation systems, it's that the fix you want is the one that makes the entire category of failure structurally harder to reintroduce, not just the one instance of it.
The dashboard that caught its own mistake
Not every obstacle was a security hole. Some were just wrong assumptions that real data corrected fast.
We built a cycle-time dashboard for recurring bills — how long does a bill actually take from creation to payment, and where does it get stuck. First version looked great in testing. Then real production data hit it, and the numbers were nonsense: a finished billing month's clean averages were quietly blending together with an active month's still-in-progress numbers, dragging the "average days to pay" figure around in a way that told you nothing true about either month.
That's the kind of thing you don't catch by staring at code. You catch it by looking at the actual output against what you know to be true, the same way you'd catch a router misroute by watching the picture, not by reading the routing table. Scoped the dashboard to one billing period at a time after that, and it started telling the truth.
What "surviving" actually looked like
If there's a theme, it's this: nothing shipped because it looked done. Every change got verified against something real — a clean build, a live database query, an actual end-to-end test with a throwaway record created and deleted afterward, not just "the code compiles so it must be fine." When we added automatic email notifications for supervisors on liquidation submission, we didn't trust the code path until a real email landed in a real supervisor's inbox and they confirmed it. When the transmittal-tracking feature shipped, we found — again, only by checking — that two working backend methods had never actually been wired up to routes, sitting there unreachable behind buttons in the UI that had been quietly 404ing.
The app that's running AllTV's finance operations today handles the full lifecycle — submission through multi-level approval, automated CRF generation, payment tracking, a full audit trail — and it does that reliably not because it was designed perfectly the first time, but because we kept treating "looks like it's working" as a hypothesis to test, not a conclusion to trust.
That's really the whole transferable lesson from twenty years of broadcast systems into eighteen months of building this: the system that survives isn't the one that never breaks. It's the one built by people who never fully believed it wasn't broken somewhere they hadn't checked yet.