> ## Content Index > Fetch the complete content index at: https://blog.melardenio.com/llms.txt > Use this file to discover other available public pages before exploring further. # 41 Invoices Stuck in the Queue, and the Duplicate Check I Should Have Done First - URL: https://blog.melardenio.com/41-invoices-stuck-in-the-queue-and-the-duplicate-check-i-should-have-done-first/ - Published: 2026-11-06T00:21:07.000Z - Updated: 2026-11-06T00:21:07.000Z - Author: Rommell Ardenio - Tags: Software & Homelab, AI & Automation I opened my Gmail and saw a wall of emails tagged **LT-FOR-PROCESSING**. That label means "an invoice arrived and the encoding workflow hasn't touched it yet." There were 41 of them. In broadcast terms, that's a playout queue that has quietly stopped rolling while the on-air light still says everything is fine. Here's what happened, what I changed, and the one dumb thing that took down the fix halfway through. ## The setup Vendor invoices come into a Gmail inbox as PDFs. An n8n workflow on my homelab picks each one up, labels it, sends the PDF to a local Ollama model to extract the fields, and pushes the result into the Liquidation Tracker, the app I built for our cash advances and invoices. Gmail labels serve as the state machine: *FOR-PROCESSING*, then *PROCESSING*, then *PROCESSED*, *ERROR*, or *NO-PDF*. Normally it's fine. Then a batch of about 60 invoices was forwarded at once. ## What actually went wrong Nothing crashed, and that was the problem. The Gmail trigger fired a run for every arrival, and roughly 13 runs overlapped, each taking 12 to 24 minutes because local LLM extraction isn't fast. Some emails got picked up and some didn't. Whatever was left behind just sat there with its label waiting for the only retry the workflow had: a once-a-day "queue sweep." So the design had three gaps: 1\. Processing only happened on the trigger, with a single daily safety net. 2\. Nothing told anyone that the queue was stuck. 3\. Nobody got a summary of what was processed, skipped, or failed. ## Duplicates were costing me ten minutes each When I looked closer, I found something that bothered me more than the backlog. Of the 41 stuck invoices, **8 were already in the Liquidation Tracker**. The workflow would have sent each of them to Ollama, waited about ten minutes for extraction, and only then let the API say "already have this one." The invoice number was already in the email subject the whole time. The workflow even parsed it later on with a simple regex. We were asking a language model to read a PDF to learn something the subject line had told us for free. So I moved the check up front. Before anything goes near Ollama, the workflow now pulls the invoice number from the subject and runs one SQL lookup against the tracker's database. Duplicates get a new label, **LT-DUPLICATE**, and leave the queue right away. I kept that separate from LT-ERROR on purpose. A duplicate isn't a failure, and if I mix them, the error label stops meaning anything. Two details I care about: **It uses a read-only database role.** n8n gets its own Postgres user that can only SELECT from the two tables it needs. An automation tool doesn't need write access to production just to ask "have we seen this number?" **It fails open.** If the database is unreachable or the subject has no number, the invoice goes through the normal path. The API's own duplicate check is still there as a backstop. A broken shortcut should never become a broken pipeline. ## A watchdog and a morning report For visibility, I added two small workflows. The **watchdog** runs every 30 minutes. It flags anything waiting more than an hour, anything stuck in PROCESSING across two checks in a row, and any unread errors. It sends one Telegram alert per newly stuck invoice, not a repeat every half hour, and a single "queue is clear" message once things drain. I've been on enough monitoring desks to know that an alarm that keeps ringing just gets muted. The **daily digest** goes out at 8 AM. It lists one row per invoice, grouped as processed, duplicate, error, no PDF, or still waiting. It goes to me first until I've checked the format, then to the person who actually reconciles these invoices. ## The part where I got the clock wrong I wanted to run the sweep right away instead of waiting a day, so I checked when it was scheduled. "Hour 14," the config said. I told myself, and then told the people I was working with, that it runs at 2 PM Manila. It doesn't. The n8n container is set to Asia/Manila, but the schedule triggers use n8n's own default timezone, which is America/New\_York. Hour 14 means 2 PM in New York, which is **3 AM in Manila**. I corrected it, fixed the wording in the watchdog alert, and wrote it down so I don't make the same assumption next time. Anyone who has run a broadcast across time zones knows this one: always ask "whose clock?" To force a sweep, I ended up temporarily setting the trigger to a New York time a few minutes ahead. One attempt didn't fire and I never found out why. The next one did. ## Then Telegram said 502 The sweep started, worked through a few invoices, and then died. The cause was `502 Bad Gateway` from Telegram on a "this invoice needs review" notification. The notification node had no retry and no error handling, so one failed chat message stopped the whole run, and every invoice behind it stayed in the queue. The homelab logs showed other outbound network errors around the same time, so it was most likely a short internet hiccup rather than anything in the workflow. That doesn't really excuse it. A *notification* should never be able to stop the *work* it's reporting on. All three Telegram nodes now retry three times, five seconds apart, and if Telegram is still down, the run moves on. You might miss a ping, but you won't miss an invoice. I restarted the sweep. The first email it touched was one of the known duplicates. It got tagged LT-DUPLICATE and left the queue in seconds, with no Ollama run. Seeing that was what made the whole day feel worth it. ## What I'm taking away Do the cheap check before the expensive one. If the data is already in the subject line, don't pay a model to read a PDF for it. Silent backlogs are worse than loud crashes. A crash at least tells you something happened. A queue that quietly stops looks exactly like a quiet day. Side effects shouldn't be able to kill the main job. Notifications, logs, and alerts should all fail soft. And always ask whose clock a schedule is using. The backlog is still draining as I write this, at a few minutes per invoice. This time I'll get a message if it stalls.