Building CreatorModo: Backend Architecture (Part 1 - High Level Design)

Hello again. Hope the intro post didn't put you to sleep.

Last time I gave you the trailer: what I'm building, why I'm building it, and a high-level skim of the stack. This time we actually open the hood, specifically on the backend, because that's where most of the real engineering happened. And, not coincidentally, where most of my regrets currently live.

Here's the diagram from before, since we're about to walk through every arrow on it.

Why Two Backend Services Instead of One?

The obvious move would've been one FastAPI app doing everything: receiving webhooks, validating them, saving automations, processing events, firing DMs. One repo, one deploy, one service to monitor at 3am.

I didn't do that. On purpose.

Instagram doesn't send you webhooks politely. It sends them whenever it wants, as many as it wants, and it expects a fast response or it starts retrying — which means duplicate events, which means duplicate DMs, which means creators yelling at me. So the one thing I knew for certain going in: whatever receives the webhook cannot also be the thing that does the slow, expensive work of figuring out which automation matches and firing off a reply.

But honestly, the bigger reason was simpler than all that. I wanted to build this thing once and forget it exists. Keep it small, keep it dumb, ship it, never touch it again. Keeping it separate meant Instagram's webhook quirks, retry behavior, whatever weird edge case shows up this week, none of it could ever hinder or change how I build the actual product on the Main API. Build it once, forget about it, never worry about it again.

That single constraint is basically why this whole system looks the way it does.

Splitting Receiving From Processing

So I split it into two services:

• Automation Service (Main API) — the actual product. Creators log in here, create automations, manage their account. This is the FastAPI app with Async SQLAlchemy, Alembic migrations, JWT auth over httpOnly cookies, Fernet encryption for Instagram tokens, and Resend wired in for emails. Boring, stable, does its job.
• Webhook Ingester — a much dumber, much faster service. Its entire job is: receive the Instagram event, shove it onto a queue, say thanks, goodbye. No business logic. No DB writes. Nothing that could slow it down or make it the reason Instagram starts retrying.

That second service existing at all is the whole point. It's the thing standing between "random person comments on a post" and "my database falls over."

Enter RabbitMQ

Between the two services sits RabbitMQ, using aio-pika on the publishing and consuming side since everything here is async.

The idea is unglamorous, but it works:

• Webhooks come in fast and unpredictably.
• The broker holds onto them.
• A separate consumer processes them at whatever pace it can sustain (with a prefetch of 10, a burst can't bury the worker).
• The ingester never blocks on slow downstream work.

It's not particularly sophisticated, but it gives you a bit of backpressure and keeps the ingestion path responsive even when processing falls behind.

One caveat I'll own up front: a message that fails three times gets nacked without requeue and dropped — there's no dead-letter queue yet. So "nothing ever gets lost" is a claim I haven't actually earned.

There's also a Queue Management UI sitting next to the Broker, but it's not part of the data path — it's just there so I can see what's piling up in the queue without SSH-ing into a box and guessing.

P.S. The ingester publishes seven event types but the consumer only listens for comments today. The rest wait under their routing keys. New feature = new consumer; I never touch or redeploy the ingester.

Walking The Flow End-to-End

Putting the diagram into actual sentences:

1. Random person comments on a creator's post.
2. Instagram fires a webhook at the Webhook Ingester with event metadata — who commented, on what, what they said.
3. The ingester does the bare minimum validation and immediately publishes the event to the Broker. It publishes and acknowledges — fast and non-blocking, no DB lookup, no Graph API call.
4. CreatorModo Consumer, a completely separate worker process, pulls events off the Broker whenever it's free.
5. The consumer asks Postgres for the automation details tied to that post/keyword — what's the matching automation, what should the reply contain, is it even still active.
6. If there's a match, the consumer fires a comment and/or DM back through the Instagram Graph API.

Notice the Main API isn't anywhere in that runtime path. Its only job is letting the creator create and save the automation in the first place, which gets written to Postgres ahead of time. By the time a real comment shows up, the Main API has already done its part and gone home.

Why This Decoupling Actually Mattered

I'll be honest, I didn't fully appreciate why this mattered until I started actually load-testing webhook bursts. A popular post can generate a wall of comments in seconds. If the ingester had to do a DB lookup and an Instagram API call per comment before responding to the webhook, it would buckle immediately, and Instagram would start retrying the same events on top of the backlog. Decoupling isn't a buzzword here; it's the only reason this doesn't fall over the first time a creator's post actually does numbers.

What I'd Probably Do Differently

A few things I already regret slightly, in the spirit of not sanitizing this:

• Queue monitoring was an afterthought instead of something I set up day one. I found out about backlog issues by creators complaining, not by dashboards telling me first.
• I underestimated how much retry logic Instagram's webhook delivery actually does, which meant my "fast enough" ingester wasn't initially fast enough.
• No dead-letter queue yet — a message that fails three times gets dropped, not parked for inspection. 'Nothing gets lost' is a claim I haven't actually earned.

More on these in a later part, once I'm emotionally ready to write about them in detail.

What's Next

Next up — I'm torn between going low-level on the queue/consumer internals, walking the Instagram API integration, or covering the frontend. Probably whichever one I'm least tired of by the time I sit down to write it.

The product's still in beta because Meta approval takes forever, as always. If you want in anyway, drop a request on the CreatorModo contact page.

More soon.

Oh yeah, more coming soon, but I forgot to mention deployment. The whole thing is deployed on Railway. Honestly, luv ya Railway <3 — for like $5–10/month, the entire project is up and running smoothly, with an easy place to view logs and monitor everything.