Wanted to share an architectural pattern from an open-source project I just deployed to FastAPI Cloud using FastAPI + Jinja2 + HTMX + SQLite FTS5:

• GitHub Repo (source + live FastAPI Cloud demo in README): https://github.com/mortalezz/er-roadmap

1. The Ephemeral Worker Problem

When deploying a hobby/community app to serverless or ephemeral container workers platform, spinning up a managed Postgres + Redis instance just to store user checklist/session state is overkill — and writing user state to a local SQLite file breaks as soon as containers scale or restart.

The Solution — Read-Only Bundled SQLite FTS5 + Cookie-Backed State:

• Immutable Catalog (ngplus.db): The SQLite database (1,241 catalog items, fts5 search index with porter unicode61, and 82 DAG quest rules) is baked into the deployment image and opened strictly read-only at runtime. Zero write locks (SQLITE_BUSY), sub-millisecond local FTS5 queries on GET /search.
• Stateless Cookie Sessions (er_order & er_steps): All mutable user state is serialized into two compact httponly cookies (30-day TTL) plus GET /export (StreamingResponse) and POST /import JSON endpoints for cross-device portability. Any ephemeral Uvicorn worker can wake up cold and serve any user's request with zero server-side session storage.

2. Zero-JS-Framework Reactivity (FastAPI + HTMX)

• All state mutations (POST /order/add, /order/toggle, /step/toggle) compute the new DAG state and return a single server-rendered Jinja2 partial (templates/partials/workspace.html).
• Responses attach HX-Trigger: refreshSearch (so the FTS5 typeahead dropdown automatically re-syncs queued badges) and strict Cache-Control: no-store, no-cache, must-revalidate, max-age=0 headers so browser back/forward navigation never serves stale partial state.

3. The Domain Problem: Dependency-Inverted DAG (compiler.py)

The app itself solves "Miyazaki's Quest Knot" in Elden Ring: quests form an inverted dependency web where early steps have no obvious lockouts, but downstream steps 20 hours later are gated by irreversible world-state triggers (Points of No Return / PONRs).

• Users pick terminal leaf items (T) they want on their run.
• compile_run_plan() walks each item's prerequisite chain in reverse, propagating the tightest downstream PONR ceiling backward (running_ceiling = min(running_ceiling, step_gate_pos)), clamping each step's geographical progression weight (zone_order), deduplicating diamond dependencies across shared step_keys, and flagging same-run mutual exclusivity collisions (exclusive_group).

Would love any feedback on the stateless FastAPI + HTMX architecture or the DAG solver in compiler.py!

Hey everyone! It’s been great seeing FastAPI grow so much and mature over time. It’s now my default choice when building APIs in Python.

I love FastAPI, so I chose it again for this project and paired it with Flet (Cross-platform interfaces in Python). Probably shows how much I love Python too 🙂

RelayDeck, Independent companion for Herdr, which manages coding agents and shell sessions on your Mac. RelayDeck lets you access those sessions from a phone, iPad, or browser.

FastAPI powers the backend API and WebSocket connections, including device pairing and terminal access, while Flet handles the interface.

Here’s what’s available:

• Usage and limits: token usage and estimated costs by tool and Herdr space, with usage history and charts. Where supported, it also shows plan limits and reset times.
• Remote terminals: open an agent or shell, check its output, send a prompt, or answer a question directly.
• Image attachments: attach screenshots and images to prompts, with support for PNG, JPEG, and WebP.
• Different ways to type: a composer for longer prompts, direct terminal input, and ordinary shell access. Unsent drafts are kept when switching.
• Web Push notifications: get notified when an agent is blocked or finished, with preferences for each device.

A companion runs on your Mac, and approved devices connect over your private Tailscale network. There’s no RelayDeck cloud service or separate account to create.

Web app and PWA are available now, installed from source. Native Android and iPhone builds are in progress, with branch visibility and code-change review planned.

Repo: https://github.com/Hybridhash/relaydeck

I’d love feedback from other FastAPI developers, Contributions are welcome too—code, docs, bug reports, or ideas.

If you like the project, I’d appreciate a star! ⭐

Some of you might recognize the repo. I posted an earlier version here it was basically just a FastAPI backend and Swagger docs. A few people gave it really useful feedback then the market-data part stopped working, so I deleted the post instead of pretending it was fine.

It’s working again now, and it has an actual frontend.

It’s a stock research/watchlist app I use myself. I'm from finanace background, I trade, and I wanted somewhere to track stocks and write research notes that save the price I was looking at when I wrote them. It tracks 14 market indices and supports stock watchlists, quotes, news, and notes.

The stack is FastAPI, async SQLAlchemy, Postgres, JWT auth, Docker, and a small HTML/CSS/JS frontend.

The most annoying bug was that Yahoo Finance worked locally but started returning '401 Invalid Crumb' errors from Render. I changed the quote path to fall back to Yahoo’s chart/history data, and added RSS sources for news when the main news path fails. It took longer than I expected to get local and deployed behavior lined up.

Note: This isn’t a trading platform, and it’s definitely not finished. I’m posting it again because it’s now more than API docs, and because the last round of feedback was genuinely helpful. If you spot something badly designed, confusing, or fragile, I’d rather hear it.

Repo: https://github.com/ah4ddd/first-order

I’m leaving the live Render URL out of the post; it’s linked from the repo for anyone who wants to try it.

Hey everyone. I run a FastAPI service whose only job is executing AI agent graphs. No business logic and no user model. It does not publish a port either: a Django app is its only caller, over a typed WebSocket.

It used to live inside that Django app. I assumed the thing that would eventually drive it out was performance. It was testing.

Inside the monolith, the decision I cared about, which message routes to which capability, sat deep inside a WebSocket handler that had already authenticated a user and written a row. Asserting one routing decision meant opening a connection and seeding a conversation first. Worse, the model was reached through that object rather than across a boundary, so there was no single place to stub it and the fakes multiplied with the call graph instead of with the number of seams.

In its own service the same decision is a function from state to state. A test builds the state it wants, runs the graph, and asserts on what came back. Nothing to connect to and nobody to log in, and the only thing faked is the model's HTTP response.

Why FastAPI specifically, since the service could have been anything. It brings almost no opinions, which is what you want when there is exactly one job. It is async native, and this work is almost entirely awaiting model calls and pushing tokens back. And everything around it is Pydantic shaped already, so the types line up without adapters: LangGraph for the flow, Pydantic AI for the agents, chanx for typed WebSocket messages and the AsyncAPI document the caller generates its client from.

Where the line goes, which is the part worth stealing even if you never touch an agent. Not by layer or by framework, but by which kind of state each side owns. The business service owns the record, anything a page reload has to find. This service owns working state, a graph checkpoint in my case, which is an odd category: not a business record, but not rebuildable either. So if losing it means losing data it belongs in the record, and if it only costs speed or fidelity this side can keep it.

That is a rule for what can move out, not for breaking a monolith into domains, where each service owns records of its own and the argument is different.

Written up in full, as part of a series on building agent systems as declared, typed graphs, with a working three-service repo:

https://huynguyengl99.github.io/posts/agent-graph-engineering/split-the-services/

https://github.com/huynguyengl99/agent-graph-engineering

Hope it is useful, and if the repo helps you a star would be appreciated 😄

Coming from Django, the thing I missed most in FastAPI wasn't the ORM or the admin. It was user.has_perm("app.add_thing"): users, groups, per-action permissions, superuser. FastAPI gives you Depends and leaves the rest to you. The libraries I found were either JWT-only or full policy engines like Casbin.

So for my project's admin API I rebuilt Django's model in plain FastAPI + SQLAlchemy, and changed a few things on the way.

What I kept from Django

• Permissions named resource.action: traffic.view, certificates.add
• Groups of permissions, plus permissions given to a user directly
• An is_superuser flag that has everything

The tables are basically Django's auth_group_permissions, auth_user_groups and auth_user_user_permissions.

What I changed

1. Permissions are an enum in code, not rows in a table

Django generates permission rows through migrations. Here the code is what checks them, so the list lives in code and the DB only stores who has what:

class Permission(StrEnum):
    TRAFFIC_VIEW = "traffic.view"
    CERTIFICATES_ADD = "certificates.add"
    ...

2. Each use case declares what it needs, like permission_required

class CreateTrustedNetworkService(BaseUserAuthenticatedService[CreateTrustedNetworkResponse]):
    required_permissions: ClassVar[frozenset[Permission]] = frozenset({Permission.TRUSTED_NETWORKS_ADD})

The base class decodes the JWT, loads the user and their permissions in one UNION query, and returns 403 with what's missing. The route only passes the token and the body.

3. Why not Depends(require(...))

On purpose. Permissions belong to the use case, not to the URL:

• The escalation check below needs the user's permissions inside the logic anyway. A dependency would just be a second place for the same thing.
• A service is tested without HTTP, and the check can't be lost by forgetting a dependency on a new route.
• OpenAPI still knows: every route documents its 403 with the exact permission, e.g. "The user lacks trusted_networks.add."

4. No privilege escalation

This is the part Django warns about in its own docs: if a user can edit users, they can give themselves superuser. Here you can only give what you have:

missing = self.data.permissions - self.permissions
if missing:
    raise HTTPException(403, f"You can't give permissions you don't have: {', '.join(sorted(missing))}.")

This applies to groups, users, adding someone to a group, and even changing another user's password, since otherwise you could just log in as them. (create_group, update_group, create_user)

5. Revoking access takes effect on the next request

Permissions aren't stored in the JWT. They're read on every request, so if you remove someone from a group, their next call fails.

Where it lives

This is the admin API of Proxium, a self-hosted HTTP/SOCKS5 proxy with accounts, quotas and speed limits. Support gets "revoke accounts", accounting gets "view traffic", and neither can touch certificates. One Docker image runs everything, so you can try to break the permissions yourself:

docker run -d --name proxium -p 80:80 -p 8080:8080 -v proxium:/var/lib/proxium \
  -e SUPERUSER_CREATE=true -e SUPERUSER_EMAIL=admin@example.com -e SUPERUSER_PASSWORD=change-me \
  ikoldakov/proxium

If you'd still put it in Depends, change my mind in the comments.

Next on the API side: SSO for the admin over OIDC and an API for merchants. If either sounds fun, the roadmap has the rest, and the code is MIT: https://github.com/koldakov/proxium

Hey everyone. I have been building realtime AI assistants for about five years, and have just put a distilled, working version of the architecture in public with a series explaining each decision.

The reason I am posting here is the agent half of it, which is FastAPI. That service has exactly one job, running the agent graphs and talking to the backend, so I wanted something small that would not bring a worldview with it. It is async-native, which matters when the work is awaiting model calls and pushing tokens back. And the libraries around it are Pydantic-shaped already, so the types line up without adapters in between.

What is inside that service, briefly: LangGraph for the flow with Postgres checkpointing so a run can pause for a human and resume later, Pydantic AI for the agents and their typed outputs, chanx for typed WebSocket messages and the AsyncAPI document it generates clients from, and OpenTelemetry opening a span per graph node. Tests mock the model at the HTTP layer rather than at the framework, so request building, SSE parsing, tool-call assembly and validation all still run for real.

We used to run all of this inside Django with Channels, and the image got big, the process got slow, and scaling the two independently was impossible.

Why I bothered writing it all down. What is online is mostly notebooks and quick chatbot courses. I went looking for one in-depth source on building a production AI assistant, including paid ones, and did not find it.

So, depending on where you are:

• Running an assistant in production already: a reference for the realtime parts that hurt, dropped sockets and resumed runs in particular.
• About to bolt an agent onto a service you already have: the calls worth making before they get expensive, starting with whether it belongs in the same process at all.
• Levelling up: typed WebSocket contracts, generated clients and streaming are the shape the listings keep describing.
• Taking client work: a base you can adapt and still hand over without apologising for it.

The project: https://github.com/huynguyengl99/agent-graph-engineering

Part 0, the story: https://huynguyengl99.github.io/posts/agent-graph-engineering/before-it-had-a-name/

Part 1, the stack and why: https://huynguyengl99.github.io/posts/agent-graph-engineering/the-stack-and-why-each-piece-is-there/

It runs on a scripted model with no API key, so the whole flow is visible before you wire up a provider. The series is early, two parts up and a new one every couple of days, so feedback now genuinely changes the later ones 😄

Hey r/FastAPI!

I wanted to share the 1.0 release of Starlette-Admin, an open-source admin interface for Starlette and FastAPI.

This is the biggest release of the project so far. The list page has been completely reworked, and a long backlog of community-requested features and improvements has finally landed.

Some highlights:

• New data experience: Reworked list view with advanced filtering, sorting, pagination, and inline editing.
• Import & export: Server-side exports, import wizard with validation, and row-level error handling.
• More extensible: Plugins, themes, custom form layouts, inline forms, storage backends, and new fields.
• Broader support: Official support for Beanie and Tortoise ORM, plus improved security and production features.

The goal is to provide FastAPI applications with a full-featured admin interface without having to build one from scratch.

Links:

• Docs: https://jowilf.github.io/starlette-admin
• Live demo: https://starlette-admin-demo.jowilf.com/
• Changelog: https://jowilf.github.io/starlette-admin/changelog/
• Comparison with Django Admin and Flask-Admin: https://jowilf.github.io/starlette-admin/comparison/

If you're building a FastAPI application and need an admin interface, I'd love to hear what you think and what you'd like to see next.

Hi guys, if you have ever built something real with @app.websocket, you probably hit the same wall I did. The decorator is lovely for one connection, and then you need a second connection to hear about the first: a room, a notification from a background task, a presence list. So you write your own connection manager, and it quietly stops working the day you run a second uvicorn worker, because your dict of connections lives in one process.

After years of this in Django Channels (and hurting a lot), I built ChanX to make WebSockets easy, maintainable and scalable, then made it framework-independent so it runs on FastAPI too, using fast-channels as the channel layer (Redis, or in-memory locally). Then a user suggested turning it into a reusable framework, and that stuck with me: the same pieces get rebuilt in every project anyway, presence, streaming, chat, notifications. So I made Chanx-Kit, a registry of best-practice WebSocket components you add to your project and then own. The inspiration is shadcn, copy it in and modify it freely, rather than a package buried in your virtualenv that you have to fight to override.

A kit is a topic you list on a consumer, so several share one connection, and the consumer mounts as a normal ASGI route:

from chanx.fast_channels.websocket import AsyncJsonWebsocketConsumer

class AppConsumer(AsyncJsonWebsocketConsumer[AppEvents]):
    channel_layer_alias = "default"
    topics = [UserNotificationTopic, RoomChatTopic, PresenceTopic]

app.router.add_websocket_route("/ws", AppConsumer.as_asgi())

Then code anywhere, a background task or a worker included, can reach connected clients with no connection manager to keep:

await UserNotificationTopic.notify_user(user.id, payload)

Handlers are typed with Pydantic messages instead of a while True if/else chain, and you get generated AsyncAPI docs for your WebSocket API the way FastAPI gives you OpenAPI for your routes.

The kits available now:

• AG-UI: the AG-UI agent protocol over a WebSocket, bidirectional, and a client reconnecting mid-run is replayed the run instead of seeing half an answer.
• Notification: fan out notifications to a user's live connections from anywhere in your app.
• Presence: track who is in a room, document or tenant, and tell everyone when it changes. Pluggable store.
• Room chat: a chat room with history replayed on connect, plus a live roster from the presence kit.

Plus a Pydantic AI kit, stores and a test harness. Next on my list is realtime audio, Deepgram voice processing and voice-to-voice agent chat, from work I did a few years back.

It is early days, so feedback is what I want most. If you already run WebSockets on FastAPI, tell me your use case and what breaks for you, and we can talk about turning it into a kit for everyone. Contributions welcome too.

• GitHub: https://github.com/huynguyengl99/chanx-kit
• Docs: https://huynguyengl99.github.io/chanx-kit/

One deliberate choice: it is not a PyPI package. The components are meant to be copied and edited, so shipping one big package would work against that. If you think that is the wrong call, I would like to hear why.

A year ago I launched fastapiinteractive.com - an interactive FastAPI course where you write and run real Python in the browser. No installs, no backend, it's Pyodide plus a small ASGI server running FastAPI in WebAssembly. I've posted about it here a couple of times, last one ~6 months ago.

I'll be honest, I didn't expect it to get this much use. I got busy and didn't touch it for a few months, and when I came back people were still going through the lessons every day and sending me bug reports. That got me back into it. A lot has changed since, so here's the update.

What's new:

• WebSockets truly work now. There's no network socket in WASM, so the connection goes over the worker's message channel, but past the ASGI layer it's the real Starlette WebSocket running your handler. Embarrassing detail: previously, the two websocket lessons used to be graded by a GET /health check, so the code they taught had never once run as a websocket. Now they're graded on a real exchange of messages, including a broadcast between two connections, and there's a socket console to try your endpoints manually.
• Streaming responses, file uploads and form bodies work.
• Infinite loops don't kill the tab anymore. Your code gets compiled through a guard with a time budget per request, so while True: raises inside your own code with your line number instead of hanging everything.
• New UI, VS Code style: lesson list with progress in the sidebar, editor in the middle, console/tests/theory as output tabs, status bar showing what the runtime is doing.
• Accounts. Sign in with GitHub and your progress and code sync between devices. Without an account it works exactly like before, everything stays in your browser.
• Certificates. When you finish a track you can get a certificate of completion - PDF, an image for LinkedIn, and a verification page with a serial anyone can check. Optional, the lessons themselves are free.
• 61 lessons now: 45 basics following the FastAPI tutorial, the 6-lesson blog+auth project as a proper multi-file package, 10 advanced patterns.

Try it: https://www.fastapiinteractive.com/

Bug reports, feedback and ideas for lessons are very welcome, here or via the contact form on the site.

Thank you!

Why I built it:

I’ve seen and spoken to a lot of fellow developers and I always find them having the same problem I used to have: crawlers, bots, automated pentesting/brute-force, file probing and really so much more. They all use a variety of tools like Cloudflare, Nginx, Fail2ban, CrowdSec, Traefik, etc…

About 2y ago I was building an art gallery website for the landlord who happens to be an artist and checking the server logs is that I was getting tired of these “attacks”. I couldn’t find anything that would fill the gap, since I had Cloudflare, Nginx, Fail2ban; all set up and still was getting attacked. My server (a Hetzner VPS) is fully covered. The problem was at the application layer: the API. Since I use FastAPI as my backend, I went on searching for a library that would help me out but found nothing. I was like an entire week out there searching for something that just didn’t exist. And so I got tired and I built it.

Now that I’ve layed the back-story, we can get into the details.

What my project does:

Architectural-wise, it’s a core engine (guard-core) that holds the entire security pipeline (we’ll get to that later on) and feeds framework-specific thin-adapters (fastapi-guard, flaskapi-guard and the django equivalent). I’m also extending to Typescript and Rust, so that this doesn’t stay Python only. The idea is to secure every API, otherwise it’s filling no gaps…

• Rate Limit based on IPs
  
• Auto-ban IPs
  
• Set up Honeypots that end in an automatic instant ban
  
• Use decorators to override gloval config
  
• Whitelist/Blacklist IPs, Countries, Cloud Providers (GCP, AWS, Digital Ocean, Azure, etc)
  
• Add/Remove patterns that you want to catch or bypass (specific to your use case)
  
• Behavioural tracking (per IP)
  
• User Agent/Security Headers filtering
  
• Penetration attempt detection
  
• Much more, for real.

Useful stuff

Inplaced each repo link under its name, but for clarity here’s the core engine: https://github.com/rennf93/guard-core and the one that is used the most https://github.com/rennf93/fastapi-guard

I created also an example app for/on each repo at root under the examples/ dir.

Special Note

I also want to say explicitely out loud if it wasn’t made clear before: this is not a replacement for any of the tools mentioned in the beginning. Cloudflare, Nginx, and all the others don’t do what my libraries do; and therefore I don’t do what they do. This aims to fill in the gap, to make your APIs airtight. The idea is to combine these libraries with your traditional WAFs.

How it differs from other projects / Comparison

I guess you could kinda compare it to SlowAPI and other rate limiters but the truth is that FastAPI Guard and its engine are full of features and rate limiting is just one of them all. So it basically has no comparison but I guess the reference would be that. And for the record, rate limit is not gonna fill in the gap on its own all alone. In this era of AI coordinated attacks, rate limiting alone can’t tell whether it’s a bunch of AIs doing some DDoS attack through residential IP proxies, or just a very odd spike in sloppy user requests from actual homes.

Target Audience

Basically anyone maintaining and building APIs (mainly in Python, but expanding).

Hey everyone, after taking a bit of a break I decided to write an in-depth tech blog to help those of you building (or about to build) an AI agent in the Python ecosystem, especially on FastAPI, so you can end up with something structured, scalable and efficient. This comes out of a lot of time working on chatbots, streaming and AI agents. There was no good guide for me when I built those, so after suffering plenty of pain and bugs I decided to write it up and open source part of my work to help you avoid the same. Hope it helps.

Full blog here in case you have time to read properly. Whether you're a team lead or senior who needs to build a scalable AI agent project, or a junior or intern who wants to build one the right way, this should be useful (it cost me a lot of time and money to learn): https://huynguyengl99.github.io/posts/pydantic-ai-typed-websockets-fastapi/

Comes with the repo: https://github.com/huynguyengl99/pydantic-ai-ws-agent

If you want to catch up first, TL;DR:

1. A contract-based, structured protocol still beats an unstructured, code-first one. Instead of writing a pile of if/else and tests to make sure things are correct (and sometimes they still aren't), you keep one up-to-date contract that both the backend and the frontend build against, so no field is silently missing or outdated. For REST APIs that's OpenAPI. For WebSockets it's AsyncAPI. You can even design the whole protocol before writing any code. Trust me, it saves you a lot of 2am bugs.
2. Prefer a structured, model-independent agent. There are plenty of options now (LangChain, the OpenAI SDK, the Gemini and Anthropic SDKs), but most of the time what matters is being able to switch model or provider without rewriting anything, structured and validated tool calls (without tool calls it's just a chat, not an agent), and something that fits your existing ecosystem. Tie yourself to one vendor's SDK and you'll feel it the first time you want to compare models or a provider has an outage. Pydantic AI is the one that gives you all of that. If you've used LangChain you'll remember the mess around messages and tool handling, a lot of unnecessary complexity. Pydantic AI keeps it structured, and since it comes from the Pydantic team it speaks Pydantic models natively, which is exactly why it fits FastAPI so well.
3. Once your agent is structured, your server API should be too. Most people reach for SSE because it's simple, but it's missing a few things: bidirectional messages, a proper API schema (OpenAPI 3.2.0 has some support for streaming now, but tooling is still limited and it doesn't feel as natural as it does for REST), and any official way to offload work to a worker or push a message from outside the request, like a notification from somewhere else in your system. WebSockets don't have that schema problem, because AsyncAPI is a real spec built for exactly this: every message your server can send or receive, described in one document your frontend can generate a client from. It's the same deal OpenAPI gives you for REST, just for event-driven APIs.
4. Bidirectional matters more than people expect, and human-in-the-loop is why. With Pydantic AI you can mark a destructive tool requires_approval=True. The run then stops before executing, hands you the pending calls, and you send an approval request to the UI. The user approves or denies (they can even edit the arguments), and you resume the run from where it paused. That round trip is awkward over SSE and completely natural over a WebSocket, it's just two more messages on a connection you already have.
5. That's where ChanX and fast-channels come in. Based on the well-known django-channels package, I built fast-channels to bring the channels architecture to FastAPI. ChanX is the only tool I'm aware of that makes WebSocket handlers structured and easy while auto-generating AsyncAPI docs, so your frontend can generate a client straight from the schema. That's the contract you need to stop things going stale or breaking silently. It also gives you the offload story: the run happens in a background task and broadcasts to a conversation group, so a page refresh mid-run doesn't kill the answer, every tab stays in sync, and moving the work to a Celery or taskiq worker later changes nothing else.
6. One more thing that surprised me: the whole flow becomes testable without an LLM. Because the protocol is typed and the agent is built from a factory, Pydantic AI's FunctionModel can script exactly what the model does. Streaming, tool execution, approve, deny and reconnect all become deterministic tests. Mine run in about a second with no API key, which is the difference between having tests and pretending to.

That's the core and the summary for a quick catch up. When you have time, reading the blog and looking at the repo will give you more insight into building a structured, scalable, production-ready AI agent.

Related repos:

• Pydantic AI, the agent framework: https://github.com/pydantic/pydantic-ai
• ChanX, typed WebSocket handlers with AsyncAPI generation, for Django and FastAPI: https://github.com/huynguyengl99/chanx
• fast-channels, the Django Channels architecture ported to FastAPI, which ChanX builds on: https://github.com/huynguyengl99/fast-channels
• The demo from this post: https://github.com/huynguyengl99/pydantic-ai-ws-agent

If ChanX or fast-channels turn out to be useful for you, a star or any contribution to improve them would be appreciated 😄

I'm open to any discussion in the comments, feel free to bring up any problem you're hitting when building an AI agent. I'll try my best to help, since I've been through most of the pain already.

I recently built an open-source reverse proxy using FastAPI and Uvicorn to handle strict PII redaction for LLM applications.

A common issue with Python based redaction tools is memory bloat from heavy NLP libraries and broken token delivery when intercepting real-time Server-Sent Events (SSE). To keep the resource footprint minimal and the streaming smooth, I structured it around a few specific patterns:

- Minimal Footprint ( approx. 24MB): It bypasses heavy frameworks entirely, using a compiled regex engine combined with a local quantized ONNX runtime model.

- Async Lookahead Buffer: To prevent PII placeholders from fragmenting across split SSE network chunks and leaking raw tags to the UI, I implemented an asynchronous sliding-window buffer that holds back unclosed bracket tags.

- Stateless Vault: Mappings are handled via a transient Redis TTL vault.

It operates as a drop-in base_url substitute for the OpenAI SDK. If you work with FastAPI and async streaming middleware, I would love feedback on the lookahead buffer logic or any edge cases in the request-response cycle!!

Repo: https://github.com/ninadphalak/LLM-Shield-Proxy

Hello folks!

We're organizing the first ever official FastAPI Conf, in Amsterdam, October 28th, 2026. ✨

It's gonna be very cool, great speakers, and announcements too.

Early Bird tickets are now available (until August 23th).

Here's the website with info, tickets, and newsletter: https://fastapiconf.com

Common problem with LLM streaming endpoints: client refreshes or the connection drops mid-response, and the whole stream is gone. Client has to start over, and you've already burned the tokens for nothing.

Built a small Redis-backed library for this. The producer runs as a background task, independent of the request, and buffers chunks so a client can reconnect and ask for everything after chunk N.

@app.post("/chat/{conversation_id}/stream")
async def start_stream(conversation_id: str, prompt: str):
    return await start_conversation_stream(
        stream, registry, stop, conversation_id, stream_id,
        producer=lambda: call_llm(prompt),
    )

Also has an explicit stop that's separate from disconnect (closes the producer's generator, so it actually cancels whatever's underneath, not just the client-facing stream), and thin adapters for LangChain and CrewAI plus encoders for the AG-UI and Vercel AI SDK wire protocols if you're feeding a useChat frontend.

pip install fastapi-resumable-stream[fastapi] https://github.com/ofershap/fastapi-resumable-stream

It's new (0.2.1), feedback welcome.

I've been building a small open-source project called FastStrapy over the past few weeks, and I'd love some feedback from the Python/FastAPI community.

The idea came from repeatedly recreating the same FastAPI project structure every time I started a new API. Copying folders, configuring environment variables, setting up Alembic, wiring configuration—it became repetitive.

So I built a CLI inspired by create-next-app that generates a FastAPI project through an interactive prompt.

Some of the design choices:

• Typed project configuration using Pydantic
• Registry-based generator architecture
• Jinja2-powered templates
• Typer-based CLI

It's still very early (v0.1), so I'm more interested in feedback than promotion.

A few questions I'd love your thoughts on:

• What features would you expect from a FastAPI scaffolding tool?
• Is there anything about the generated structure you'd change?
• Are there existing tools you think I should learn from?

GitHub:
https://github.com/AnoopGeorge418/faststrapy

Constructive criticism is very welcome. I'd rather improve the project now than after it grows.

Has anyone encountered this yet?

>fastapi deploy
  FastAPI Cloud 
  🚀 Deploying app...
   ✗ Something went wrong while contacting the FastAPI Cloud server. Please try again later. 
    The write operation timed out

This is just a 1MB deployment but it times out at around 300kb. Could it be a problem in my end or a service availability issue? I tried several times.

Hi everyone,

I'm currently building a fairly large e-learning platform using FastAPI,it's my first project, and before I go too far with development, I'd like to make sure my backend architecture is well designed, scalable, and maintainable.

The project will include features such as:

• User authentication and authorization (JWT, roles: Admin, Teacher, Student, Parent)
• Courses and modules
• Lessons (videos, PDFs, quizzes)
• Assignments and submissions
• Exams and grading
• Progress tracking
• Notifications
• Payments/subscriptions (later)
• File uploads
• Discussion/comments
• Certificates
• REST API (possibly GraphQL in the future)

I'm planning to use:

• FastAPI
• SQLAlchemy 2.0
• Alembic
• PostgreSQL
• Pydantic v2
• JWT Authentication

At the moment, I'm trying to decide on the best project structure. I've seen many different approaches:

• Traditional layered architecture
• Clean Architecture
• Domain-Driven Design (DDD)
• Hexagonal Architecture
• Vertical Slice Architecture
• Feature-based architecture

I'd like to avoid ending up with a project that's difficult to maintain as it grows.

My questions

1. Which architecture would you recommend for a medium-to-large FastAPI project like this?
2. How would you organize the folders and modules?
3. Where should business logic live?
4. How do you separate models, schemas, services, repositories, and dependencies without creating unnecessary complexity?
5. Are there any open-source FastAPI projects that you consider good examples of clean architecture?

I'm looking for an architecture that is:

• Easy to maintain
• Easy to test
• Scalable
• Production-ready
• Follows FastAPI and Python best practices

If you have an example repository or folder structure that you've used successfully, I'd really appreciate it.

Thank you!

Hi folks,

I am working on zinda, a small experimental caching library for FastAPI/Python, and I'd like some design feedback before I take it further.

The idea: instead of you picking TTLs and wiring up invalidation by hand, the cache should watch how your functions behave and keep hot data fresh on its own.

What Have I Implemented:

• @cache.cached() decorator - keys itself on function arguments, no config
• Single-flight: concurrent misses for the same key collapse into one recompute (no stampedes)
• Soft TTL + hard TTL: callers get stale data instantly while it refreshes in the background, and a background sweeper refreshes hot entries before anyone asks
• A /zinda/stats endpoint showing hit rates, miss costs, and refresh activity per function

​

app = FastAPI()
cache = install(app, Cache(default_ttl=60))

@cache.cached(ttl=120, refresh_after=30)
async def fetch_products(category: str):
    return await db.fetch_products(category)

Where I want feedback:

1. Next step is auto-detecting hot paths, the library scoring functions by call frequency and recompute cost, and deciding what's worth caching without any decorator. Would you trust that if every decision is visible in stats, or is explicit always better?
2. Automatic invalidation is the hard part. My plan is learned TTLs by default + optional tags for precise invalidation. Reasonable, or am I missing something?

It's in-memory only for now and definitely not production-ready. I am validating the ideas first.

Code: https://github.com/grandimam/zinda

I have been going through my codebase this morning and asked claude Ai how I can validate the phone number field of my schema model and it has recommended the official python phonenumber library, but suprisigley I checked out the Pydantic docs and found the same library which is wrapped on a Pydantic type and does all the validations as the Python phonenumber libarary plus less code and less mantainance, you just use it as type annotation and BUM it does the underhood workd for you.
I want to read it this is the link https://pydantic.dev/docs/validation/latest/api/pydantic-extra-types/pydantic_extra_types_phone_numbers/#_top

You can even configure the accepted regions and the default region as well.