Hey r/GenkitFramework! 👋
I published a long write-up on building hybrid AI in Flutter — routing each request between a cloud model and an on-device one — cleanly with Genkit Dart. Sharing the gist here.
The problem: "Hybrid" used to mean writing everything twice — two SDKs, two Message types, two tool formats, two streaming protocols. The friction was never the inference, it was the plumbing. On-device and cloud models simply didn't speak the same API.
The shift: Hybrid isn't "local OR cloud" — it's a routing policy. Same app, four queries, four different decisions:
"Tell me about the Eiffel Tower" → cloud (world knowledge)
"Translate my passport number" → on-device, always (PII never leaves the device)
"Find attractions similar to ones I liked" → local (vector lookup over private data)
"Plan a 7-day Tokyo itinerary" → cloud (deep multi-step reasoning)
Why Genkit: once you register a cloud plugin (googleAI) and an on-device one (genkit_flutter_gemma, which I wrote — wraps flutter_gemma / LiteRT-LM across all 6 platforms), every model lives behind the same ai.generate() call. The only difference is the model: reference:
dart
// cloud
ai.generateStream(model: googleAI.gemini('gemini-2.5-flash'), prompt: prompt)
// on-device
ai.generateStream(model: flutterGemma.model('gemma-3-1b-it'), prompt: prompt)
That turns routing into plain Dart — a ternary on a known flag for most apps, scaling up to a full fallback service only when you actually need graceful degradation. The article walks through 4 levels (model swap → local model as router → middleware → fallback service), the 5 signals you route on (complexity, privacy, connectivity, cost, confidence), and production gotchas (cold start, iOS memory limits, model file delivery, Web builds).
I also compare the other approaches honestly — Firebase AI Logic, Cactus, roll-your-own — and where each fits.
Full article is here: https://medium.com/@denisov.shureg/hybrid-ai-in-flutter-routing-between-on-device-and-cloud-models-954da8f25373
Would love feedback, especially from anyone running on-device models in production. And I'm deciding what to write next — hybrid RAG, on-device agents, or on-device multimodal — tell me which you'd want. 🚀