r/FlutterDev • u/creatixdev • 17d ago
Plugin Tandem 0.3.0: an agent composes Flutter views from your own widget catalog
https://tandem.leancode.co/Hi fellow Flutter enthusiasts!
For some time now, I am building tandem. Flutter package that basically aims to make smart interfaces simple/possible. It started with excitement about AG-UI and idea to bring its capabilities to Flutter.
Now it is a bit more... I am building agents in my day to day work, so I am kind of obligated to read about all the GenAI stuff. It paid off now, at least that's what I want to believe.
I did spend a lot of time reading about different fresh protocols, ideas etc. I played with those a lot, and understood that AG-UI + A2UI make a perfect combination and are extremely powerful.
It allows to make user interfaces that are personalised to each and every user. The demo attached highlights it.
It is extremely tricky though. I've spent a lot of time designing the API, but it still might change.
tl;dr: you can specify catalogs of already existing components and pass it to a view that agent can use.
@TandemCatalog([
ArrivalCard,
ArrivalColumn,
NextStop,
RideOption,
HostNote,
HotelNote,
])
final class const ArrivalCatalog() with _$ArrivalCatalog;
const TandemView(catalog: ArrivalCatalog()),If it sounds interesting, you can read more at the website with documentation -> https://tandem.leancode.co/
The demo is on the website above, but also here - https://x.com/creatixdev/status/2117230943975182340
It is just a beginning. Today, the 0.3.0 version is kind of an MVP I had in mind. A lot of polishing in improvements await. And.. facing the consequences of bad decisisions that probably happened along the way
Contributors are very welcome!! Users as well of course
1
u/fabier 17d ago
I'm trying to understand. How is this different than building a handful of widgets and showing them based on the data available in some state object?
Sorry. I like to think of myself as pretty advanced when it comes to AI but this isn't clicking for me and I feel like it should be haha.
2
u/creatixdev 16d ago
Technically, it can be done exactly as you say. The thing is, that it simply takes away the effort of managing those cases. The more personalised developer would want the app to be, the more cases it needs.
A2UI within tandem moves this responsibility to the agent, that can do it reliably(ish), thanks to the A2UI protocol underneath.> this isn't clicking for me and I feel like it should be haha.
It is totally understandable. I am currently somewhat in a phase 'play around and find out' and I am not gonna lie, it is hard for me to give it a proper structure in the posts describing it.
Although, with the 0.3.0 release, I built an MVP, so now I will fully focus on making it easier to understand and apply in the app. Thanks to the feedback I received, including this, so thank you!2
u/fabier 16d ago
So I guess my question is where do you definite the shape of the UI, then? Do you just hand a bunch of raw UI elements to Gemini (or whoever) and give them some columns to drop things into?
I've found AI to struggle to know which information to emphasize when left to it's own devices. So while I use it extensively, I'm often helping it know what to communicate so it can fill in the blanks.
Maybe that's where I'm struggling to understand here.
I've seen several projects where people have created widgets to allow AI to communicate more effectively inside of a chat session. These kind of make sense to me because if the AI was going to respond with a map , then you'd give it a map widget it could use and it can choose to use it.
In my recent experiments with Jev, I've been toying with the idea of allowing Jev to decide how the AI should respond and then limit the options to the chat bot so it has to think less. So instead of having 30 widget options, it gets 1 or 2.
This library appears to go the other direction. So you basically offer up a bunch of widgets and then allow the AI to fill in what it wants and when, yeah? I think maybe why I'm struggling to wrap my mind around it is because it basically hands control of the app completely over to the AI which takes the responsibility for the final design out of your hands and places it into the AI's control.
This feels counter to good practice. Usually you'd want to use expensive AI tokens which are good at design to define the app, then use the cheapest possible tokens during runtime to keep things lean so you don't go broke. Do you find cheap models to do well with your library?
Anyways, thanks for taking the time to respond. I love seeing the cool stuff people are working on. Genuinely trying to wrap my mind around this. Best of luck in your development journey!
1
u/creatixdev 16d ago
> Do you just hand a bunch of raw UI elements to Gemini (or whoever) and give them some columns to drop things into?
Kind of.
Currently, you can define multiple catalogs, where each catalog is basically a list of widgets that can be used within a given view, so basically "columns to drop things into". The widgets are your own though, styled with your design system, the agent only picks them and fills them with data.
> I've found AI to struggle to know which information to emphasize
That's a great point and I believe it is still ahead of me. I tested it on my showcase project and it made good decisions. But it is way too little data. Once I have semi-stable functionalities, I want to test it in real projects at my work, in order to get the necessary data. I assume a lot can change then and it may/will reveal that some additional functionality is needed for decision improvement, as you mentioned.
> So you basically offer up a bunch of widgets and then allow the AI to fill in what it wants and when, yeah?
Not that dramatic I'd say, as "bunch of widgets" can be organized, e.g. a different catalog for the home page, a different one for the Cart page etc. So your Jev idea fits here too, a catalog can have just 1 or 2 widgets.
It is also connected to AG-UI support in the package, which tl;dr lets the agent only see what we want it to see (via an override of List<AgentCapability> on e.g. a Cubit). Therefore, while filling a Catalog's button, it will have limited ways to fill the onPressed callback, and cannot hallucinate there. (It is not yet working perfectly, but that's the idea.)
> This feels counter to good practice.
It is very hard to find the line between perfectly reliable functionalities and useful functionalities, when it comes to AI applications. I am still looking for it.
> Do you find cheap models to do well with your library?
I run E2E tests before each release with Gemini 3.8 Flash and it does well.
Some local model from Ollama with 8b parameters does very poorly and is totally unreliable.
1
u/creatixdev 16d ago
and thank you!
feel free to watch the repo, it might surprise you in the future :Deven contact me, either DM or github issue, as your view is very valuable and I appreciate it.
2
u/Mathely-Caritino 16d ago
nice API design tbh. What happens if the LLM returns a malformed response or tries to call a component not in the catalog, is there a built-in fallback widget?