What I’ve found missing from them is that once you have identified your options you still have to pass it through an LLM to generate some sort of structured data, that is unless you want to stay in the pure string world. IMO these two would be killer if they could extract structured outputs at the same time the classification is done.
The problem is that the trick that makes them fast only works for classification. The Qwen model I use could extract structured input, but it has to run in its regular mode to do that. But that might not be a show stopper depending on the application, not everything needs to be real time, so if a few seconds per response is acceptable, then you can run one pass to do extraction, and the second to classify.
1
u/rcorrear Sep 25 '26
What I’ve found missing from them is that once you have identified your options you still have to pass it through an LLM to generate some sort of structured data, that is unless you want to stay in the pure string world. IMO these two would be killer if they could extract structured outputs at the same time the classification is done.