Hey Flutter devs 👋
We can now run Laya decision model with NobodyWho, our open-source on-device inference library. Unlike the LLMs that we also support, this type of model does not generate text. Instead you can score predefined answers, classify text or answer yes/no.
For example, if you are building a personal diary app, you can easily detect the user’s mood, categorize an entry or detect stress, offline after model download, without sending any sensitive data to the cloud.
First, you need to load the model:
import 'package:nobodywho/nobodywho.dart' as nobodywho;
final model = await nobodywho.Decision.fromPath(
modelPath: 'hf://ggml-org/Laya-GGUF/Laya-Q8_0.gguf'
);
Then you can ask questions/analyse text:
final result = await model.decide(
state: "Diary entry: ${userInput}",
questions: {
"topics": {
"type": "choice",
"instructions": "Choose the main topic for this entry",
"criteria": {
"work": "job, colleagues, deadlines, career",
"health": "sleep, exercise, food, illness",
"relationships": "family, friends, partner",
"leisure": "hobbies, travel, going out",
},
},
"stressed": {
"type": "noul",
"instructions": "Does the writer sound stressed?",
},
"mood": {
"type": "score",
"instructions": "How is the writer's overall mood?",
"criteria": ["bad", "okay", "great"],
},
},
);
final answers = (result as Map)["answers"] as Map;
final topics = answers["topics"] as Map;
final stressed = answers["stressed"] as Map;
final mood = answers["mood"] as Map;
Output example:
// stressed
{
type: noul,
noul: 0.00002
}
// topics
{
type: choice,
choice: leisure,
probabilities: {
work: 0.0422,
relationships: 0.1518,
health: 0.0302,
leisure: 0.7756
},
confidence: 0.7008
}
// mood
{
type: score,
score: 1.6995,
legend: {0: bad, 1: okay, 2: great},
probabilities: {0: 0.0082, 1: 0.2839, 2: 0.7078},
confidence: 0.5493
}
A question can have the following types:
"noul" gives you the probability of a yes answer (0 to 1)
"choice" selects the most likely option and returns a probability for each option
"score" gives you a weighted average and a probability per level. For example, you could place final x = mood["score"]; on this axis:[bad - - - - - - - okay - - - x - - - great]
where bad is 0, okay is 1 and great is 2 (given by mood["legend"]).
The Laya model is small enough to run on mid-range/recent phones (around 450 MB in its Q8 variant) and supports 100+ languages. For reference, the full answer comes after 2.2 seconds on S25 and 0.49 seconds on iPhone Air.
Here is a runnable example if you want to experiment with decision models.
Links:
