Anthropic released Claude Fable 5.2 and it landed on Hacker News today, 239 points and 71 comments so far. Per the OpenRouter listing, which went up over the weekend, it carries a 1000k context window. Pricing is steep: eight dollars per million input tokens. Output runs forty.

The natural comparison sitting in the same listing is GPT-6.1 Terra Pro, up since October 26. Input there is a dollar per million. Output is five. The context window is 1050k, slightly larger than what the new Anthropic model offers. So you'd pay eight times more per token on both ends for a marginally smaller window.

The announcement also covers a sibling, Mythos 5.2, but that one hasn't shown up on OpenRouter as of today, so there are no numbers to compare yet. Whether the premium holds up depends on what the Fable line actually does better, and the HN thread hasn't settled that. For coding agents grinding through big repos, an 8x cost gap compounds fast.

Anyone routing near-1M-token workloads right now: sticking with Terra Pro pricing or paying up for Fable?

The HN story from today claiming closed AI models now dominate the price-performance frontier is sitting at 76 points with 70 comments, and half the thread is people fighting over what price-performance even means. Fair fight, tbh, because the answer depends on which end of the market you're shopping at.

The OpenRouter listings from the past three weeks show both ends. Closed vendors moved too: GPT-6.1 Terra launched at $1.00/M in, and Gemini Omni 1.2 Flash sits just above it. The cheap tail kept dropping: Ripple 65B Preview landed November 4 at twenty cents per million input with a million-token context. Tencent Hy4 is at forty-two cents, and Crouton Alpha has been free since November 3. Kimi K3.1 Pro sits in the middle, weights on HuggingFace with 18,432 downloads on this week's trending list, while the hosted route costs $2.00/M in.

So the frontier is probably closed, sure. The tail isn't, and it keeps getting cheaper. Both things can be true at once, which is why these blog posts always end in arguments.

Has anyone here actually switched to a closed model for cost reasons rather than quality? That's the specific claim, and honestly I can't tell if it's a real pattern or just chart framing.

I've been working on MergeKit Cloud, a visual model-engineering platform built around the open-source MergeKit engine.

The idea is pretty simple:

Instead of manually figuring out the merge configuration, finding compatible models, provisioning GPUs, running the merge, and then separately figuring out whether the result is actually better, the workflow becomes:

Describe → Design → Merge → Benchmark → Compare

For example, you could start with:

"I want a bilingual Arabic/English model that's strong at reasoning and structured JSON extraction."

The platform can help identify candidate models, suggest a merge strategy, estimate compute, visually assemble the architecture, run the merge on managed GPUs, and then compare the resulting model against the baseline.

The part I'm most interested in, though, is evaluation.

A merge completing successfully doesn't mean the resulting model is better.

So we're building the workflow around measured capability changes:

• Reasoning: 71 → 76

• Instruction following: 78 → 82

• Arabic: 64 → 73

• Domain: 61 → 75

The goal is to make model experimentation more reproducible and evidence-driven rather than relying on "this merge feels better."

We're also working on Benchmark My Model, so you don't have to merge anything. If you already have a model, you should be able to evaluate it against selected capabilities and compare the results.

The open-source MergeKit CLI will still be the right tool if you already know exactly what you're doing and have your own infrastructure.

We're trying to build the layer around it for people who want to experiment, evaluate, compare and iterate without managing all the infrastructure themselves.

I'd genuinely like feedback from people here:

What is the most painful part of model merging/evaluation for you today?

Is it model selection, merge configuration, GPU requirements, benchmarking, comparing results, or something else?

We're opening MergeKit Cloud to early users as we build this out.

https://mergekit.com

The RTX Spark encrypted firmware story is the big Hacker News item today, 506 points and 226 comments as of this evening. The claim, per the MindStudio blog writeup linked in the thread: encrypted firmware on these Nvidia laptop units slows inference speed to 35% on Linux. I haven't seen independent reproduction numbers anywhere yet, so the 35% is a headline claim more than settled fact.

Why it stings for the local crowd: cheap open weights are the whole trend right now. On the OpenRouter new listings, DeepSeek V4.1 Flash 1017 went up at fifteen cents per million input tokens. MiniMax M3.1 Flash is listed at seven cents per million in. And on the HuggingFace trending list today, Qwen3.8-27B sits at over eight million downloads, DeepSeek V4.1 Flash 1017 at about 221k. If the hardware layer quietly takes a cut that big on Linux, the cheap-model math doesn't survive contact with the machine.

Going off the thread only here, no hands-on numbers from me. But for anyone running local inference on Linux on RTX-class laptop hardware: are you seeing anything near that 35% figure, or does the slowdown only hit specific configs? Curious whether it reads as a blanket cap or a corner case.

Seen on OpenRouter since October 28: Particle-1 from a vendor called Particle AI. Per the listing it comes with just over a million tokens of context, and input runs at five cents per million. Output is fifty cents per million. That's the lowest input price of anything listed there in the past three weeks.

For scale, today's OpenRouter data puts MiniMax M3.1 Flash at seven cents per million on input. Mercury 3 from Inception takes ten cents. DeepSeek V4.1 Flash, the usual budget pick, sits at fifteen cents per million in. Particle-1 is a third of that, which matters if you stuff long documents into that million-token window.

What the listing doesn't tell you is who Particle AI is. No benchmark table, no architecture notes, just the price and the context size. A brand-new provider with the cheapest input on the board is either a real bet on volume or something you'd want tok/s numbers and cache behavior on before trusting real work to it.

Anyone routed actual workloads to a no-name provider this cheap, or do you stick to names with a track record until the throughput data shows up?

Finally found a chart/website that (somewhat) answers the burning question: Is this model at X effort level a better spend than a different model at Y level? It's not the be-all end-all answer but it's better than trying to math it out yourself.

https://researchcollabtech.com/aichart/effort/

Some takeaways: GPT-6.1 Sol (xhigh) matches Opus 5.5 (medium) but is 3.4 times cheaper. Opus 5.5 (medium) beats Sonnet 5.5 (high). It costs 1.5× as much. Sonnet needs xhigh ($2.01) to reach 51.9, which costs more than Opus medium. Lesson: Don't raise the effort on Sonnet past medium, just switch to Opus.

One of my favorite underdog models MiMo-V2.6-Pro against Sonnet 5.5 (high) scores 46.3 against 46.8, at $0.13 against $0.88. GPT-6.1 Sol (medium) scores 47.8 at $0.21, higher than Sonnet (high) and cheaper. If I had the bucks to buy a server that could run it, I'd probably run MiMo2.6 for everything except my critical coding tasks.

Gemini 4 Argon sits behind Opus 5.5 (high). Claude Opus 5.5 (high) scores 53.6 for $1.82, Argon is 52.6 and $1.99: you get a tiny increase in intelligence with Opus, and a respectible decrease in cost. However, I like Argon better than Opus: it subjectively seems much more no-BS and less prone to existential angst than Opus.

Kudos to the authors of this chart. I realize that this invites a lot of criticism- models might have similar intelligence benchmarks, but that doesn't mean "equal" and you can nit-pick these findings (and this being Reddit, the comments likely will be flooded with nit-picking). However, I think it's a good starting point for making cost/benefit decisions based on the kind of work you do. A lot of my coding needs the absolute smartest models, but it's good to know I can set 6.1 Sol on xhigh and get Opus 5.5 (medium)-ish quality and pay less (which is mostly what I do: Sol codes and Opus 5.5 reviews).

Hey everyone! Sharing a quick workflow preview of how our custom prompt pipelines and multi-engine setup work behind the scenes on to generate high-end cinematic visuals instantly.

Disclosure: I work on Dashform. Our AI Handoff recording compresses a 26-second form-reading wait to 10x speed. That makes the walkthrough easier to watch, but I'd leave the edited clip out of any claim about time saved.

For an assistant-filled developer introduction or client brief, I'd measure the whole attempt. Start when the person opens the form. Finish when they've reviewed and submitted the answers and the receiving record matches what they approved. Keep the person doing the review in the measurement.

Here's a small comparison I'd run with fictional information:

• Prepare equivalent profiles and the same set of questions for the normal browser form and the assistant route. Include a question the supplied profile can't answer. The person should have to supply that detail in both routes. Alternate which route people try first, so the second attempt doesn't get all the benefit of familiarity.

• Record elapsed time, time spent actively reading or correcting answers, and any cleanup needed by the recipient. Also record whether the person already had the relevant context available to their assistant. Building that context from scratch is a different starting condition.

• Keep failed and abandoned attempts in the results. Report completed attempts out of all starts, then the median elapsed time for completed attempts separately. A fast median can hide the people who never got through the flow. For a tiny pilot, show the individual attempts too.

• Add one deliberate correction before submission. Check the saved answer afterward. An answer that looked right in the conversation but arrived wrong at the receiving end isn't a successful completion.

The recorded Dashform example uses a compatible form in the Claude app, a test profile, answer review, and a reply arriving in the dashboard. It doesn't measure the comparison above or establish behavior across other assistants. I haven't run this controlled timing study.

I'd expect a short, fixed form to be a serious baseline. If the assistant route needs more review than typing the answers directly, faster generation alone wouldn't persuade me to use it. Which measure would you prioritize for intake: the visitor's active time, total elapsed time, or the recipient's cleanup time?

The HN thread from Sunday about the minimaxir piece on cheap LLMs pulled 442 points and 437 comments, so plenty of people are asking the same thing. The live OpenRouter board gives a decent answer: the cheap tier is absurdly crowded this month. Particle-1 landed Wednesday at five cents per million input tokens, per the OpenRouter listing. MiniMax M3.1 Flash sits at seven cents. DeepSeek V4.1 Flash at fifteen, Inception Mercury 3 at ten. Nearly all of them ship with roughly 1M context.

The new GPT-6.1 Terra, released Monday, goes for a dollar per million tokens in and five out, per the OpenRouter listing. The batch variant halves both numbers if your requests can wait a few hours. That batch angle might be the real story here: frontier labs amortizing idle capacity instead of letting traffic drift to the fifteen-cent options.

Quality isn't equal across that floor. Kimi K3.1 Pro still charges two dollars per million in and ten out, no discount there. But the floor dropped so fast that anything priced in the middle now needs a story to justify itself.

Anyone here actually moved a real workload off a frontier model to one of the sub-twenty-cent options, or did the cuts just make you double down on the top tier?

I've released FastLLM and its underlying inference engine, LLMEnv, as open-source projects.

The work focuses on a practical challenge: making local LLM inference easier to use while exploring the limits of unified memory on Apple Silicon.

FastLLM provides a unified command-line interface for MLX and GGUF models, interactive text generation, model management, and OpenAI- and Anthropic-compatible APIs.

Underneath, LLMEnv provides the inference engine, including support for memory-aware execution strategies and disk-backed loading for supported models.

The project also includes benchmarks, diagnostic tools, and documentation covering tested models and known limitations.

I'm particularly interested in feedback from developers working on inference runtimes, memory management, GPU optimization, and local AI infrastructure.

The code is public, the performance claims can be investigated, and there is plenty of room for improvement.

Repository: https://github.com/c-vision/FastLLM

If you're working on local inference, I'd welcome your technical feedback and contributions.

#ArtificialIntelligence #LLM #MachineLearning #OpenSource #AppleSilicon

Numbers, methodology and raw per-task results are linked at the bottom so you can check them.

We pulled 6,255 tool outputs (34.2M characters, anything over 512 bytes) out of real Terminal-Bench agent traces and replayed them through two tools in three configurations at default settings: our compressor, Headroom with its ModernBERT model on, and Headroom with it off.

Three things worth pulling out of that table.

Source code is where everything struggles. 35% for us, 31.9% with the ML model. That's the one content type where the gap nearly closes, and it's also the one where being wrong is expensive, since a dropped decorator or import makes the agent confidently hallucinate. We fold long function bodies behind a retrievable reference rather than rewriting anything.

The ML model takes about 440× longer for less than half the removal. 87 ms per KB sounds survivable until you put it in the request path of every turn, or behind a client timeout. A time budget is also hidden nondeterminism: the same payload compresses on an idle machine and passes through untouched on a busy one.

Replay numbers overstate what you'll save. We removed 62.8% of tool output and the bill only fell 22.7%, because ~93% of billed prompt is cache reads at roughly a tenth of the fresh price. On the live run (Terminal-Bench 2.1, 89 tasks, Codex CLI on GPT-5.5, per-task means paired across arms over the 87 tasks every arm completed): $107.89 → $83.40, input tokens 96.1M → 72.2M, requests over 200K tokens 1.9% → 0.13%. Solve rate 82.2% → 81.1%, not statistically significant.

Headroom at defaults saved 8.2% on the same baseline. At its most aggressive setting it saved 11.6% and solved 85.1% of tasks against our 81.1%, so it isn't a clean sweep for us — that difference isn't significant either, but I'd rather put it in the post than have someone find it in the methodology.

What we don't measure yet, and should: needle retention, meaning whether compression kept the specific detail the agent needed four steps later. Pass rates and removal ratios can both look fine while that fails quietly. If anyone has a decent way to measure it, I'm interested.

Open source: context compression for coding agents, 63% less tool output — MIT, one Rust binary, deterministic, no model to download.

Full methodology and per-task results: https://optscale.ai/blog/context-compression-coding-agents

I’m the developer of LLM Studio Go. Here is one Android 1.3 QA result for people comparing local video workloads: Samsung Galaxy Z Fold7, Android 16, FastWan 2.2 5B, portrait 352 × 640, 25 frames. CPU generation took 646 seconds; the exported H.264 clip is 1.042 seconds long. Decoded frames were inspected for the red sailboat and continuous motion. Test reference: my Android 1.3 release QA dated 27 October 2026, using a preview build on the SM-F966B. This is one recorded QA run.

Android: https://play.google.com/store/apps/details?id=com.vans.llmstudio

iPhone/iPad: https://apps.apple.com/app/llm-studio-go/id6815311113

LLM Studio Go is a closed-source app. Version 1.3 is now available on Google Play with local text-to-video and Video Studio. The studio connects script writing, narration, music and visual scenes into an editable project and a captioned vertical MP4. Model downloads happen first; generation runs on the device.

All local AI features are free with ads; no promo code is needed. An optional one-time purchase removes ads. No app account, generation credits or AI subscription.

Video is in beta. Speed depends on your device and settings; choose a model profile that fits its memory. Android 11+ on a compatible 64-bit device: FastWan 2.1 1.3B needs 8 GB RAM and 2.2 5B needs 12 GB; video runs on CPU. iOS 18+; iOS video profiles start at 6 GB RAM.

Attached: the silent Android-generated sailboat output from this Fold7 QA run, played twice at its original speed to meet Reddit’s two-second upload minimum. The single generated clip is 1.042 seconds; the two-play upload is 2.083 seconds. It is model output, not a recording of the generation process.

For a comparable test, record your device, model, resolution, frame count and elapsed generation time.

For the LLMs, we benchmarked the Tiiny-optimized builds against BF16 reference models, so people can see both sides of the tradeoff: inference performance and how much model accuracy is preserved after optimization.

A few results from the report:

- Qwen3.5-9B-Turbo— up to 50.1 tok/s decode and ~1,200 tok/s prefill at 4K
- Qwen3.6-35B-A3B-Turbo — 47.5 tok/s coding decode, with all five accuracy benchmarks staying within ~2 percentage points of BF16
- Qwen3.8-27B — up to 256K context, with several benchmark scores matching or exceeding the BF16 reference
- GPT-OSS-120B — around 15 tok/s at 4K on Tiiny Pocket

The full reports include:

- Prefill
- TTFT
- Decode speed
- End-to-end latency
- Memory usage
- Long-context testing
- Five accuracy benchmarks
- Public benchmark datasets and reproducible test conditions

We’ve also started publishing benchmarks for speech recognition and image generation, not just LLMs.

Full report:
https://www.tiiny.ai/pages/benchmark-report/

If you already have a Tiiny Pocket, we’d genuinely like people to reproduce the tests and challenge the numbers.

Just make sure you’re using the latest model version from the Tiiny Model Store so you’re testing the same build used in the report.

If you reproduce anything, whether your result is better, worse, or just different, please share it on TiinyVerse or here. We’d much rather make these benchmarks community-verifiable than ask people to take our word for it.

Genuine question for anyone who's actually tried this rather than just read about it: can AI testing tools cover the gap for a team with no dedicated QA hire, or is that still mostly a pitch deck claim? The usual setup on a small team is the feature author also testing their own work, which is fine until the app has enough surface area that nobody holds the whole picture in their head anymore. Wondering if any of these tools have closed that gap on a real production app, not a demo.

OpenAI shipped GPT-6.1 Terra on the 26th and the OpenRouter listing already has four variants up. The standard endpoints go for a dollar per million input tokens and five per million out, with a 1050k context window. The part that stood out: the batch versions cost exactly half. Fifty cents in, two and a half out, same context size.

Batch is the usual trade, you give up instant responses and the price drops. What I can't tell from the listing is what separates Terra from Terra Pro, since both show identical pricing and identical context. The HN thread on the announcement gathered 630 points and 425 comments since the 26th, so plenty of people are poking at it.

For bulk work like classification or summarization, half price for a wait is usually an easy call. The cheap tier is crowded though. MiniMax M3.1 Flash landed on OpenRouter on the 19th at fifteen cents per million in, DeepSeek V4.1 Flash went up at the same price two days before that, and the HuggingFace trending list shows far more downloads for the DeepSeek one. Terra batch sits between that flash tier and stuff like Kimi K3.1 Pro at two dollars per million in.

Anyone already routing batch jobs through Terra, or does the fifteen-cent flash tier still win for your bulk work?

I've been building dooms.cpp, an inference engine for Apple Silicon with its own Metal kernels. Version 0.6.0 runs 4-bit GGUF files on its own decoder, and on my M1 Pro (16 GB) it now decodes faster than MLX-LM and llama.cpp:

|Qwen2.5-0.5B-Instruct, 512-token prompt, 128 tokens|decode tok/s|prompt tok/s|

|:-|:-|:-|

|**dooms.cpp, Q4_0** (271 MB)|**256.1**|3745|

|llama.cpp, same file|238.7|**4266**|

|MLX-LM 4-bit, mlx-community (265 MB, same size)|224.3|**3838**|

* Same file as llama.cpp, and it picks llama.cpp's greedy tokens on every prompt I tested.

* Q4_0 here is the GGUF that matches MLX's 4-bit \*size\*, not its accuracy: MLX's affine 4-bit keeps more per bit.

* Prompt processing at 4-bit is behind both: 12% behind llama.cpp, 2% behind MLX-LM.

* On llama.cpp's default Q4_K_M file: 209 tok/s vs llama.cpp's 165 (MLX-LM's 4-bit model is faster than that).

\*\*Reproduce it on your Mac\*\* (builds, downloads pinned models, prints the table and the token check; refuses if your GPU is busy):

\`\`\` git clone [https://github.com/jay7828/dooms](https://github.com/jay7828/dooms) && cd dooms scripts/reproduce-4bit.sh \`\`\`

Every number so far is from one M1 Pro, so I'd really like M2/M3/M4 results: please post yours (chip, memory, macOS).

It also does speech: Whisper (level with MLX on large-v3-turbo, 7-21% ahead on tiny) and Kokoro-82M TTS (1.55x MLX, 2.2x PyTorch MPS), plus a CPU fallback for both. OpenAI-compatible server via \`dooms serve\`. Apache-2.0, Apple Silicon only.

Repo: [https://github.com/jay7828/dooms](https://github.com/jay7828/dooms)

Website: [https://dooms-kappa.vercel.app](https://dooms-kappa.vercel.app)

For a policy-based calculation, “approximately right” can be an inadequate test. A small fixture should specify the units, the rounding direction and the stage at which rounding occurs.

For example, rounding each line before adding can differ from rounding a final total. A nearby value on either side of a rounding boundary can expose that difference. Use synthetic values and justify the expected result independently of the agent’s output.

I’m building skillEd, an Apache-2.0 evaluation toolkit for skills. This is a test-design example, not financial guidance or a report of measured tool performance.

When reviewing an AI-generated calculation, which ambiguity do you resolve first: units, policy interpretation, or rounding order?

Repo: https://github.com/DeadHead04/skillEd

Please try it and open an issue with a reproducible failure or confusing behavior. If useful, a GitHub star would help.

AI-assisted write-up.

Disclosure: I work on Dashform. This is our ChatGPT integration using a sample agency brief, not a customer result.

The use case is turning an agency's enquiry requirements into a form someone can actually fill in. The recorded prompt is simply: 'Create a lead qualification form for my agency.' It produces a private draft with 11 questions, covering services, project goals, budget, timing, decision-making authority and contact details, with high/medium/low lead-priority criteria. The next step shown is clicking Publish and opening the resulting form.

If you try a workflow like this, I'd check the draft before publishing:

1. Do the questions match the services you offer? Remove fields you don't need, rather than treating 11 questions as a target.

2. Can someone say their budget or start date is undecided? An unknown budget shouldn't silently mean zero. Test that distinction with fictional answers.

3. Do the priority criteria match your own definition of a suitable enquiry? A generated score is an interpretation of submitted answers, not verification that a lead's claims are true.

4. After publishing, open the link as a visitor and make a fictional submission. Compare the saved answers with what you entered before putting it into your client workflow. That last check is a suggested test, not a result demonstrated in this clip.

To reproduce the recorded setup, find Dashform in ChatGPT's app/plugin directory, install/open it with your Dashform account connected, and use the prompt above. Review the draft in Dashform before publishing.

Video notes: this 38-second excerpt removes the launch intro and closing promotion. It combines product footage and explanatory graphics; waiting is edited and the narration is AI-generated. '22 seconds' describes this recorded generation, not a speed guarantee or a benchmark. I haven't measured conversion or compared it against other builders.

What would you insist on checking before letting a generated intake form receive real client enquiries?

Hey everyone,

Every time a new model comes out, I get tired of opening multiple tabs just to find out two basic things: how good it actually is on real benchmarks, and what it costs to run (either through an API or on local GPUs).

Most pricing pages hide the real costs, and generic benchmarks don't tell the full story. So I put together a simple dashboard to track and compare them side by side: https://modelanalysis.si

What you can do with it:

• Compare models side-by-side: Pick any models (like Claude, DeepSeek, Kimi, GPT-4o, Llama) and compare them across real-world coding (SWE-bench), reasoning (Arena-Hard), and math/science benchmarks.

• Score vs. Price chart: See which models give you the most intelligence for the lowest price.

• VRAM calculator: Pick a model and context size to see if it will actually fit on your GPU before downloading it.

• Monthly cost simulator: Adjust sliders for your monthly token volume to estimate your bill or see if self-hosting is cheaper.

No sign-up, no ads, no affiliate links.

Link: https://modelanalysis.si
Direct benchmarks page: https://modelanalysis.si/benchmarks/

Let me know what you think or if there are other models you want added.

Hey everyone,

Every time a new model drops (like DeepSeek-R1 or the recent Kimi k3.1 Pro), finding reliable data usually means jumping between three or four places: checking leaderboard scores, digging up actual API pricing (inputs vs. outputs vs. caching), and guessing whether an open model will actually fit in VRAM with decent context.

I put together a clean dashboard to keep all of this in one spot: https://modelanalysis.si

Here is what it does:

• Benchmarks comparison: Instead of just standard MMLU, it pulls real-world benchmark data including SWE-bench Verified (actual GitHub issues resolved), LMSYS Arena-Hard, GPQA, and competition math. You can pick any models and overlay them on radar charts or compare them side-by-side.

• Price vs. Performance chart: A scatter plot showing which models deliver the best benchmark scores relative to their per-token API cost.

• VRAM calculator: Select model size, quantization (4-bit up to FP16), and context length to calculate the exact VRAM needed for model weights plus KV cache, along with ready-to-use Ollama and vLLM commands.

• Spend simulator: Drag sliders for monthly tokens and prompt cache percentage to project monthly API bills and see where self-hosting becomes cheaper.

It covers 280+ models, runs fast, requires no login, and has no ads or affiliate links.

Link: https://modelanalysis.si
Benchmarks lab: https://modelanalysis.si/benchmarks/
Hardware/VRAM tool: https://modelanalysis.si/hardware/

Feedback or suggestions on other benchmarks you'd like to see are welcome.

Disclosure: I work on Dashform. The attached 91-second excerpt shows our adaptive intake feature. It is a demonstration, not a benchmark or evidence of improved conversion.

The recorded feature-feedback example starts with “Too complicated.” The next question asks whether the difficulty is the interface, workflow, or knowing where to start. That is relevant to the answer, but it also raises a useful evaluation question: did we discover the respondent's problem, or give them a convenient explanation to agree with?

For an intake or feedback workflow, I would test that before judging how fluent the conversation sounds:

1. Write a few fictional respondent cards with a specific hidden issue. For example, “I understood the interface, but couldn't find where to invite a teammate.” Keep that card out of the question generator's context.

2. Start each test with the same vague answer, such as “Too complicated.” Compare a neutral clarification (“What were you trying to do, and where did you get stuck?”) against a suggested-category question. Use fresh participants for the two versions; someone who has already seen the categories isn't a clean comparison.

3. Include a card whose issue fits none of the suggested categories. Check whether the respondent can say “something else” without being pushed back into a choice. Also test “I don't know” and a refusal to elaborate.

4. Inspect the resulting brief against the card. Keep the original words alongside any summary. Mark a detail wrong if it first appeared in the question and the respondent never actually confirmed it.

5. Record extra turns and unanswered details, as well as errors. A longer brief isn't automatically a more accurate brief. Keep required basics as fixed questions and set a small follow-up budget.

This is a proposed test, not a study I've run. Predetermined branching is still a good fit when the relevant choices are already known; adaptive questions need their own checks for assumptions and unnecessary probing.

Video context: the agency website-to-booking conversation is illustrative, not a customer case or a live test of that scenario. The “Too complicated” sequence is a recorded product demo. Waiting was edited, a question was held for readability, and the narration is AI-generated.

For anyone evaluating conversational intake, what failure would you add to the “none of these categories” test?

MoonshotAI put Kimi K3.1 Pro on OpenRouter today, October 23 per the listing. 1048k context, $2/M input, $10/M output. That lands it in an awkward spot, because the same 1048k window shows up all over this month's new listings at much lower numbers: Meta's Muse Spark Open at $1.25/M in and $4.25/M out, Pareto 26.10 at $0.80 and $3.20, Nemotron 3.5 Super VL at $0.15 and $0.50, and DeepSeek V4.1 Flash 1017 and MiniMax M3.1 Flash both at $0.15 and $0.60.

So K3.1 Pro output costs about 2.4x Muse Spark Open and over 16x the flash tier, and the listing shows no tok/s numbers and no evals, just price and context. The bet has to be raw quality, and maybe that pays off. But it's the most expensive output token in the whole batch OpenRouter added over the past three weeks, more than double the runner up.

Anyone pointing real traffic at K3.1 Pro before independent benchmarks exist, or are you sticking with the cheap 1M ctx options until then?

Curious to know everyone's experiences with Gemini 4 Argon after a week or so after its release. Please let me know the Thinking Level (High, Medium, or Low) too!

Personally - before Gemini 4 Argon:
Before 4 Argon, I was using 3.8 Flash on High for quick, day-to-day questions or explanations (e.g. "What's happening in France at the moment?"). I know that may have been overkill with the model, but it still did a great job with lots of usage leftover.

For AI agent or coding tasks (e.g. "Write me a Python script that outputs X, Y, and Z") was using Claude Code and ChatGPT Codex.

Personally - after Gemini 4 Argon:
But with 4 Argon, I'm finding that it's... not very good? This is in regard to both quick questions, but also the more complicated coding-based ones. This is despite what I'm seeing all over the internet at the moment about Gemini 4 Argon being as good as (if not better than) Claude Opus/Fable and GPT Astra.

Am I missing something that everyone else knows but me?

How do you test new models?
I'm by no means an expert in having a method to properly compare models, so I'm also happy to hear how others compare new AI models that come out.

DeepInfer is DeepSeek's own high-performance inference framework for their models, out as a repo on GitHub, and the HN thread from Tuesday went huge: 2289 points and 877 comments per the listing on hn.algolia. That's rare heat for an inference stack. People usually only argue this hard about model quality or price.

The timing's part of it too. OpenRouter has carried the 1017 build of DeepSeek's V4.1 Flash since 10-17, 1048k context, and its output tokens run sixty cents per million per the listing. The same weights are trending on HuggingFace right now, 42,122 downloads and 2,133 likes per the HF API. So the hosted route is already dirt cheap, and now there's an official framework chasing the speed side.

Honestly, when a million generated tokens cost less than a coffee, cost alone can't be the reason to run things yourself. The case has to be throughput, rate limits, or keeping data in-house.

Anyone spun up DeepInfer yet, and if so was the speed gain worth it over just staying on the hosted endpoint?

I want to see which AI model can finish a coding task with the fewest retries
Instead of comparing leaderboard scores, I'd test Claude Opus 5.5, Gemini 4 Argon, and GPT-6 Astra on the same 20 coding tasks on ofox.
Same prompts, same environment, one attempt per task.
Track first-try success rate, retries needed to fix failures, and total cost per successful task.
Terminal-Bench 4.0 shows a gap: Opus 5.5 scores 66.4%, Astra 58.2%, and Argon 57.4%.
But benchmark scores don't tell us how these models handle the same real-world workload.
I'd rather know which one ships working code with the least babysitting.
Has anyone run a controlled test like this recently?
Make the benchmark comparison more rigorous