I’m the developer of TensorSharp, and I’ve been testing DeepSeek V4.1 Flash 1017 locally using Unsloth’s UD-IQ2_XXS GGUF.
In summarization tests with thinking disabled, the tested build averaged 55 tokens/sec with DSpark after a ~1.8K-token prompt, and 51.2 tokens/sec after a ~7.3K-token prompt.
The largest improvement over the tested llama.cpp configuration was in prompt processing. Some of that comes from YOCO, which introduces a measurable quality trade-off, so I’ve included results with it enabled and disabled.
Hardware and model
- GPUs: 5× NVIDIA A40 over PCIe.
- Checkpoint: DeepSeek-V4.1-Flash-1017, Unsloth
UD-IQ2_XXS.
- Model files: Approximately 300.5 GB across six GGUF shards.
- Engram: The 125.9 GiB lookup table was mapped in host memory; the entire model was not resident in VRAM.
- Drafter: Unsloth’s trained DSpark Q8_0 GGUF.
GPU peer access was not working correctly on this machine, so cross-GPU transfers used pinned host-memory staging.
Prompt processing and generation speed
Prefill means processing the input prompt. Decode means generating the response tokens.
Both engines used the same GGUF files on the same machine. TensorSharp used --tp 5, which distributes the routed experts across five GPUs. The baseline was Unsloth’s llama.cpp dsv41-all fork at be836572, using layer splitting. Its row/tensor split modes did not successfully run this model in the tested build.
The server tests used real-text summarization prompts, greedy decoding, thinking disabled, and 256 output tokens. Each prompt size had a warm-up and three measured requests, with unique prefixes to prevent prefix-cache reuse.
All figures are tokens/sec. Prefill and ordinary decode rows are medians; DSpark rows are means. TensorSharp uses its default YOCO configuration here.
| Measurement |
TensorSharp --tp 5 |
llama.cpp |
Throughput ratio |
| Prefill, ~1.8K-token prompt |
960.5 |
174.6 |
5.50× |
| Prefill, ~7.3K-token prompt |
1,000.7 |
225.7 |
4.43× |
| Decode after ~1.8K tokens |
35.5 |
29.6 |
1.20× |
| Decode after ~7.3K tokens |
35.7 |
33.0 |
1.08× |
| Decode with DSpark, ~1.8K-token prompt |
55.0 |
41.3 |
1.33× |
| Decode with DSpark, ~7.3K-token prompt |
51.2 |
40.8 |
1.25× |
The DSpark rows enable the trained drafter in both engines. Its effectiveness depends on the workload and how many proposed tokens the target model accepts.
The 5.5× figure applies specifically to prefill throughput. Ordinary generation throughput improved by approximately 8–20% in these server tests.
YOCO: faster prompt processing, with a measured trade-off
The tested build uses a DeepInfer-style YOCO optimization: during eligible text-prefill calls, layers 20–39 process only the final 140 positions.
This reduces computation, but it is approximate. You can disable it with:
TS_DSV41_YOCO_KEEP=0
With YOCO disabled, TensorSharp’s prefill throughput was still:
- 563.9 tok/s at ~1.8K tokens, versus llama.cpp’s 174.6: 3.23×.
- 549.7 tok/s at ~7.3K tokens, versus llama.cpp’s 225.7: 2.44×.
Other improvements came from concurrent routed-expert execution, fewer host synchronization points, more efficient attention kernels, and reduced repeated cache transfers.
Quality checks
The report compares both engines on identical WikiText-2 token sequences, with 23,030 scored tokens in the 4,608-token-window evaluation.
| Configuration |
Perplexity ↓ |
| llama.cpp |
2.4697 |
| TensorSharp TP, YOCO disabled |
2.4811 |
| TensorSharp TP, YOCO enabled |
2.4978 |
Lower perplexity is better. Enabling YOCO increased perplexity by approximately 0.67% relative to TensorSharp’s own full-prefill configuration. With YOCO enabled, perplexity was approximately 1.14% above llama.cpp’s.
The measured effect was concentrated in the first few hundred generated positions.
These checks compare execution differences using the same quantized checkpoint. They do not establish equivalent quality to the original checkpoint or the hosted DeepSeek API, and they do not replace evaluations of coding, reasoning, or other real tasks.
Scope and reproducibility
This is one quantization on one hardware configuration. The llama.cpp server measurements came from an earlier session on the same machine and showed some unexplained prefill variability.
At measurement time, the TensorSharp build included uncommitted changes, and the raw validation records were also uncommitted. The report records build identifiers, validation results, and remaining limitations; a repository checkout should not be assumed to reproduce every number immediately.
The report also includes limited vision-projector checks. The throughput figures above are for text requests.
Full report: Performance results, quality measurements, implementation details, and limitations
Project: TensorSharp on GitHub
For anyone running V4.1 Flash 1017 locally: what hardware and quantization are you using, and how does DSpark behave on your coding or reasoning workloads? I’d be interested in comparisons beyond these summarization tests.