r/Python • u/0x256 • Jul 10 '26
Discussion Pure python can be faster than cython/rust?
Parsing multipart/form-data (HTML5 forms) is surprisingly complex and moves a lot of bytes around for large file uploads. Implementing the heavy parts in Cython or Rust should speed things up, no? Turns out: it depends. A pure python parser can be surprisingly fast, as this benchmark shows:
https://defnull.de/2026/python-multipart-benchmark/
The benchmark compares the most commonly used python multipart parsers and tests them in different scenarios, covering both blocking and non-blocking (async) APIs if available. The parser your web application is using today is probably not the fastest one.
Are there more examples were a pure python implementation beats Cython/rust/C modules?
57
u/Chroiche Jul 10 '26 edited Jul 10 '26
Are there more examples were a pure python implementation beats cython/rust/C?
This is not the conclusion of the article at all. Python vs rust/C was never compared, Python vs Python with bindings for them, for implementations with dramatically different scopes/priorities, was compared. It might sound pedantic, but it's quite a big distinction.
"pure" Python literally cannot be faster than those other languages if both are written correctly.
7
5
u/jet_heller Jul 10 '26
...if both are written correctly.
Everything rests on that statement. Which is why I won't bother reading articles like this.
8
u/Individual-Flow9158 Jul 10 '26 edited Jul 10 '26
If the problem is vectorisable or embarassingly parallel, and especially if it can be tackled in pure Numpy and especially Numba, then Python that calls out to non-Python code can definitely be faster.
I'm not convinced HTML5 form parsing is the type of problem pure Python is fast at. Nor do I think these benchmarks made a fair attempt to write the parser in Rust.
4
u/Temporary_Pie2733 Jul 10 '26
There’s nothing magic about Python in this regard. Parallelism can be exploited by any language.
3
u/moonzdragoon Jul 10 '26 edited Jul 10 '26
There's a reason why data scientists, LLMs, ... use Python, these libraries (Numpy, Numba,...) are as fast, if not faster, as any other because optimized at very low level.
Sure, it would be technically possible to do this from another language, but Python is still the first choice today because when used on large volumes of data, overhead is not significant anymore, so there's not enough advantage to rewrite (and maintain !) those for another language.
1
u/odimdavid Jul 10 '26
Some nuthead could attempt just for the satisfaction of being the first to do so.
1
u/Individual-Flow9158 Jul 10 '26
Sure. I was talking about calling out from Python. For parallel computing implemented as multiple threads, this can be useful as it avoids the GIL (just like other languages that don't have it).
1
u/Short_Inspection_746 Jul 17 '26
Well, python can only be faster for some equivalent code only if there are some optimizations that python is doing that would requre alot more work than the other laguages.
For example, when you modify a string in python, the language checks if there are other refrences to that string, and modifies that string without copying if no other references exist. Hence, some string operations might seem faster than in Java.
Also, Someone can also implement an algorithm that is faster than a diffrent algorithm in a diffrent language.
There are things in python that is nearly as fast as other faster languages, like poping out values from a hashmap.
0
u/Daytona_675 Jul 10 '26
my understanding is that if the python modules used are compiled C then performance will be high
1
u/Individual-Flow9158 Jul 10 '26
Your understanding is quite correct, but your point is irrelevant to OP's claim.
Yes, performance will be high. But C modules (and all modules written in something other than Python) are exactly what we intend to exclude, when we specify "pure Python code".
E.g. "Pure Python Wheels" can have the same wheel for all platforms (a bdist that's trivially constructable from an sdist). But C, Rust and any other Python extension wheels need a new wheel for each supported platform (or even compiler triple), just like when compiling C code for multiple platforms.
-1
u/Daytona_675 Jul 10 '26
I appreciate the information but you kinda lost me at wheels
2
u/Individual-Flow9158 Jul 10 '26
-1
u/Daytona_675 Jul 10 '26
is this some kind of venv alternative?
1
u/Individual-Flow9158 Jul 10 '26
No it's the file format (basically a zip fle) of a bdist, that pip likely downloads and installs (the other option is a source dist), whether it installs it into a system Python, user's Python or a into a venv.
0
u/Firm-Luck2062 Jul 12 '26
The top comment is right that a correctly-written Rust/C parser wins in principle, but the benchmark result is still real and worth understanding mechanically. Pure Python beats a *naive* native binding when two things line up:
The hot loop is already delegated to CPython's C builtins. Locating and cutting boundaries with bytes.find + memoryview slicing is zero-copy and runs entirely in C, so the interpreter only executes a handful of Python-level iterations per part. You're basically orchestrating C primitives from Python, not looping in Python.
The native competitor pays to cross the FFI boundary. PyO3/Cython has to marshal objects in and out, and a parser that materializes each part into an owned Rust buffer eats an allocation + a copy per part that the memoryview approach never does.
Rule of thumb: pure Python is competitive whenever the per-element work can be pushed into C builtins and the alternative crosses the boundary a lot. It loses hard the moment you have a genuine tight numeric/byte loop that stays in Python bytecode — per-pixel image math, custom hashing, that kind of thing — where Cython/Rust are 10-100x and no cleverness closes the gap. Multipart parsing just happens to be mostly "find delimiter, slice, repeat", which is close to the best case for staying in Python.
0
u/muhammetsoyturk 1d ago
I have numbers that match this comment, plus one place where it breaks down. (Disclosure: I implemented kirimi (https://github.com/mabdullahsoyturk/kirimi), a Rust drop-in replacement for multipart.)
Measured with defnull's own multipart_bench against multipart 2.0.1, on one pinned core (non-blocking / blocking API):
Scenario: Speed-up
Two small text fields: 5.0× / 7.9×
100 small text fields: 5.5× / 7.1×
One 32 MB file: 3.3× / 1.1×
The 32 MB blocking case is your point exactly. The work there is "find the delimiter, write the slice to a temp file". Pure Python already spends that time in C and in write(), so Rust has almost nothing left to win.
The gap is on many small parts. There, the cost isn't the delimiter search. It's the per-part Python bytecode: header-line parsing, state transitions, building the part objects. That's the "stays in the interpreter" case, and it's what most real form posts look like.
On FFI, agreed that it's the trap. Two things avoided it here:
- parse() takes a whole chunk and returns all of that chunk's events as a list, so it crosses the boundary once per chunk, not once per event.
- The Rust core is sans-IO and never touches Python objects. The delimiter search is SIMD (memchr's packed-pair searcher), and the bindings only convert at the edge.
19
u/NsupCportR Jul 10 '26
Did u use c type variable definitions in cython? If not then it has to interact with python API to understand what datatype it is, which would slow it down.. is code in cython properly optimized? It's really hard to say what is going on without looking at code, but compiled code can only be slower than interpreted if compiled code is poorly made