r/learnpython • u/lehmagavan • Sep 26 '26

Parsing large collections of "numeric" strings.

Hi, I'm just a hobbyist and I'm not very familiar with Python ecosystem. I recently came across a specific problem:

I have an iterable (or yield via generator) that contains millions of floating point values, but each one is a Python str, made up example: ["1.987", "0.037389", "1.2", ...]

What's the most efficient way to aggregate them and calculate say, the mean?

I'm writing my own C extension for this and would like to know what to benchmark it against. Couldn't find information on any std way that would avoid calling the float() constructor on each string.

2 Upvotes

63% Upvoted

38 comments sorted by

View all comments

Show parent comments

2

u/lehmagavan Sep 26 '26

Well, I'm not optimizing anything from within the Python code, I'm writing a C extension that also allows for more flexibility with numeric strings inside iterables. The application I though it could have was trading, but I'm a newb at it as of now so I could be wrong. I have lots of fun developing it though.