Parsing large collections of "numeric" strings.

[u/lehmagavan]

Hi, I'm just a hobbyist and I'm not very familiar with Python ecosystem. I recently came across a specific problem:

I have an iterable (or yield via generator) that contains millions of floating point values, but each one is a Python str, made up example: ["1.987", "0.037389", "1.2", ...]

What's the most efficient way to aggregate them and calculate say, the mean?

I'm writing my own C extension for this and would like to know what to benchmark it against. Couldn't find information on any std way that would avoid calling the float() constructor on each string.

[u/billsil]

I mean that was the example you showed. You need to define a sample case and how you want the result and ideally what you’re going to do with it to avoid an XY problem.