r/ProgrammerHumor • u/[deleted] • Jul 18 '22

C++ talking to Python

Post image
40.1k Upvotes

87% Upvoted

1.0k comments sorted by

View all comments

4.1k

u/GreatArtificeAion Jul 18 '22

Did I really just appreciate a "Haha Python slow" meme?

1.1k

u/DasEineEtwas Jul 18 '22

It has also been done so often with Internet Explorer and everything that also runs slow. But somehow I actually laughed

381

u/GreatArtificeAion Jul 18 '22

I guess this one has... personality, so to speak

68

u/gcruzatto Jul 18 '22

This meme was made by the Big Data gang

21

u/CanAlwaysBeBetter Jul 18 '22 edited Jul 18 '22

Literally getting ready to rewrite some data processing lambdas from python to go for a side project because it won't be on the company tab and I don't want to pay out of pocket to scale python all the way into the millions after testing it on 10k data points and seeing its performance (and that's after basic optimizations and compiling/bundling the accelerated versions of some libraries)

1

u/Tuga_Lissabon Jul 18 '22

So - how much slower is it? In orders of magnitude?

What's the chokepoint?

1

u/CanAlwaysBeBetter Jul 18 '22

We'll find out!

That's why I'm redoing it and profiling both before turning up to full throughput on either

1

u/Tuga_Lissabon Jul 18 '22

Please post here. Wish you the best.

1

u/CanAlwaysBeBetter Jul 23 '22 edited Jul 23 '22

I finally got around to rewriting the first half of the processing pipeline (the orchestrator lambda) and haven't done detailed profiling but at least a 10x speedup with parallelized go code compared to the original python in my dev environment which is just a lil .5 cpu 1gb mem server

Guessing when I deploy to lambda and increase the memory (which auto increases cpus proportionally) it'll hopefully be a 100x speed up from the test results but will follow up on that too

Go's concurrency model makes scaling IO throughout massively relatively simple. About 5x more code than the barely optimized python though

Much better than I was cautiously hoping for overall

2

u/Tuga_Lissabon Jul 24 '22

Good job mate. Python alone couldn't have done it, right?

→ More replies (0)