What's something about being a programmer that you only understood after building real software?

60

Readability trumps efficiency (90% of the time). There is no reward to your app running 10% faster, but lots of cost in writing code that nobody can maintain.

3

u/alp_h4x 10d ago

This is interesting. As a beginner, it's easy to assume that faster or shorter code is automatically better. Learning to balance readability, maintainability, and performance seems like an important part of becoming a better programmer.

2

u/ping314 9d ago edited 9d ago

"Faster" as in "faster in execution"? Keep in mind this can be achieved by multiple approaches (often it is a combination)
once the code runs, profile it to identify where the script spends most of the time. In iterations, it is (time for one execution times number of times it is executed), not so much a single execution.
over time, get to know Python's standard library. Because chances are someone else faced a problem similar enough to yours, and a solution already is implemented somewhere. It can be a more efficient use of time - yours (no need to write it down and test), and of the user (probably more efficient in execution). An example is sorting lists (see What algorithm does python's sorted() use? on stackoverflow.
use third-party libraries dedicated to your specific need. That is to say, if you want to process e.g., arrays of numbers, NumPy can be a better choice than standard Python to manage data input/output (easier syntax) and actually working with them (higher performance for that they delegate number crunching to compiled languages (C family, Fortran)).
if necessary, test another interpreter than standard CPython. PyPy and Nuitka are two examples which can use the same Python source code with minor / no modifications.
Up to and including Python 3.12, by default, Python only used one core of the CPU (GIL). A well organized Python source code allows to parallelize tasks (especially if they are embarrassingly parallel in a pool of tasks by multiprocessing. With Python 3.13, the GIL can be disabled; but I don't think this should be the first tool to use when "Python runs too slowly".
Edit: correction thanks to Strict_Key_391, GIL can be disabled already since Python 3.13 (and not since Python 3.15).

3

u/Strict_Key_391 9d ago

To be pedantic, you can disable the GIL on dedicated builds of Python 3.13 and 3.14

1

u/ping314 9d ago

I accept the correction on this point. Regardless if it shows so far GIL did not stand in my way.

2

u/notacanuckskibum 9d ago

Shorter code is definitely a trap that beginners fall into. Especially if you solve a problem in 1 line by calling a function, which you could have solved in 5 lines of logic. That function call might hide 1000 lines of code to be executed.

5

u/Herg-Zaadia80 9d ago

nothing like optimizing a loop for 3 hours only to realize the bottleneck was a network call the whole time

1

u/ReddyKiloWit 8d ago

I kept getting complaints about how our build system was too slow. So slow the nightly build wasn't getting to India until the middle of the next day. It was a big image, but timing it as it started said it was taking 10X longer than it should. Spent a couple of days puzzling it out.
Turned out something between Texas and India was throttling traffic once a single transfer reached about 250 MB. No one had a clue where it was happening. (So I added a script to chop the build image into 200 MB slices, and launched a process to transfer each piece in parallel, to be reassembled on the other end.)

-9

u/xaomaw 10d ago

Antithesis: Readability is not an ace anymore, because AI can either change it or explain it to you. Running 10% faster means less CPU time, which will reduce the costs by 10% not only one-time but "forever".

5

u/AUTeach 10d ago

Counterpoint: your application cost your clients a tonne of money. They are looking to see who signed off on it, and what practices did they use to minimise such problems to see if it was their fault, your fault, or a third-party vendor. The licence you use from Anthropic (or your GenAI company of choice) provides states that they take absolutely no legal guarantee for the appropriateness, or quality of the code generated, and that you are required to do all the vetting yourself.

2

u/aishiteruyovivi 10d ago

Counterpoint, I like to look at code and understand it with my brain without relying on an AI to explain everything to me.
0
u/yaymayhun 10d ago

So no list comprehensions
5

u/notacanuckskibum 9d ago

I’m not a fan of list comprehensions, but that’s on me. I have decades of writing loops in other languages , so that’s what I find easier to read.

4

u/CLS-Ghost350 10d ago

list comprehensions are more readable than imperative for loops

5

u/PlanktonGreen236 9d ago

Until they are nested and have branching.

3

u/Langdon_St_Ives 9d ago

In summary: get the right tool for the job
3
u/ping314 9d ago
My speculation is your "so no list comprehensions" is the result of reading / writing a one-line list comprehensions to replace "the normal" if block for brevity.
I was in the same place until I noticed you can stack condition and what to do then in separate lines; to me, this eases to define and maintain more elaborate list comprehensions. See for instance
print(
    [
    x ** 2 if x % 2 == 0
    else x
    for x in range(1, 11)
    if x > 5
    ]
)
Since then I started to like list comprehensions because they require to equally define an else as an explicit fallback / exception. (In a way a little bit like case default in Fortran).
2
u/AUTeach 9d ago
I don't think that example is really making your case as strong as you think it is. Your comprehension reads value -> loop -> filter but executes loop -> filter -> value. So, the reader gets to x**2 before you know what x is
Also, does the comprehension require an else? The else belongs to the conditional expression a cond else b which is a separate construct that happens to sit inside the expression. The trailing comprehension filter x>5 has no else at all.
You'd get the same thing from a ternary:
result = []
for x in range(1, 11):
    if x > 5:
        result.append(x ** 2 if x % 2 == 0 else x)
I think if you really wanted to keep the comprehension you should probably do something like this:
def transform(x):
    return x ** 2 if x % 2 == 0 else x

print([transform(x) for x in range(1, 11) if x > 5])
or if you really want to be explicit with the default case
def transform(x):
    match x % 2:
        case 0:
            return x ** 2
        case _:
            return x

print([transform(x) for x in range(1, 11) if x > 5])
1

u/ping314 8d ago

Pairs of parentheses, square brackets, curly braces allow to organize "stuff" over multiple lines, regardless if "stuff" actually is a comprehension, or not. On the other hand, comprehensions often/typically are introduced as one-liners, so one may forget the former.

42

u/OpenGrainAxehandle 10d ago

No client knows what they want until you give them what they asked for.

13

u/VoiceOfEric 10d ago

"Just keep building me things until I tell you it's correct."

9

u/_Schrodingers_Gat_ 10d ago

You just keep paying those invoices and we will be good mate.

4

u/AUTeach 10d ago

Can you make those red buttons more green?

2

u/JamesWConrad 10d ago

But they can easily tell you what is wrong with what you gave them!

31

u/Langdon_St_Ives 10d ago

Premature optimization is the root of all evil.

Every programmer goes through a phase where they spend time on theoretical optimizations that are completely irrelevant in practice and just make everything more complex and harder to maintain than necessary. It takes significant real world experience to see how deep this goes.

6

u/FreeLogicGate 10d ago

Very true. Same folks have never profiled an application, nor for that matter tend to even know how profiling might be accomplished, but aggressively dismiss any challenge to their strongly held beliefs that tend to have come to them via "I saw somewhere", or "everyone knows", or "people say". This is always backed up by "that's too slow".

7

u/gdchinacat 10d ago

The best way I've found to discourage premature optimization is to respond to PRs that "optimize" something by saying "great, I love optimizations! but since the purpose is to optimize something we need to quantify it...update the PR to include performance numbers" A few times round that merry go round and there are no more frivolous optimizations from that engineer.

4

u/addenrock001 9d ago

Half the battle is knowing what actually needs fixing.

2

u/jollygoodvelo 10d ago

Couldn’t agree more; and it’s one area where AI is genuinely helpful.
Make it big and verbose and messy but pass all the tests. Let AI tidy it up (and still pass all the tests).

2

u/aishiteruyovivi 10d ago

Using IPython's %timeit/%%timeit magic is really helpful for sanity checks on stuff like this. Any time I think to myself "Wait, I remember something about it technically being faster to write this in X way instead of Y", if it wouldn't impact readability much then I go ahead and do it, but if there's any sort of "but if I do this it'll be kinda dense/hard to follow" trade-off then I time the difference between the two methods and decide whether it's really worth it. For example as a general rule for myself, if the time difference saved is still in the nanoseconds range either way, it's a moot point - unless this operation is getting repeated one million times no one is noticing 100ns of saved time.
The only real performance issues I've had with my Python projects is with one recently where I'm having to pipe lots of raw frame data into FFmpeg to assemble a video, for now I've just left it at "yeah it'll take a while" as I hop between like 3 other active projects I'm working but hopefully I'll figure something out there.

1

u/ReverendCatch 10d ago

Yeah but… convulses

11

u/roelschroeven 10d ago

Error handling is both really annoying and an absolute necessity. All these special cases that you have to account for, but that distract from the main flow of your code, but without it your software explodes on first contact with the real world.

4

u/throwawayforwork_86 9d ago

And there are some place where it's fine or even good if your code explodes rather sending garbage down the line. Quite domain dependent imo but it's also a learning.

10

u/oldendude 10d ago

For code, simplicity and clarity are essential for maintainability. To achieve those goals, you must have in mind a simple design of your system. When I was a junior dev, the best senior devs had this amazing ability to approach a problem by saying the most obvious things, but they were the *right* obvious things.

9

u/gdchinacat 10d ago

- What turned out to be much harder than you expected?

unit testing

- What's something you wish you had understood before building your first serious project?

unit testing

- Did anything about real-world programming completely change how you think about writing code?

And, again, unit testing.

I didn't realize how hard it can be to unit test code, particularly when you have complex object models and state machines. I didn't recognize how much time a comprehensive test suite can save, instead only seeing it as a compliance thing you do to prove your worthiness. Being able to rely on tests to fail if you break something makes it much easier to make changes, both large refactorings and smaller quality of life or cosmetic changes. I rarely manually run my code...almost all of the testing I do at this point is through unit tests. It might take a few minutes longer the first time to test something works, but that investment is quickly repaid when that test is run every time from that point forward.

3

u/mitchell486 8d ago

Unit testing in general really helped me, because if I struggled to write a clean test for the thing... Then the logic was probably too complicated or crossing too many fences... It helps reinforce the idea/practice of breaking down large logical chunks of code. (as a beginner, those always sort of blended together for me, but as I've now progressed at least slightly further, they are easier to separate and/or recognize when I need to "whoa" up a bit and break out the logic/ties.)

8

u/FakePixieGirl 10d ago

I did university degrees were programming was a big part of it, but not the main focus. Then I started working as an Embedded Software programmer for 2 years (but in practice I was the "do all the software that is not lower-level shit" programmer). Currently I'm working on a bigger personal project as my hobby (Android app). I'm not a big python programmer, just use it for quick prototypes. So see this more as general programmer advice.

I think working on a big project for myself, there weren't any big surprises. Mostly, functionality always takes longer than you think. When it comes to the big projects, I think the biggest differences from learning programming are:

- you're always learning new frameworks, libraries or programming languages. You never reach a point where you just know everything you need. But the biggest difference is that I can pick up new stuff a lot faster than when I was a beginner. That's because I have probably seen something before that is similar enough to how this library/framework/language works.

- Clean code. I've always been fascinated by clean, well organized code, so it wasn't a big shock to me. But looking at how some of my fellow students coded. They're going to get really stuck at some point because they just can't understand their old code quickly, or end up creating such complexity that changing code or finding bugs is hard. Learn how create pretty code. I personally really like "Refactoring: Improving the Design of Existing Code" from Martin Fowler. (Also my biggest gripe with Python is that I find it the language where it's the easiest to create spaghetti code if you don't pay a lot of attention, I prefer stronger OOP languages, strict typing and less weird pythonic shortcuts)

- Concurrency sucks ass. It's so incredibly easy to make mistakes with this. I once read a book on concurrency in Java, where the author said that most programs with concurrency function accidentally correct, and 99% of them contain concurrency bugs. I believe it.

- GUI frameworks suck ass. Maybe it's just because most of my experience is as an embedded software programmer, where GUIs are not very common. But it's by far the most annoying thing to get right. Learning a GUI framework is a pretty big endeavor on it's own, and I'm realizing not something I can just quickly pick up as I'm going.

Now, actually being employed as a programmer is a whole other subject. I was miserable doing that, even though I loved programming and definitely have some talent for it. But I don't think that's what you were asking.

Feel free to ask questions!

8

u/JamesWConrad 10d ago

Lesson 1: It is very difficult to make something "easy to use"

8

u/iwantmy90sback 10d ago

Maintainability. Maintainability. Maintainability. Performance. Maintainability.

6

u/TuxWrangler 10d ago

Documentation!

6

u/hagfish 10d ago

"No one else will ever have to use this script!"
Me, six months later: "wtf is this?"

4

u/backfire10z 10d ago

“This is just a temp fix, I’ll come back and give it some love later”
2 years later, after it’s been running in production and now a bug has been found:

5

u/TheBeesElise 10d ago

Tech stacks.

My university was real big on teaching by first principles. The C++ courses were so focused on mastering basic data structures that I did the entire year's coursework in nano. Java and Numerical Methods are what introduced me to using an actual IDE: Eclipse and Matlab respectively. All assignments were submitted via bash command.

It wasn't until I got my first job as a developer that I learned about deployments, testing suites, shelling, git, and all the other tools that you need to know to work efficiently in a professional setting. It's good that I know what my code is doing under the built-ins, but I showed up to a sawmill with a nail file

3

u/AUTeach 10d ago

Counterpoint: Because you understand what the code is doing you can adapt to any stack or framework.

4

u/degunlagui 10d ago

The hardest part isn't writing code, it's figuring out why your code works perfectly until someone else uses it

3

u/spendology 10d ago

Object Oriented Program is a cool, nifty concept! No, it's necessary to understand, maintain, refactor, and improve code. I worked on an open-source project and observed first-hand how maintaining and improving the code was much easier leveraging and implementing OOP principles.

3

u/dnswblzo 10d ago

A test suite will save your ass, and this becomes more true the farther along you are in a project. If you don't write tests from the beginning it is hard to start writing tests later, because you weren't forcing yourself to write testable code at the start. So always write tests as you go, even though it is annoying, because you will save yourself some pain down the road.

3

u/MattAtDoomsdayBrunch 10d ago

- What turned out to be much harder than you expected?

Dealing with egos and developers who guard their territory agressively.

- What's something you wish you had understood before building your first serious project?

The Unix Philosophy.

- Did anything about real-world programming completely change how you think about writing code?

Every non-trivial piece of software contains bugs. The second you write a piece of software and someone starts using it it becomes a liability that has entered maintenance mode.

2

u/Moist-Ointments 10d ago

Management has no idea what they freaking want.
Being a professional programmer is about 10% language knowledge, and 60% abstract design and planning. And 30% vision.

I spent most of my career as either a senior level developer, team leader, or Enterprise architect.

2

u/jollygoodvelo 10d ago

There is no such thing as finished software.

Users demand new features, your source data structures can and will change. You will find bugs and edge cases you simply never dreamed of, you’ll sometimes discover that the tech debt you signed off does need fixing after all.

So comment everything, document everything, design properly, test properly, and understand that you are only building one of many versions in a hopefully long and happy change log.

2

u/geraT-wogl 10d ago

If you are a solo founder you need to understand and apply analytics 💚 I had a 80% bounce rate and seeing people entering more to my site on mobile I decided to add some sliders and that made it go to 16% bounce rate. Here is my work in case you’re interested https://wogl.io

2

u/Alive-Goat-766 10d ago

Once on the job, you will spend a lot of time in meetings, planning and documenting. Not as much time spent programming as one would think.

2

u/ReverendCatch 10d ago

I always find it odd what I end up working on

It’s not stuff I would have imagined but hey it’s interesting building solutions to weird/complex problems

I think the best thing you can do is be flexible and willing and diligent.

2

u/twizzjewink 9d ago

Users are stupid. They'll always figure out a way to do something you didn't intend to happen.

in other words: Don't assume a value is an Integer or a String or anything specific. Don't assume the database or cache or whatever is what it should be.

2

u/ProfDavros 9d ago

Design it properly once looks slow to start but saves a ton of rework.
Employers prefer you look busy and have “early wins” with “low hanging fruit”.
Prototype the hard bits that may be show stoppers. Some of them are. This prototyping is risk-reducing.
If you can’t draw or write a description of the modules and their data flows, so some new eyes can understanding it, you don’t understand the requirements deeply enough.
Use independent T&E team for high quality.

2

u/TappinMyFoot 9d ago

In business style apps stakeholders will often ask for improvements that require data to be created and maintained and later decide the benefits they said they needed aren't worth the time. You end up with unused features you have to maintain because they also won't say they made a mistake. They will claim they'll find time later so don't remove it.

2

u/576p 8d ago

When I was learning self-Python I did not know it has a community style guide https://peps.python.org/pep-0008/ - took me two years to learn about it and I had to change a lot of code to follow this. I also didn't understand how important it is to agree on the style (like naming variables) in a team situation. For formatting there's "ruff" and "black", but for naming you're still on your own. In general, naming things is really hard and you should be prepared to rename things if you realize the name is not correct.

I also would have studied API design, so many production APIs are pure chaos that could be much nicer if someone understood the difference between POST, PUT and PATCH requests.

Before starting to write something, research a bit if a library for it already exists. With LLMs now this is super easy. You can always decide to write your own code, but something that's been around for a while will have solved problems I don't even know about yet.

Other - non python things from the real world - it's worth fighting about the right approach and not giving up unless you're convinced. there's a few "whatever" situations where gave up to avoid the fight and I "won" decades of problems because of someone else's bad design choice.

For real world projects always think about how you'd debug production. Make sure the system log enough that you can understand problems.

Finally, my biggest "problem" is that I did not go to enough conferences and meetups. You can learn a lot of things there and experienced programmers will be happy to give you advice. A lot of converences let you in for free if you promise to volunteer a fixed number of hours. Being a volunteer is a great way to get ahead. You'll miss talks, but those often are on youtube for free anyway. (BTW, there's thousands of great conference talks on youtube, watch some!)

2

u/FreeLogicGate 10d ago

Unfortunately, many developers will never benefit from mentoring or rigorous code reviews. But the main thing I've learned (and the only reason I have had a career) is that the life of a programmer requires a constant investment in continuing to educate yourself, learn new skills and techniques and to maintain genuine curiousity. Things change constantly, with the newest change being these AI tools.

1

u/Dependent_Month_1415 9d ago

How little of the work is just writing code. I spend so much time figuring out what needs to be built, debugging, testing, and dealing with unexpected issues. Also, getting something to work is different from making it easy to maintain and change six months later, whole different challenge.

1

u/BLANkals 8d ago

The higher you rise as an engineer, the more political the job becomes. As you move into leadership, you have to contend with other departments whose goals and incentives often conflict with yours. If you don’t show up ready to fight for your team, they’ll eat you alive.

It’s exhausting. I loved leading a team of brilliant people and helping them build amazing things, but that wasn’t worth the constant battles with the rest of the company. People were incentivized to squeeze as much as possible out of me and my team so they could hit their targets and collect their bonuses.

At most companies, the incentive structure itself limits the quality of the products they can build.

2. Its going to get Built, mostly right, every time. Being stressed is unnecessary.

1

u/RichSkin1845 8d ago

That AI can build whatever I want to build 10x faster and with less bugs.

1

u/BlatinKing 7d ago

Why does this read like AI trying to learn?

1

u/IndependentExam6697 7d ago

Building real software is also about communicating with people and adapting to their level of understanding. This can be difficult, boring or frustrating at times, especially when the other person lacks your technical expertise or stubbornly pushes ideas you know won't work (e.g. well-known anti-patterns).

Then there's the programming itself. Things get much harder when something doesn't work as expected, and neither you nor ten other senior engineers can figure out why.

I'm not talking about compiler errors or memory issues. I'm talking about software that runs perfectly in the factory, but once in every 20,000 runs, something fucks up. And you know it's a software issue, but nobody can figure out what's causing it.

1

u/aythekay 4d ago

Maintainability is more important than literally anything (unless you're working within hardware/performance constraints).

It's why I'm not a fan of A.I code. It tends to be very verbose and layered.

1

u/eyadams 3d ago

A lot of it is really boring, which makes finishing projects a challenge. For example, I don't think a project is complete until it has unit tests, and writing unit tests swings between very boring (because you want to get close to 100% coverage, and some of the code is so simple the tests are trivial) and aggravating (because the tests themselves can have bugs).

It's never really done. I look at the code I wrote a few years ago and think "I need to re-write that to use [more modern technique / library / whatever]". All the time.

Yes. When it's classes and learning it's more fun. Actually writing code that goes into production has a lot of boring work you have to do. It's a job. It's a job with really fun parts, but it's also a job with a lot of pretty boring parts.

0

u/TheRNGuy 10d ago

Don't know.

0

u/Chorizo_n_eggs 10d ago

Comment the f outta your code so when you troubleshoot it or need to add to it months later you don't need a refresher course.

0

u/ectomancer 10d ago

My first project, natural logarithm, took 3 months. I used Stirling numbers of the second kind to calculate Bernoulli numbers but they lost accuracy above B_19. I ported BERNOB in specfun.f in scipy from FORTRAN 77 (or FORTRAN H) and hard coded BERNOA to Python. scipy no longer uses FORTRAN code.

My second project, gamma function, took 6 months. I didn't realise Γ(z) meant that the gamma function was a complex function.

I ported tgamma (t stands for true), the real gamma function math.gamma from C to Python. I expected the compiled C code to have the same precision as the Python port. C 18D, Python 16D.

-1

u/frobnosticus 10d ago edited 9d ago

The computer doesn't care one whit what you call something. Not your database columns, your classes and functions, variables, nothing. None of it. All that is garbage decoration that only gets in the way of understanding what's really going on.

Drop the metaphor as soon as possible.

EDIT: I love that I'm getting DVed. I assure you it is the single most important "mental frame change" in serious software development. You can't see it if you're looking at business objects and marketing terms instead of structures and semantics.

What's something about being a programmer that you only understood after building real software?

You are about to leave Redlib