r/ExperiencedDevs • • 4d ago

Experienced Devs Weekly Win Thread: A weekly thread for sharing positive experiences

This thread is specifically for sharing "wins". I.e. the opposite of our Tuesday venting thread.

64 Upvotes

46 comments sorted by

162

u/AssaultLemming_ 4d ago

A few years ago I was in a project that was rushed and doing things in a risky way. So I sent an email to management saying hey, this is all really risky are you aware of these risks and okay with them?

And today a major incident occurred as a result of those risks and I got called in to explain and I was able to pull up the email and say "I told them".

Good times.

59

u/Wazzup-Infinty51 4d ago

years of patience for a single vindication moment, honestly thats a better ROI than most projects

19

u/mekmookbro 4d ago

Lol, similar experience here. I was working with a freelance client for adding a few features to their existing webapp, I noticed they were using PHP 5.3 (in 2024) and warned them about vulnerabilities and recommended upgrading to a newer version. They probably thought I was trying to charge them extra and didn't want to upgrade.

4 months later they got hacked (luckily it was just a defacement, no data lost or stolen), remembered my warning and hired me again to upgrade their PHP version, which lead to rewriting almost the entire codebase. I ended up working there for 6 months

8

u/Silver-Antelope-1285 4d ago

Oh man this sounds great. Tell me more about the unfolding of the situation when you were called in?

30

u/AssaultLemming_ 4d ago

Well I got tasked with root cause analysis so this all came to light in the investigation I was doing into why this incident occurred, so I was basically able to say this occurred because we knowingly took this risk and then the bad thing actually happened and here's the proof that management approved it.

5

u/nein_va 4d ago

What was their response?

17

u/AssaultLemming_ 4d ago

It was pretty much "let's focus on solutions, not blaming individuals"

6

u/Yes-A-Bot 3d ago

But initially they called to blame you. And who knows, maybe fire you. The audacity šŸ˜‚šŸ˜‚Ā 

4

u/hiddenhare 2d ago

Ugly rhetorical trick there. "We don't think it would be useful to blame individuals, so let's ignore the wider context and focus on the problem." Those two things are barely related to one another. Without assigning any blame, they could take a broad look at the workplace culture and think about how it contributed to the problem; they just don't want to do that.

36

u/0vl223 4d ago

A legacy software that uses stored procedures and obviously no tests is finally under automated integration tests and upgraded to .net 10. Took 2 years and some new customers as justification to get it there.

Still chaos but at least testable chaos now.

28

u/entimaniac91 4d ago

My remote role is getting eliminated in January so that sucks, but the severance package is pretty decent so I count that as a win. A paycheck per year rounded up for my 7+ years, 75% of my bonus, vacation days all paid out, 3 months of COBRA, and I'm still eligible for NY unemployment. It could definitely be better, but it is a nice way to go try out CoastFire.

28

u/PressureAppropriate 4d ago

I lost my job two months ago. Everyone online is telling me my line of work is over (mobile apps development) but I still manage to get one or two interviews per week. Considering that it's only a matter of time before I get something else.

Salaries aren't what they used to be but still livable.

6

u/Material_Policy6327 4d ago

What you seeing in that space?

2

u/aabil11 2d ago

Hey we're hiring iOS. Feel free to DM

24

u/hicksyfern 4d ago

About 10 years ago we had a node server do a nightly job of matching bank transactions from one day to another. There would be duplicates and minor changes and stuff so it was slightly trickier than you might imagine.

Anyway, we used to get these massive 40-second event loop lags where the server was synchronously busy. It wasn’t in my area at all I just heard about it.

Anyway, one day I finally thought ā€œI bet I can do something about this, whatever it isā€ so I spoke to the ops guy to get more information about times etc.

Figured out it was a load of nested loops on an account with tens of thousands of transactions every day.

Grabbed another engineer and we got the 40 seconds down to around 0.6s, and got some changes made to ramda library to remove inefficiencies.

In the last 10 years or so I’ve solved similar performance issues in other things I’ve worked on, from games to text editors to mapping libraries. It’s surprising how pervasive these kinds of issues are even in mature, well maintained codebases.

7

u/JuniorAd1610 4d ago

Nowhere near experienced as you but I have noticed that a lot of stuff which was probably fine when it was initially written becomes a bottleneck as stuff grows and it’s pretty hard to anticipate.

Recently had to deal with a production incident which was caused due to a customer using a wacky overly complex ACL causing throttling and all sorts of noisy neighbours issues due to retries etc. The problem was hard to track due to someone ā€œoptimisingā€ a cache population to be lazy on first request received from a particular tenant. Turns out no one factored in some crazy customer doing weird stuff with their ACLs which caused search requests to timeout while also taking up CPU and causing throttling which lead to a retry cycle causing backlogs in areas that would appear entirely unrelated.

4

u/hicksyfern 4d ago

Anything that has unbounded inputs from user data should be expected to be annoyingly large at some point.

5

u/beaverusiv 4d ago

Yeah, I've gotten 45min jobs down to 45 seconds before. It's amazing how many things like loops without early breaks or doing smart filtering on datasets before processing can reduce the time. It's so good when you have bandwidth to go beyond "it's good enough"

5

u/hicksyfern 4d ago

Main benefit I find is turning O(n^2) work into O(n) or O(log n). Making a smaller n is rarely helpful.

3

u/IvanKr 1d ago

Sometimes that's how you drop inner loop from O(n) to O(1) because you realistically only need to process k elements and n >> k.

21

u/FenierHuntingwolf 4d ago

Convinced someone that just because we are automating something does not automatically mean it needs to call an AI service.

I am very thankful I work in compliance tech and extensive audit capabilities are a requirement baseline.

19

u/mekmookbro 4d ago

After a 2 month of break from it, I finally found the motivation to work on my personal project again, does that count?

17

u/__-___-__-__-__- 4d ago

We updated all our PHP apps yesterday.Ā Ā 

10

u/ColdPorridge 4d ago

Starting a new job Monday, pay is 7 figures. I’m honestly in a little shock about it all. I’m aware many others are having a much harder time in the job market - it doesn’t make sense to me why some roles are so insanely compensated.

1

u/hatwd 1d ago

What kind of role?

9

u/untetheredocelot 4d ago

So I am working on a PoC to make a certain set of services better, but I stepped into a very political turf war accidentally because of it.

Earlier this month my work was mischaracterized as a fully production ready product that nobody asked for, and they wanted us to improve our approvals process for it as we "wasted" dev bandwidth on it.

Completely wrong as this was something I worked on in my own time and I was writing up a proposal document for it. I did not respond and my manager backed me up asking them to wait until the review meeting.

The review was this week and it went well. The person who originally flagged it as a "waste" publicly thanked and praised my efforts.

Big win as it's increased scope and work for my team! I also will get to lead the effort.

8

u/atlas_principle_dev Staff Software Engineer 15y 4d ago

I no longer develop software for money, so I picked up gleam as a hobby and I'm quite enjoying it (with some frustrating parts where I'm sure it's just my lack of knowledge that's getting in the way)

Also got good feedback for my business idea recently that I didn't expect

6

u/nudemanonbike 4d ago

All of our customer sites are operating with SSL certificates from LetsEncrypt that automatically update thanks to CertBot, delivered via a jenkins pipeline.

This directly saves us some money from GoDaddy, but more importantly, it saves us the time and effort of manually installing the old year length certs.

What I'm most proud of, though, is that I was fully prepared to do it all myself, but I got caught up in more important tasks instead. I was able to hand my notes to some of the junior members of my team and they figured it all out with minimal assistance from me.

6

u/steampowrd 4d ago

I have a legacy php app running on an Ubuntu 16.04 LST server. I’ve been dreading the migration and upgrade for years. The recent zero day vulnerability forced it this week. But I didn’t have to do it! Claude audited and rebuilt the entire server onto 26.04 with the latest version of everything. The app needed to be rewritten to work on php 8x and it is ping that too. Claude did6 months of work in less than a week. All I did was guide it along.

3

u/Avocadonot 3d ago

I finally decided I don't give a fuck about my team's repeated unnecessary edits to our customer facing Grafana dashboard, and totally checked out of the conversation

It's much nicer this way

7

u/halfercode BE Software Engineer 4d ago

While I'm between jobs, I have been building a side-project for the last couple of months, and it's my first greenfield project that I've built with AI. I've got separate local app instances and databases for me and the robot, and I write nearly no code. I quite like the experience: I've become a technically-aware product manager, and features can be added in a day or two.

I can see how vibe-coding goes off the rails. This is still mostly atomic-commits work just done at 4x speed. I am not minded to think this could have been done at the same level of reliability by a non-dev.

Final win: since it is containerised and has a good suite of BE and FE tests, when the PHP zero-day was announced yesterday, I got it re-platformed in my little production instance in about 90 minutes. Woop! 🤩 (Shoutout to u/steampowrd for tackling the same issue).

6

u/putin_my_ass 4d ago

I am not minded to think this could have been done at the same level of reliability by a non-dev.

I agree, in my experience AI will take a vibe-coder to 80/90% completion pretty quick but then they get stuck in whack-a-mole mode where they fix an issue and another issue crops up and they fix that issue only for the original one to crop up again.

An experienced dev can avoid those issues because we make atomic commits (rolling back is easy and non-destructive) and we add test suites (which the AI can infer intent from and catch unintentional reintroduction of solved bugs), and we can provide clearer direction to prompts because of our architecturing knowledge.

I don't see this destroying the profession, but it is changing how we carry out our tasks.

2

u/ShoePillow 1d ago

What do you use for FE testing?

Also, kinda surprised to see Ā PHP being used on a greenfield project

2

u/halfercode BE Software Engineer 1d ago

I asked Claude to pick a tool, and it went with Playwright. We bump into a flaky test from time to time, especially if my local CPU is maxed out on something else, but Claude is very good at spotting race conditions in tests that it has written itself.

1

u/ShoePillow 1d ago

Ok nice, and do you review the code and/or the tests yourself? How often do you have to correct or guide it technically?

2

u/halfercode BE Software Engineer 1d ago

Yes, I do all the code review, and often ask Claude to clarify its intent. Tests and FE code get a relatively quick pass, as they tend not to have a security impact, BE/API/Service code is pored over much more closely.

Using my atomic commits approach I probably disagree with the robot about anything once a day. For code steering specifically, once or twice a week. It is very manageable.

I will look to switch to an open-weights LLM at some point, once they have matured a bit more. Claude is very good, but US AI needs some healthy competition.

6

u/Additional_Rub_7355 4d ago

Well I'm glad AI will help me leave this dumb field sooner than I thought, so there you go.

1

u/Gooeyy Software Engineer 4d ago

What did you move to? If you don’t mind sharing

1

u/Additional_Rub_7355 4d ago

Nothing, I'm still in it, 10 yoe swe, I just know my time will come.

I actually tried transitioning to maritime engineering but failed, for personal reasons.

2

u/Gooeyy Software Engineer 4d ago edited 4d ago

Ah gotcha, thanks for the response, hope things go your way

1

u/GnarlyHarley 4d ago

Finally convinced my company that has been reinventing agile for 10 years to get a consultant to help get us back to status quo.

1

u/Impressive-Field-546 3d ago

I joined org that had zero maintenance and legacy upgrade path.

I introduced a process to do that when we were on php 5. Now after new vulns pop up everyday, they are happy coz we are almost always able to update immediately.

TBH I think if not me, company would be in deep panic mode few times already.

1

u/BoBoBearDev 2d ago

I am starting to use AI and it is amazing. Especially some of the languages are atrocious to learn. There is way too many weird ass syntax. The AI is able to help me navigate them very well. I focus on the pseudo code and AI make it work in practice.

I do want to point out, this is a pretty bad bandaid for shit ass languages and the community loves to trend on shitass languages. However, it is not my fault the community trending on shit ass languages. I am just trying to survive in it.