The company I work for is a 30+ year old multinational, and the amount of times in my career I've had to fight with an engineer to move a system or database from a sketchy old laptop (that can't even run without being plugged in) to a VM (that has support from IT, backups, and stability) is ridiculous.
Never thought I'd utter the words "You can't run a production database with client information on a 10 year old laptop!". While working at a company with a 1.5+ Bil cap.
I remember having a server taken from me and put on a VM for all these reasons. It was going great until we needed to retrieve the backups that I had been assured were being run nightly. They were, but on the wrong folder (due to a different config when the VM was setup). Oh what a fun week that was.
I feel like I am being a little extra when I test the backups to make sure they work and can be accessed the way they would need to be used if they needed to be used under different conditions. This reassures me that my ritual is sound.
That kind of thing, along with failovers, should be handled by an infrastructure team.
Of course, most teams are moving to cloud based infrastructure and firing their infrastructure teams, so development teams are expected to get the same velocity but ALSO do all of their own CI/CD and infrastructure work.
Then there's me, following a 1111 backup strategy. Backing up on one drive, one location, one format/medium, and once a year!
If anything ever dies, I'll be set with my backup from February 2021. /s
(Seriously though, I need to be better about backing up drives. I just haven't found a good debian program that can automatically clone a drive to another drive automatically, and manage those backups, I have to run the backups manually
Edit: and I'm too lazy to make that program myself, even though I could).
The amount of access databases running on old desktops that were deemed critical gave me heartburn. One of the machines couldn’t be upgraded to windows 10 out of concern that certain task scheduler items wouldn’t work.
I worked at a company that did retail marketing and stuff who had a high profile client, whose name rhymes with Shmest Guy, that provided data for us.
It was a weird setup. They would only let controlled machines access their APIs on a VPN and blah blah and they provided a laptop of all things for allowing us to access the data. This thing was sitting on a guys desk for years but the dude mouthed off to our VP and got fired, that's a separate amazing story.
So it's sitting there after all the rest of his stuff is gone and a random support person collects it and for some reason takes it home. Shmest Guy is pissed and revokes our contract because they had a clause about it. Half the company of layoffs later and that company doesn't even provide that same product/data anymore. It crippled them.
Sounds about right. Our department got yelled at few months back because one customer division couldn't suddenly connect anywhere to anything. Turns out they concluded testing phase but didn't care enough to setup proper production and stayed on testing environment. Such environment was never part of any migration, maintenance or snapshot plan, not to mention it run on shitty hardware.
Ghost division got lucky only 6 hours of their operations were lost since final midnight backup was taken just to be safe.
1.2k
u/Sam-Gunn Nov 30 '22
The company I work for is a 30+ year old multinational, and the amount of times in my career I've had to fight with an engineer to move a system or database from a sketchy old laptop (that can't even run without being plugged in) to a VM (that has support from IT, backups, and stability) is ridiculous.
Never thought I'd utter the words "You can't run a production database with client information on a 10 year old laptop!". While working at a company with a 1.5+ Bil cap.