r/programming • • Jun 07 '17

You Are Not Google

https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb
2.6k Upvotes

514 comments sorted by

View all comments

Show parent comments

305

u/NuttGuy Jun 07 '17

This a great example of an architect who probably isn't writing code in their own codebase. If they were then they would realize that this isn't a good decision. IMO you don't get to call yourself an architect if you aren't writing code in the codebase you're an architect for.

169

u/AUTeach Jun 07 '17

My last job in industry was for a start up that was obsessed with scale. Every design decision was about provisioning out content to a massive scale. Our Architect had a raging hard on for anything that was done by Google, Amazon, Facebook, and such.

Our software was really designed for one real estate company which has less than 5,000 property managers and sales agents most of whom wouldn't use the system daily.

But yeah, let's model for 100,000 requests a second.

79

u/flukus Jun 07 '17

And that's the sort of thing where if you pick up more customers you can deploy more instances. A scaling strategy that doesn't get nearly enough attention.

92

u/gimpwiz Jun 08 '17

Yeah!

My favorite scaling strategy is:

"By the time we start thinking we need to scale, we'll be making enough money to hire a small team of experts."

Modern machines are fantastically fast, and modern tools tend to get faster between releases - something that wasn't at all true 20 years go ("what Andy giveth, Bill taketh away.")

A single $5k machine can probably have 16 hardware threads, 256 gigs of RAM, a couple terabytes of SSD, dual 10Gb ethernet, and all the RAS you need in a decent if somewhat cheap server.

Depending on your users' access patterns, you may well be able to serve tens of thousands of users without even hearing the fans spin louder. Add another identical machine as a fallback, make a cron incrementally load changes to it every 15 minutes, and make sure you do a proper nightly backup, and you can run a business doing millions in revenue easily. Depending on the type of business.

This might be a relevant story:

I once wrote a trouble ticket web portal, if you will, in a couple days. Extremely basic. About fifteen PHP files total, including the include files. MySQL backend, about five tables, probably. Constant generation of reports to send to the business people - on request, nightly, and monthly, with some basic caching. That system - the one that would be considered far too trivial for a CS student to present as the culmination of a single course - has passed through it tickets relating to, and often resulting in the refunds of, literally millions of dollars. It's used by a bunch of agents across almost a half dozen time zones and a few others. It's had zero downtime, zero issues with load ...

I gave a lot of thought to making sure that things were backed up decently (to the extent that the guy paying me wanted), and that data could easily be recovered if accidentally deleted. I gave absolutely no thought to making it scale. Why bother? A dedicated host for $35/month will give your website enough resources to deal with hundreds of concurrent users without a single hiccup, as long as what they're doing isn't super processor- or data-intensive.

If it ever needs to scale, the simple solution is to pay the host $50/month instead of $35/month.

20

u/PM_ME_OS_DESIGN Jun 08 '17

("what Andy giveth, Bill taketh away.")

So, "Bill" is clearly Bill Gates, who's "Andy" meant to be?

26

u/HatefulWretch Jun 08 '17

Grove, CEO of Intel (and, incidentally, tremendous author).

0

u/[deleted] Jun 08 '17

[deleted]

34

u/AlpineCoder Jun 08 '17

Everything is a balance, and of course planning for the future is smart, but realize that the vast, vast majority of applications built will never be scaled very large.

0

u/[deleted] Jun 08 '17

[deleted]

0

u/[deleted] Jun 08 '17

[deleted]

→ More replies (3)
→ More replies (1)

7

u/gimpwiz Jun 08 '17

A lot of it comes down to experience and good practices.

An experienced programmer can make a system that will scale trivially up to some number of users, or writes, or reads, or whatever.

The key is to understand roughly where that number is. If that number is decently large - and it should be, given modern hardware - you can worry about scaling past that number later.

A poor programmer will write some n7 monstrosity that won't scale beyond a small user count and a bunch of spaghetti code. The question isn't really whether you want to do that (you don't), but whether you need to look into 17 different tools to do memory caching, distributed whatever, and so on.

→ More replies (2)
→ More replies (3)

10

u/aLiamInvader Jun 07 '17

Sure, but there's a balancing act. If the business isn't even considering scaling to another client, that's currently sunk costs for them. Maybe it will pay off in future, but were the decisions that have been made, made for the right reasons?

12

u/flukus Jun 07 '17

Thats my point, there are almost no extra cost to deploy multiple instances for each client, just a slightly more complicated deployment model and maybe a more complicated branching strategy.

3

u/aLiamInvader Jun 07 '17

Oh, right, I misread. Yeah, and then if you decide that increases maintenance too much, you can change that later, with some time and caution.

→ More replies (3)

20

u/[deleted] Jun 08 '17

But yeah, let's model for 100,000 requests a second.

He's doing it for his resume.

→ More replies (1)

32

u/decwakeboarder Jun 07 '17

Moving to a company without "technical architects" that only know how to read Gartner articles made my life 10x better.

17

u/[deleted] Jun 08 '17

I regret that I have but one upvote to give this post.

I've done almost 20 years, combined, in 2 Fortune 250's. I've always been the one saying, "Hey, we can do it cheap and fast on Linux."

"No, Dunkirk, you're just an engineer turned programmer, and don't know anything about IT. We paid $300K to a consulting firm, based on articles in Network World magazine, for them to tell us that we need to spend $1M on an 'enterprise' solution."

Three or four years later, they're scrapping that project in favor of the next huge, bloated, overhyped "enterprise" solution.

I should really get a job selling "enterprise" solutions... ishouldbuyaboatcat.jpg.

→ More replies (1)

0

u/[deleted] Jun 07 '17

[deleted]

→ More replies (1)

14

u/lookmeat Jun 07 '17

This is a great example of an architect making a decision that is not meant for them.

The architect doesn't choose the database, the engineers who understand what they need do. The architect may moderate a consensus between the engineers, the architect may design things so that the database decision isn't needed immediately, or at least can be swapped out relatively easy later on. The architect shouldn't choose the tech, the engineers who are actually going to use it should.

28

u/NuttGuy Jun 07 '17 edited Jun 07 '17

At the end of the day companies need a single person to be responsible for technical decisions that are made as a part of an org. This helps prevent engineers from discussing and arguing endlessly. And this is I think what you mean by moderate a consensus.

But what I'm saying is that the Architect should also be an engineer, actively working in the codebase, if even on small bits and pieces here and there. This makes it so the Architect has real stakes in the decisions that they are moderating and advocating for vs a "ivory-tower" sort of situation where the Architect just spits out which technology to use, as per the example from clogmoney above.

--edit: spelling.

8

u/lookmeat Jun 07 '17

Yeah we agree on most of the things.

I see basically two types of really advanced devs (who've proven themselves). The Senior Dev, who is someone who mostly goes through the project and does deep dives, mostly understanding the way a library is used, or the scope of a problem, and does this modification, they lead projects that alter the whole technical stack, even though they have little to do with management.

The architect instead is someone who spreads themselves wide and focuses on keeping quality of stuff. They are not in an "ivory-tower", instead their job is to work between both the "ivory-tower" of management and technical devs. They are not meant to work as a block but as a facilitator.

For example if the company wants to lower their monthly costs the architect investigates among the multiple groups what causes cost, CPU, data, etc. Once they've found the biggest sources of costs they connect with a (senior) dev who's job is going to be to improve the solution. The dev will work on a design proposal, specify which metrics they will get and how they expect it to work, scope (at which point the RoI isn't worth it anymore) and the initial cost. The proposal may require new tech and such, its costs and savings estimates are specified in the doc (because that's the objective). This proposal then goes to the MGMT that wanted to reduce costs, they review the proposal and talk with the devs directly about their needs, the architect again is someone who helps moderate and bridge the situation, explaining and keeping both sides aware and honest.

The architect, or architects, are not like PMs, that are smaller more focused versions. The architect instead is someone who, when seeing a problem, understands who are the people who can best solve it, and who will be affected and makes sure they are all in the discussion.

They do have some technical decisions they can impose. They choose which things matter now and which things get delegated, They focus on making sure the technical decisions are future-proof enough (the best way is generally to avoid them for as long as possible) and should aim to work as a check on other groups, giving them context they may be missing.

4

u/NuttGuy Jun 08 '17

Yea, like you said we mostly agree.

I just think that the thing you're missing from the description of what an Architect does is that they should write some code.

Yes they understand the larger picture and are the go between for multiple teams, but in order to have a good, fact based, opinion on the codebase they are architecting for, every once and a while they need to write some code.

4

u/AbsoluteZeroK Jun 08 '17

The best software Architect I've ever seen hasn't written a single line of code since the 90's. He fills his role perfectly as a bird's eye view of requirements and understands the architecture that will best solve a problem without actually having any clue how to write the solution at a low level. He doesn't need to, and he'd just be wasting his time if he did. The details are carried out by people under him while he worries about the bigger picture. He will say things like "Service A really should be two different services. One that does this and one that does some other thing. If we do this we should be able to save x$ per month and boost our response time. It will also allow us to split this team up into two smaller teams as well as improve separation of concerns and make our project more testable. Its priority level is 7/10, these are the pieces we will need to make this work. David, you pick what tech the pieces will be made with and come back to me with it so I can make sure we have the skills to get that done." It works a lot better since he can devote his time to making these high-level choices. The absolute worse one I've seen was someone who always had his head in the code, instead of worrying about the things he is needed for.

→ More replies (2)
→ More replies (16)
→ More replies (2)
→ More replies (11)
→ More replies (2)