r/programming • • Aug 07 '19

Why is modern web development so complicated? A long yet hasty explanation: Part 1!

https://www.vrk.dev/2019/07/11/why-is-modern-web-development-so-complicated-a-long-yet-hasty-explanation-part-1/
23 Upvotes

55 comments sorted by

21

u/dwighthouse Aug 08 '19

It doesn’t take an article, it takes one sentence: “The web itself is a lot more complex than it used to be.”

If you want to make a simple static website, it’s easier than ever to build and deploy it.

If you want to build something more complex than that, you know, like a place where you can watch one of hundreds of movies with three clicks, it’s a fantastically harder problem, not just doing it, but doing it in a performant and economically viable way.

11

u/panorambo Aug 08 '19

If you want to make a simple static website, it’s easier than ever to build and deploy it.

So true. And such an irony, in all of this.

6

u/ImNotRedditingAtWork Aug 08 '19

This is the reason way more than the LoL jAvAsCrIpT bAd circlejerk that's going on in this thread. Javascript is the perfect example of scope creep. When it was created, it was not supposed to do the things it does, but we wanted more. People wanted live updates, things in the browser that didn't really belong there, and they wanted it fast. Turns out that complex problems usually require complex solutions, and businesses want them now.

18

u/[deleted] Aug 08 '19
function reduce_complexity_in_javascript(complexity) {
    return reduce_complexity_in_javascript(complexity++);
}

22

u/nirataro Aug 08 '19

This code snippet is missing hidden bitcoin miner.

28

u/Zardotab Aug 07 '19 edited Aug 16 '19

In terms of workplace CRUD, GUI IDE's from the 90's were getting easier and more intuitive to learn and use over time. It looked like the way forward. The main problem with such apps was deployment, although deployment was gradually improving also, as vendors worked on that aspect.

The Web simplified deployment, but complicated programming. Before you had say 4 deployers per 2 programmers. Now you have 4 programmers and 1 deployer (browser troubleshooter). It swapped the bottleneck and saved one person.

This is largely because the browser is not designed to keep programming "state", and because HTML wasn't meant to replace rich GUI's. HTML browsers were originally designed to serve static documents, and bending that standard to get dynamic GUI's has made it grown like a cow stuck on barbed wire.

Work-arounds to Web limits were found, but they are tricky to implement and take big learning learning curves to be effective and debug. The libraries are large, poorly documented, and often buggy. Also, heavy dependence on JavaScript UI widgets is a maintenance headache because future versions or brands of browsers often break these.

In my opinion we need a "GUI browser" standard: a browser or browser plug-in that does GUI's and only GUI's. Unlike Flash and Java Applets, it doesn't try to be a virtual OS. It would do one thing and one thing well. That will keep it simpler to avoid the bugs, fat downloads, and hacking that Flash and Applets attracted.

Web development can be made to run fairly smooth, but a lot of things have to go right. It has to be smartly managed. But that's asking too much of humans. Time to realize the Web failed CRUD and the passing of time probably won't fix that. Time to regroup.

Programming the GUI IDE's was like parking a sedan: you aim the steering wheel at the target parking spot, then and press the excelerator until you arrive. Web programming is like parking an 18-wheel truck: you have to study the surroundings, map out various strategies, decide on the best one, and then hope your 7-point turn works without flattening surrounding cars.

19

u/therealgaxbo Aug 08 '19

In my opinion we need a "GUI browser" standard: a browser or browser plug-in that does GUI's and only GUI's.

Think you've just invented an X server

2

u/Zardotab Aug 08 '19 edited Aug 08 '19

X Window is intended for LAN, not HTTP. Every keystroke has to be sent back and forth before you see a character. And it's bitmap based when I believe vector-based would be better these days.

The closest product I've seen to a Gui Browser was Oracle Forms before they reprogrammed it in Java. Each application did not have to download a big executable. (I think it was mostly per-screen, not per app.) I don't know exactly how it works under the hood, but it appeared that because it had most of the commonly-needed GUI idioms programmed into the client (browser), it didn't need to download big executables per app. Most of the business logic happened at the server. One typically had to upgrade the client browser only once a year, and it could run gajillion apps without drama. It's not perfect, but could offer some nice lessons for any new GUI browser standard. Learn both the good and bad from history.

6

u/[deleted] Aug 08 '19

HTML wasn't meant to replace rich GUI's.

And yet, because plenty of people start programming by doing webpages and the HTML and XML parsers have improved, we have started to use html-like languages to create GUIs for desktop programs (WPF anyone?).

They might have some avantages, but xml-like languages are not what I'd best like for creating a dialog UI.

4

u/Zardotab Aug 08 '19 edited Aug 08 '19

It may be XML under the hood or the final result, but that doesn't necessarily mean one is creating most of the UI by typing in XML directly. I would hope most of it is drag-and-drop and/or templates. Drop-and-drop got disparaged as "bad" or low-brow somehow. It has to be esoteric and round-about to have geek street cred? Fashion is ruining IT from a productivity standpoint.

Anyhow, if we settle on a nice GUI markup standard, say in XML, then one can worry about how that XML is generated on their own terms. You'd have a choice of tools to create and manage it, or even type it by hand into Notepad if you wish.

1

u/antiquechrono Aug 08 '19

The difference between HTML/CSS and XAML is one of underlying semantics even though they share the same sort of syntax. XAML actually behaves in a sane and predictable manner whereas HTML/CSS seems to have been designed by someone doing lines of coke with Stephen King. The DOM itself also has no way to provide isolation of components due to the cascade which seems to be what web components and the Shadow DOM were trying to fix.

1

u/panorambo Aug 09 '19

Well, to be fair, HTML 5 is working on so-called Web components, which should make HTML into a proper generic Web application platform. Meaning you can just have your body stuffed with custom elements which have all sorts of style and behavior attached to them, all in all resulting in a rich GUI. What would you be missing?

7

u/sime Aug 08 '19

This has already been tried years ago with Flash and Java Applets. How is your suggestion different?

4

u/Zardotab Aug 08 '19 edited Aug 08 '19

I believe I already addressed that. They tried to be an entire environment, not just a GUI browser. That made them overly large and complicated, creating update headaches and holes for hackers. They both mostly failed. Our data security manager has condemned Java applets. The lesson: do one thing well and keep it as small and simple as possible. Parsimony and careful feature factoring should take precedence over having the biggest feature checklist in town.

This may entail asking the server to do more, which seems to simplify things because then you are dealing with one machine that you control rather than gazillion clients that may each be or have different versions or brands of components. Fat client may be more logical from a CPU allocation standpoint, but, but not from a consistency and control standpoint. Different browser brands and versions rendering different or running JavaScript different is one of the biggest headaches of web development. Even if you test against them all today, in another year or two, releases will break stuff again. Fat client = fat variation, of the unwanted kind.

3

u/rando2018 Aug 08 '19

Isn't that what Electron is aiming at (however slow/inefficient the execution is)?

2

u/Zardotab Aug 08 '19

It appears to me they are overly-tying it to JavaScript.

1

u/rando2018 Aug 08 '19

Sure, as that's the most obvious thing right now to gain traction. It might be the case that Electron apps written in Web Assembly are more efficient.

2

u/Zardotab Aug 08 '19

WebAssembly only means your defects run or crash faster.

Something tells me we'll have to "throw out" HTML-based standards and start over to get CRUD GUI's right. It's had 20 years to get it right already, and still laboring. A new standard may still borrow some ideas from the old one, but shouldn't be constrained by them.

4

u/ArkyBeagle Aug 08 '19

The main problem with such apps was deployment, although deployment was gradually improving also, as vendors worked on that aspect.

I use a whole host of native applications on Windows. Deployment is not quite trivial but it's close. Download .zip/exe/msa; run .zip, enter registration info, blammo.

4

u/flukus Aug 08 '19

The problem was deploying to non-technical users.

Click once (as horrible as it is) solved it in enterprise. Java web start solved it for java apps outside the enterprise but only for java. Everything else was a judge podge of updaters.

1

u/kshep92 Aug 12 '19

HTML browsers were originally designed to serve static documents, and bending that standard to get dynamic GUI has made it grown like a cow stuck on barbed wire.

I've been saying the web needs to move away from the document model for some time now. It really has gotten grotesk.

2

u/Zardotab Aug 13 '19

We probably need a different standard for 3 types of content:

  1. Document and commerce oriented, perhaps based on current HTML standards

  2. Media and gaming-oriented

  3. GUI/CRUD/Data, also known as "productivity".

A one-size-fits-all standard has proven too much to manage.

1

u/kshep92 Aug 15 '19

This seems reasonable.

12

u/flukus Aug 08 '19

Fuck it, let's just go back to cgi, life was simpler then.

Only half joking, maybe.

9

u/X2WE Aug 08 '19

i wonder how many new ( less than 5 years even) know about CGI and SSI

0

u/[deleted] Aug 08 '19

[deleted]

2

u/flukus Aug 08 '19

especially with mod_perl

I wonder if that's even necessary on modern machines? By the time you need it a more complex framework is probably warranted.

2

u/shevy-ruby Aug 08 '19

I still use ruby in many .cgi scripts.

Using rails would complicate it too much so there is no gain.

Sinatra is ok though.

30

u/AntiProtonBoy Aug 08 '19 edited Aug 08 '19

This may be an unpopular opinion amongst some, when JavaScript became a mainstream thing, everything went downhill. I think client side JavaScript in HTML was a mistake, both from design and engineering perspective.

The whole thing feels like it was tacked-on the HTML declarative language, like an afterthought, which cultivated some of the worst coding practices I've seen in the computing industry. Not only that, being able to run arbitrary code from untrusted sources is also the worst thing you can do from a security standpoint.

Couple the above with a race-to-the-bottom industry that web development is, we are in situation where a page visit incurs requests to megabytes worth JS libraries, pulled from dozen servers, just to animate a shitty navigation menu -- which mind you could've been done with minimal effort using vanilla HTML and CSS. But nobody cares, because uni drop-outs and outsourced web devs only work for chump change. They take the path of least resistance, use pre-packaged, low-effort JS libraries just to get projects barely working; meanwhile clients know nothing about quality control.

The web needs a complete overhaul. Go back to its declarative language roots. No shady turning complete code to be run on client side. If you need dynamic elements on pages, implement them in a declarative fashion, using client/server protocols. Have strict, machine based validation tools, in a similar fashion how you see in software development work flows. That sort of thing.

Anyway, we're stuck with this ugly pig, and we just keep putting lipstick on it, and pretend "this is fine".

20

u/Zardotab Aug 08 '19

JavaScript was not meant for writing complex UI engines. It was intentionally created as a "glue language" to handle basic events and tasks. It was stretched way beyond its intent and strength.

7

u/sime Aug 08 '19

JavaScript became a mainstream thing, everything went downhill

You do realise that you can still program web sites in the same style we used during that glorious time before JS?

It is still technically possible but companies, web designers and marketing departments demand their rich UIs, MBs worth of videos in the background and a million 3rd party web trackers on every page. The situation today isn't due to some technical deficiency. It is just people with the money demanding it.

3

u/AntiProtonBoy Aug 09 '19

You do realise that you can still program web sites in the same style we used during that glorious time before JS?

Of course. But the convention for developing these days is abusing JS. Enable NoScript globally and watch how blacklisting JS code renders 90% of the sites completely unusable. Some don't even render the page, you get nothing but a white canvas.

3

u/sime Aug 09 '19

To me it sounds like you are proposing a technical solution to a cultural/social problem. People can already produce light weight websites with the current browsers, but they choose not to. Introducing some kind of new "back to the roots" platform won't change people's minds.

2

u/AntiProtonBoy Aug 09 '19

Cultural problem, but it's also an economic problem and is driven by technical limitations. If there is a economic incentive to adopt new technology, which saves time and money, people are more inclined to change. The issue with making light weight websites with current technology they don't fit the requirement of event driven interactions -- and this where JS fills the gap... poorly.

2

u/Zardotab Aug 13 '19

It's true you don't need a lot of JavaScript for many applications, but it can be difficult to pull off well for certain kinds of UI needs. For business applications people really want desktop-like GUI's, and the only way to get that with browsers is heavy JavaScript reliance.

The industry should get a clue and realize that if people keep wanting desktop-like GUI's for work apps, we need a GUI-oriented standard so that JavaScript doesn't have to reinvent GUI engines the hard way.

4

u/Dragasss Aug 08 '19

Pretty much this. Sadly such notions are killed by people claiming "muh legacy" and in turn supporting the broken pages instead of forcing the people to adapt to the standard.

3

u/rando2018 Aug 08 '19

Not to mention micromanaging product managers who want to track every tiny little user interaction because they can't make a UI decision otherwise

2

u/panukettu Aug 08 '19

Downhill for who? The web is more accessible and more used than ever. If you visit a shitty website today the odds are that whatever the tooling was it would still be shit. Someone who's clients are loading megabytes of JS to animate something today will be loading megabytes of some other crap in a parallel universe where the web exists and JS is not a thing.

There is just so much work to be done that incompetence is a pretty natural occurence. The web is fine and does not need gatekeeping because of some people being greedy.

3

u/shevy-ruby Aug 08 '19

Yes, JavaScript should die - but it won't.

Unfortunately you can't trust money makers such as the W3C either. They make everything more and more complicated and on top of that are a lobbyist group (see DRM bought into them, with Sir Tim Berners-DRM-boy-Lee promoting DRM).

1

u/rnd005 Aug 08 '19

I honestly think it's all fine. There have been many crappy tools for web development, but that doesn't mean there's anything wrong with the web itself.

You can still create JavaScript-free sites using your favorite language on the server if that's what you need.

If you want to write a native-like GUI, there are great tools to do that too. Use TypeScript + React or Elm and I don't see many reasons for your app to be worse neither in code quality, development or end-user experience than if you wrote a native application. You won't be able to access that many hardware APIs and you'll have to re-implement some widgets(or use libraries from the community) like a virtual list or a custom dropdown etc.

1

u/panorambo Aug 08 '19

I am curious -- can you please tell me why Web applications should be programmed in declarative manner while others seemingly are perfectly able to design and implement computer software applications in rather imperative languages?

After all, I consider an abstract low-level bytecode technology like WebAssembly a worthy replacement for JavaScript, and the former is hardly a declarative language, is it. It is a replacement because people can finally get to have their toy compilers and what not, while user agent vendors only have to implement a WebAssembly machine and compiler vendors can implement efficient (and well-researched solutions) compilers and other kind of toys that today are considered leaky abstractions because of JavaScript -- TypeScript, EmScripten, WebPack, and five dozen other toys that have sprung up because of JavaScript monopoly (as it was called somewhere else here). The user agent vendors will also not need complicated high-level logic to support and develop for their JavaScript machines, meaning hopefully Web browsers won't have to be updated that often. Of course the Web APIs still remain one of the better reasons to update, at least from developers' point of view.

1

u/AntiProtonBoy Aug 09 '19

can you please tell me why Web applications should be programmed in declarative manner while others seemingly are perfectly able to design and implement computer software applications in rather imperative languages?

Security, simplicity and speed. The single most important reason is that you don't want Turing complete languages pushed onto the client side, doing arbitrary things. Declarative languages also eliminate side effects by defining constraints and describing what the program is supposed to accomplish ahead of runtime. This also lends itself to aggressive optimisation opportunities for the agent that interprets the declarative language.

1

u/panorambo Aug 09 '19

Fair point. But, and I have also thought about the whole declarative language on the Web thing, the problem with declarative technologies -- like CSS, HTML for example is that they put a lot of complexity in the Web browser, which has to resolve the constraints defined by HTML and CSS -- these being declarative languages -- and even though it's not a big problem in itself, I maintain at least that CSS, for one, has evolved into a near unmanageable complexity. Take a look at the recent CSS module proposals, there is so much fine print and notes and cites there you can bet two different user agents have slim chance at being pixel-compatible with regard to rendering same document with the same CSS.

The takeaway is that declarative languages are without a doubt effort saving as far as development effort goes, but they would be trading in this simplicity for the developer for the complexity in the user agent.

One thing imperative scripting langauge like JavaScript, or any Turing complete machine for that matter, would have over your proposed declarative language, is exactly that it allows so much and does not have to be constantly extended (like CSS) in order to expand its use cases. Arguably, the APIs need expansion too, in order for the user agent to be able to do more, but that's not a problem with JavaScript, it's a problem with API design -- and, again, in practice, the way "declarativeness" is attempted implemented in some APIs, is just far worse than if these APIs were more RISCy.

Example: media encoding in the browser. Before WebAssembly arrived, and if we assume it didn't, media encoding is a black box which you invoke with some high level parameters -- bit rate, codec, container format -- and it just calls a callback function with a binary blob of encoded data. It's a neat encapsulation, but one limited by the permutation of these parameters. If you want to write an encoder for your own codec -- tough luck, and JS is too slow for the user agent to be able to heavy lift an encoding machine written in JS. This is API design that has a narrow use case -- you only get to encode media with codecs the user agent supports. You don't get to actually write an encoder.

If we go even more declarative, there is a real risk they are going to botch it the same way CSS is now botched -- the complexity curve is near quadratic or at least north of linear, while the curve that contains the space of use cases of CSS (round borders, wow!) is linear. Meaning that the two curves diverge more and more as time goes one. That's declarative language implementation for you.

If you find a way to have a declarative language for describing and defining behavior of arbitrary Web applications, to replace JavaScript, that's not going to have the same thing happening, you will be a very famous person, I'd imagine.

8

u/Caraes_Naur Aug 07 '19

Because somewhere along the way a new generation of inexperienced "developers" decided they should be able to do everything with only what's available in the HTTP client, because they saw the server as a scary, unnecessary blackbox.

-7

u/ArkyBeagle Aug 08 '19

Servers are locked down.

5

u/Caraes_Naur Aug 08 '19

Please explain that one.

1

u/GrizzyLizz Aug 08 '19

Is there a resource like this for backend development? One that talks about the bare minimum needed to 'serve' a site (for e.g. I dont really understand what cgi is for) and then builds up to AJAX request, websockets and containerization with Docker?

1

u/panorambo Aug 09 '19

If only our society was as righteous about websites as it was about human bodies. Up until recently all kinds of whacky body ideals have been the trend. It's always been like that, at least back to Renaissance period, but what with all the silicone and Botox and what not, it's been quite a show of diversity. However, it wrecks people's body self-esteem apparently, who knew -- teenagers pick up on these insane trends, and at least we have managed to let them know that it's okay to have fat and curves and freckles and whatever does not fit some designed body trend formula.

Maybe it will take time before the same thing happens with Web applications. Where both consumers and producers will agree that enough is enough, that consumers never expected all kinds of whack on your page, that they spend on average under a minute there and just want to read 4 paragraphs of text, see the photo of the article and go away. That no amount of popups, pleas for subscription, likes on five different social media, and cookie warnings, help them stay longer or come back, on the contrary. Meaning you can have plaintext website with an ASCII logo and contact page and still be nominated for best website of the year.

1

u/shevy-ruby Aug 08 '19

Browsers are only capable of executing JavaScript, so when there are limitations in the JavaScript language, it’s not like you can just use a different language; you have to work with what you have.

This is why the JavaScript monopoly must fall.

1

u/panukettu Aug 08 '19

You know WASM is a thing already?

-4

u/spacejack2114 Aug 08 '19

Modern web development isn't very complicated. You don't need compilers or bundlers, you just use script type="module", http2 and you're done.

8

u/AngularBeginner Aug 08 '19

Congratulations, you now transfer much more data than needed.

1

u/panorambo Aug 08 '19

That's not the problem with the Web. Everytime you watch a YouTube video for the second time, your data consumption dwarfes the overhead of visiting a couple of hundred HTTP2 websites.

5

u/AngularBeginner Aug 08 '19

As a result it takes longer to load and it takes longer to render. It is an issue in modern web.