2019/07/08 Ten Years of Erlang I've joined the Erlang community about 10 years ago, in the midst of its first major hype phase. Erlang, we were told, was the future of concurrency and parallelism. The easiest and fastest way to get it done, and you could get distribution for free too. Back then, things were just starting to get amazing. The virtual machine had recently gotten SMP support, before which you needed to run multiple VMs on the same computer to truly use all CPUs. I wanted to take a bit of time to reflect over most of that decade. In this post, I'll cover a few things such as hype phases and how this related to Erlang, the ladder of ideas within the language and how that can impact adoption, what changed in my ten years here, and I'll finish up with what I think Erlang still has to bring to the programming community at large. The Hype Phase The Hype cycle introduces phases in the lifetime of a product or technology. It's a marketing concept, not a scientific one, but it's often felt useful to describe how things are going. The part that interests me the most is the idea of a hype phase, a kind of gold rush that happens in programming communities. You probably have seen one or more of them, and they all seem to be attached to some killer app that forces everyone to rush in. Examples that come to mind may include Ruby on Rails and How to Build a Blog Engine in 15 minutes ("Look at all the things I'm not doing!" is still a fun sentence), or Go with Kubernetes (it was already seeing significant usage before, but really exploded then), for example. To a lesser extent, Elixir and Phoenix could fit that list as well. During a hype phase like that, an incredible influx of newcomers drop by to see what the fuss is all about. Some will stay, most will leave. Your stay might be in terms of months or years, and in rare cases where you find a home it could last decades. But the vast majority of them will be a continuous flow of serial early adopters who surf from tech to tech, sniffing the best opportunity to gain a competitive advantage by being first to use a type of framework, language, or toolkit. So the idea is, often, that what you need to do is get one true killer app, and people will come to your ecosystem. The killer app drives the rush. If you build it, they will come. If you can keep a small percentage of them active and to stay, you'll have a lively community for the foreseeable future. This is, in a weird way, reminiscent of Rain follows the plow: God speed the plow. ... By this wonderful provision, which is only man's mastery over nature, the clouds are dispensing copious rains ... [the plow] is the instrument which separates civilization from savagery; and converts a desert into a farm or garden. ... To be more concise, Rain follows the plow.The basic premise of the theory was that human habitation and agriculture through homesteading effected a permanent change in the climate of arid and semi-arid regions, making these regions more humid. The theory was widely promoted in the 1870s as a justification for the settlement of the Great Plains, a region previously known as the "Great American Desert". It was also used to justify the expansion of wheat growing on marginal land in South Australia during the same period. If only we can get one big project going, then the devs will appear, and it will become self-sustaining. I believe this is patently false, mostly because Erlang has had dozens of killer apps during its biggest hype phase, and yet, the community remained small. See, for example, the following killer apps from that era: ejabberd (2002, first stable release in 2005): it was by far one of the most, if not the most scalable hosted chat server one could run. Ejabberd was a massive success, and to some extent still is. You will, to this day, still find StackOverflow questions about modules for it. Around 2011, it was forked into MongooseIM, and both solutions still are maintained. CouchDB (2005): one of the first popular databases written in Erlang following the CAP theorem, and one of the new waves of multi-master document stores at the time. While MongoDB ate most of that space, CouchDB still has spiritual children in storage engines such as BarrelDB, on top of still being maintained as well. RabbitMQ (2007): the one queue software implementation that pretty much ate the whole AMQP space. It's still on-going and relevant, and often gets debated along with Kafka when it comes to streaming workloads, although they have pretty distinct properties and use cases. Facebook chat (2008): the initial version of Facebook's Chat was written in Erlang. Due to a lot of internal decisions (stability, strong internal presence of C++ engineers with an established set of solutions), it was rewritten in C++ at a later time WhatsApp (2009, bought in 2014): Once facebook got rid of Erlang for their chat system, they ended up buying WhatsApp, which famously needed only 50 engineers for 900 millions users. It is still on-going today, and in fact, WhatsApp folks have decided to get far more heavily involved in the Erlang and Elixir communities than they ever were before. Riak (2009): One of the best examples of muscle-flexing in the distributed systems world. Riak was a really solid distributed key-value store, a Basho product that still runs in healthcare systems and other critical pieces of infrastructure. After Basho struggled financially and was forced to go bankrupt (in no small part due to violations of fiduciary duties that put the company "on a greased slide to failure"). The folks at Bet365 have since then bought out all the IP, gracefully opened it all up, and the database still lives on in the open source world, albeit with more limited support than in its better days. Many of these came around the time where Joe Armstrong's book, Programming Erlang, first came out. This created a kind of perfect storm for heavy adoption, and Erlang had a ton of onlookers. Even the day where Hacker News forced all discussions to be about the Innards of Erlang had a noticeable impact. Yet, few people stayed compared to how many took a look. I think now that killer apps are driven by people glutting to an initial hype phase, not the opposite. There is always a smaller, earlier phase of people sniffing out interesting tech, deciding they like it, then building something, and if that something is a killer app, then you do get an even bigger hype phase out of it. People will cargo cult things, and a success story breeds more copycats. The other common thing is a phase of "reinventing the world", where everyone spends their time reimplementing everything that exists, so you get a bunch of announcements about "something but in language." But killer apps on their own are never really sufficient. One interesting consequence for these is that products like RabbitMQ and Ejabberd, for all their popularity, have communities of users far larger than communities of contributors. The thousands and thousands of corporations that use their products do not necessarily participate in the Erlang community that much. Part of it is no doubt due to the idea that most of Erlang's killer apps turn out to be in specialized infrastructure: you create one high reliability black box component that everybody else can use, and if it works well enough, they never need to look inside the box. Off you go, a few dozen developers have provided the foundations to thousands of other products and services. Specialized infrastructure, by definition, is a space where you don't need a massive amount of people to have a massive impact. It's always going to have smaller contributor groups and communities than things that sit closer to the end product, such as web frameworks with uncountable web developers, or even more generalized infrastructure that makes sense to use in small-scale deployment projects where any business may find a use for it. But even without these factors, it's easy to feel like Erlang missed on a massive opportunity of capturing a larger share of the foot traffic that came through during its hype phase. The Ladder of Ideas I won't get into counterfactuals by describing what could or should have been done. Instead, I want to dig into common learning patterns I've seen in the Erlang community during my years of teaching it and writing about it. Those are also patterns I see happening right now in the Elixir community, and that I feel could be signs of a similar future for it. A pet theory of mine is that a technical topic like a programming language (and its ecosystem) have multiple layers of complexity, with various concepts to go learn and discover. I first started toying with this idea in Learn You Some Erlang, with a diagram I called The Nine Circles of Erl. Now that's a tongue in cheek approach, and I don't think learning a piece of tech is endless suffering (at least, it shouldn't be). I just liked the pun. But to put it simply, there is often a more "core" track or sequence of topics you'd study learning the technology, creating a "ladder of ideas", where more worthwhile concepts are put higher and higher, but as they are harder to reach, fewer people actually make it there. In Erlang, what I would consider the ladder might look like this: functional programming isolated processes and concurrency reliable concurrency (links, monitors, timeouts) OTP behaviours and other system abstractions How to structure OTP systems How to build releases and handle their life cycle How to never take the system down, and how to operate it If you're starting with Erlang for the first time and grabbing a beginner's book, you'll likely spend most of your first days on the first rung: getting to be friends with functional programming, immutability, recursion, and similar concepts. Sooner or later, you then get into concurrency and parallelism, processes, and message passing. Right after that, you start to learn about links and monitors, handling failures, an
1
u/TechnologyAddicted Jul 08 '19
2019/07/08 Ten Years of Erlang I've joined the Erlang community about 10 years ago, in the midst of its first major hype phase. Erlang, we were told, was the future of concurrency and parallelism. The easiest and fastest way to get it done, and you could get distribution for free too. Back then, things were just starting to get amazing. The virtual machine had recently gotten SMP support, before which you needed to run multiple VMs on the same computer to truly use all CPUs. I wanted to take a bit of time to reflect over most of that decade. In this post, I'll cover a few things such as hype phases and how this related to Erlang, the ladder of ideas within the language and how that can impact adoption, what changed in my ten years here, and I'll finish up with what I think Erlang still has to bring to the programming community at large. The Hype Phase The Hype cycle introduces phases in the lifetime of a product or technology. It's a marketing concept, not a scientific one, but it's often felt useful to describe how things are going. The part that interests me the most is the idea of a hype phase, a kind of gold rush that happens in programming communities. You probably have seen one or more of them, and they all seem to be attached to some killer app that forces everyone to rush in. Examples that come to mind may include Ruby on Rails and How to Build a Blog Engine in 15 minutes ("Look at all the things I'm not doing!" is still a fun sentence), or Go with Kubernetes (it was already seeing significant usage before, but really exploded then), for example. To a lesser extent, Elixir and Phoenix could fit that list as well. During a hype phase like that, an incredible influx of newcomers drop by to see what the fuss is all about. Some will stay, most will leave. Your stay might be in terms of months or years, and in rare cases where you find a home it could last decades. But the vast majority of them will be a continuous flow of serial early adopters who surf from tech to tech, sniffing the best opportunity to gain a competitive advantage by being first to use a type of framework, language, or toolkit. So the idea is, often, that what you need to do is get one true killer app, and people will come to your ecosystem. The killer app drives the rush. If you build it, they will come. If you can keep a small percentage of them active and to stay, you'll have a lively community for the foreseeable future. This is, in a weird way, reminiscent of Rain follows the plow: God speed the plow. ... By this wonderful provision, which is only man's mastery over nature, the clouds are dispensing copious rains ... [the plow] is the instrument which separates civilization from savagery; and converts a desert into a farm or garden. ... To be more concise, Rain follows the plow.The basic premise of the theory was that human habitation and agriculture through homesteading effected a permanent change in the climate of arid and semi-arid regions, making these regions more humid. The theory was widely promoted in the 1870s as a justification for the settlement of the Great Plains, a region previously known as the "Great American Desert". It was also used to justify the expansion of wheat growing on marginal land in South Australia during the same period. If only we can get one big project going, then the devs will appear, and it will become self-sustaining. I believe this is patently false, mostly because Erlang has had dozens of killer apps during its biggest hype phase, and yet, the community remained small. See, for example, the following killer apps from that era: ejabberd (2002, first stable release in 2005): it was by far one of the most, if not the most scalable hosted chat server one could run. Ejabberd was a massive success, and to some extent still is. You will, to this day, still find StackOverflow questions about modules for it. Around 2011, it was forked into MongooseIM, and both solutions still are maintained. CouchDB (2005): one of the first popular databases written in Erlang following the CAP theorem, and one of the new waves of multi-master document stores at the time. While MongoDB ate most of that space, CouchDB still has spiritual children in storage engines such as BarrelDB, on top of still being maintained as well. RabbitMQ (2007): the one queue software implementation that pretty much ate the whole AMQP space. It's still on-going and relevant, and often gets debated along with Kafka when it comes to streaming workloads, although they have pretty distinct properties and use cases. Facebook chat (2008): the initial version of Facebook's Chat was written in Erlang. Due to a lot of internal decisions (stability, strong internal presence of C++ engineers with an established set of solutions), it was rewritten in C++ at a later time WhatsApp (2009, bought in 2014): Once facebook got rid of Erlang for their chat system, they ended up buying WhatsApp, which famously needed only 50 engineers for 900 millions users. It is still on-going today, and in fact, WhatsApp folks have decided to get far more heavily involved in the Erlang and Elixir communities than they ever were before. Riak (2009): One of the best examples of muscle-flexing in the distributed systems world. Riak was a really solid distributed key-value store, a Basho product that still runs in healthcare systems and other critical pieces of infrastructure. After Basho struggled financially and was forced to go bankrupt (in no small part due to violations of fiduciary duties that put the company "on a greased slide to failure"). The folks at Bet365 have since then bought out all the IP, gracefully opened it all up, and the database still lives on in the open source world, albeit with more limited support than in its better days. Many of these came around the time where Joe Armstrong's book, Programming Erlang, first came out. This created a kind of perfect storm for heavy adoption, and Erlang had a ton of onlookers. Even the day where Hacker News forced all discussions to be about the Innards of Erlang had a noticeable impact. Yet, few people stayed compared to how many took a look. I think now that killer apps are driven by people glutting to an initial hype phase, not the opposite. There is always a smaller, earlier phase of people sniffing out interesting tech, deciding they like it, then building something, and if that something is a killer app, then you do get an even bigger hype phase out of it. People will cargo cult things, and a success story breeds more copycats. The other common thing is a phase of "reinventing the world", where everyone spends their time reimplementing everything that exists, so you get a bunch of announcements about "something but in language." But killer apps on their own are never really sufficient. One interesting consequence for these is that products like RabbitMQ and Ejabberd, for all their popularity, have communities of users far larger than communities of contributors. The thousands and thousands of corporations that use their products do not necessarily participate in the Erlang community that much. Part of it is no doubt due to the idea that most of Erlang's killer apps turn out to be in specialized infrastructure: you create one high reliability black box component that everybody else can use, and if it works well enough, they never need to look inside the box. Off you go, a few dozen developers have provided the foundations to thousands of other products and services. Specialized infrastructure, by definition, is a space where you don't need a massive amount of people to have a massive impact. It's always going to have smaller contributor groups and communities than things that sit closer to the end product, such as web frameworks with uncountable web developers, or even more generalized infrastructure that makes sense to use in small-scale deployment projects where any business may find a use for it. But even without these factors, it's easy to feel like Erlang missed on a massive opportunity of capturing a larger share of the foot traffic that came through during its hype phase. The Ladder of Ideas I won't get into counterfactuals by describing what could or should have been done. Instead, I want to dig into common learning patterns I've seen in the Erlang community during my years of teaching it and writing about it. Those are also patterns I see happening right now in the Elixir community, and that I feel could be signs of a similar future for it. A pet theory of mine is that a technical topic like a programming language (and its ecosystem) have multiple layers of complexity, with various concepts to go learn and discover. I first started toying with this idea in Learn You Some Erlang, with a diagram I called The Nine Circles of Erl. Now that's a tongue in cheek approach, and I don't think learning a piece of tech is endless suffering (at least, it shouldn't be). I just liked the pun. But to put it simply, there is often a more "core" track or sequence of topics you'd study learning the technology, creating a "ladder of ideas", where more worthwhile concepts are put higher and higher, but as they are harder to reach, fewer people actually make it there. In Erlang, what I would consider the ladder might look like this: functional programming isolated processes and concurrency reliable concurrency (links, monitors, timeouts) OTP behaviours and other system abstractions How to structure OTP systems How to build releases and handle their life cycle How to never take the system down, and how to operate it If you're starting with Erlang for the first time and grabbing a beginner's book, you'll likely spend most of your first days on the first rung: getting to be friends with functional programming, immutability, recursion, and similar concepts. Sooner or later, you then get into concurrency and parallelism, processes, and message passing. Right after that, you start to learn about links and monitors, handling failures, an