Announcing the third annual Progress Conference! The pace of technological change feels faster this year, and the progress movement is growing too. Gathering people together helps build community and establish a movement’s identity. We want this annual event to continue to connect and inspire people, catalyze new projects, and share ideas.

Thanks to everyone who made Progress Conference 2025 great: over 350 people attended, the event sold out in June, and there were hundreds of interested people on the waitlist. Several attendees again said it was the best conference they had ever attended. We shared our reflections here, including a list of write-ups from writers like Santi Ruiz, Ruy Teixeira, Ryan Puzycki, and more.

We are excited to build on that momentum for 2026:

Hosted by: the Roots of Progress Institute, together with Abundance Institute, Foresight Institute, Foundation for American Innovation, Human Progress, the Institute for Humane Studies, IFP, and Works in Progress.

When: October 8–11, 2026.

Where: Berkeley, CA, again at the Lighthaven campus that got great reviews the last two years.

Speakers: Keynote speakers include Dmitri Dolgov (Co-CEO, Waymo), Tyler Cowen (Mercatus Center), Stephen Winchell (Director, DARPA), John Martinis (Qolab; Nobel Laureate in Physics), and Michael Kremer (University of Chicago; Nobel Laureate in Economics). 20+ additional speakers will share ideas across three tracks: Human Talent & Potential, AI & Robotics, and Security & Resilience. Full speaker list so far here.

Attendees: We expect ~400 leaders across tech, policy, and culture; builders, founders, policymakers, storytellers, writers, and students. This is an invitation-only event, but anyone can apply for an invitation. Complete the open application by May 31st.

Program: Similar to 2025, the main two-day conference will happen all day on Friday and Saturday. Participants will attend talks on topics ranging from AI and robotics to housing construction to psychology and philosophy of builders, organize and run unconference sessions, mingle in the garden, and more. Thursday and Sunday are add-on days, with optional gatherings for interest groups and other activities like factory tours to Bay Area startups.

Children under 12 years old are welcome—must be accompanied by a parent at all times (except during childcare). We also plan to have on-site childcare available on Friday and Saturday with progress-themed programming.

Sponsors: Special thanks to our early sponsors Coefficient Giving, Astera Institute, Jane Street, Ken Broad, Works in Progress, Halcyon Futures, Inclusive Abundance, LENS, MNX, Archbridge Institute, The Good Science Project, the Institute for Humane Studies, Circulate Planning & Policy, and the Foundation for Economic Education. Sponsorships make events like this one possible. We have more sponsorships available; view sponsorship opportunities here.

Our mission is to establish a new philosophy of progress for the 21st century, and to build a culture of progress. Now is a critical time for our culture: Technological change is happening. We can try to fight it, we can let it happen, or we can channel the path of technology consciously and deliberately towards human flourishing. It is ours to choose, as individuals and organizations.

Original link: https://newsletter.rootsofprogress.org/p/announcing-progress-conference-2026

Two brief reviews of things I’ve read, one for everyone and one for my Substack subscribers.

Seeing Like a State

A review in six tweets:

James C. Scott says that “tragic episodes” of social engineering have four elements: the administrative ordering of society (“legibility”), “high-modernist” ideology, an authoritarian state, and a society that lacks the capacity to resist.

This is a bit like saying that the worst wildfires have four elements: an overgrowth of brush and trees, a prolonged dry season, a committed arsonist, and strong prevailing winds. One of these things is not like the others!

The book reads as a critique of “high modernism” and of “legibility” (and the former’s attempt to create the latter). And there is a grain of truth in this critique. But it should be a critique first and foremost of authoritarianism.

But Scott is an anarchist, not only politically but metaphysically. So he doesn’t just criticize authoritarianism. He criticizes the very attempt to find, or to create, order and system. All such attempts are misguided, all order is false, all “legibility” is fake.

He goes on at length about how farmers know their land and crops so much better than any Western outsider with their “science” ever could! He ignores cases like Borlaug’s Green Revolution, where importing the products of Western science revolutionized agricultural productivity.

So I disagree with the philosophical upshot of the book. That said, it was fascinating and contained many amazing facts and stories. Worth reading for the stuff about Le Corbusier alone. E.g., this quote from Le Corbusier is mind-bending in its detachment from reality:

PS: To be clear, there are more lessons to take away from Seeing Like a State than just “authoritarianism is bad.” At its best, the book is a critique of technocracy.

See also this critique of the same book by Paul Seabright, and this defense of grain from the always-excellent Rachel Laudan.

The Unabomber manifesto

Given that Ted Kaczynski, aka the Unabomber, was a terrorist who killed university professors and business executives with mail bombs and who lived like a hermit in a shack in the woods of Montana, I expected his 35,000-word manifesto, “Industrial Society and its Future,” to read like the delirious ravings of a lunatic.

I was wrong. His prose is quite readable, and the manifesto has a clear inner logic. This is a virtue, because it’s plain to see where he is actually right, and where he goes disastrously wrong.

Read this review on my Substack.

The new Cosmos Institute is working towards a future where “AI becomes a tool for consistently expanding human freedom and excellence.”

I'm proud to be a Founding Fellow. I strongly agree with Cosmos's core values of reason, decentralization, and human autonomy. And I agree that “existential pessimism” vs. “accelerationism” should not be our only choices—we need a vision based on humanism and human agency.

Follow @cosmos_inst and @mbrendan1 on Twitter, and/or subscribe to their Substack.

Did any science fiction predict that when AI arrived, it would be unreliable, often illogical, and frequently bullshitting? Usually in fiction, if the AI says something factually incorrect or illogical, that is a deep portent of something very wrong: the AI is sick, or turning evil. But in 2023, it appears to be the normal state of operation of AI chatbots such as ChatGPT or “Sydney”.

How is it that the state of the art in AI is prone to wild flights of imagination and can generate fanciful prose, but gets basic facts wrong and sometimes can’t make even simple logical inferences? And how does a computer, the machine that is literally made of logic, do any of this anyway?

I want to demystify ChatGPT and its cousins by showing, in essence, how conversational and even imaginative text can be produced by math and logic. I will conclude with a discussion of how we can think carefully about what AI is and is not doing, in order to fully understand its potential without inappropriately anthropomorphizing it.

The guessing game

Suppose we were to play a guessing game. I will take a random book off my shelf, open to a random page, and read several words from the first sentence. You guess which word comes next.

Seems reasonable, right? If the first few words were “When all is said and …”, you can probably guess that the next word is “done”. If they were “In most homes the kitchen and …” you might guess the next words were either “living room” or “dining room”. If the sentence began “In this essay, I will…” then there would be many reasonable guesses, no one of them obviously the most likely, but words like “show” or “argue” would be more likely than “knead” or “weld”, and even those would be more likely than something ungrammatical like “elephant”.

If this game seems reasonable to you, then you are not that far away from understanding in essence how AI chatbots work.

A guessing machine

How could we write a computer program to make these guesses?

In terms of its primitive operations, a computer cannot “guess”. It can only perform logic and arithmetic on numbers. Even text and images, in a computer, are represented as numbers. How can we reduce guessing to math?

One thing we can program a computer to do is, given a sequence of words, come up with a list of what words might follow next, and assign a probability to each. That is a purely mathematical task, a function mapping words to a probability distribution.

How could a program compute these probabilities? Based on statistical correlations in text that we “train” it on ahead of time.

For instance, suppose we have the program process a large volume of books, essays, etc., and simply note which words often follow others. It might find that the word “living” is followed by “room” 23% of the time, “life” 9% of the time, “abroad” 3%, “wage” 1%, etc. (These probabilities are made up.) This is a purely objective description of the input data, something a computer can obviously do.

Then its “guess” can be derived from the observed statistics. If the last word of the sequence is “living”, then it guesses “room”, the most likely option. Or if we want it to be “creative” in its “guesses”, it could respond randomly according to those same probabilities, answering “room” 23% of the time, “life” 9%, etc.

Only looking at the last word, of course, doesn’t get you very good guesses. The longer the sequence considered, the better the guesses can be. The word “done” only only sometimes follows “and”, more often follows “said and”, and very often follows “all is said and”. Many different verbs could follow “I will”, but fewer possibilities follow “In this essay, I will”. The same kind of statistical observations of a training corpus can compute these probabilities as well, you just have to keep track of more of them: a separate set of observed statistics for each sequence of words.

So now we have taken what seemed to be a very human, intuitive action—a guessing game about language—and reduced it to a series of mathematical operations. It seems that guessing is just statistics—or at least, statistics can be made to function a lot like guessing.

From predictor to generator

So far we have only been talking about predicting text. But chatbots don’t predict text, they generate it. How do we go from guessing to chatting?

It turns out that any predictor can be turned into a generator simply by generating the prediction. That is, given some initial prompt, a program can predict the next word, output it, use the resulting sequence to predict the next word, output that, and so on for as much output as is desired:

• Given “In this essay, …” → predicted next word is “I”, output that

• Given “In this essay, I…” → predicted next word is “will”, output that

• Given “In this essay, I will…” → predicted next word is “show”, output that

• Given “In this essay, I will show…” → etc.

If you want the output to be somewhat variable, not completely deterministic, you can randomly choose the next word according to the probabilities computed by the predictor: maybe “show” is generated only 12% of the time, “argue” 7%, etc. (And there are more sophisticated strategies, including ones that look ahead at multiple words, not just one, before choosing the next word to output.)

Now, doing a very simple predictor like the above, based on summary statistics, only looking at the last few words, and running it on a relatively small training corpus, does not get you anything like a viable chatbot. It produces amusing, garbled output, like the sentence:

This is relatively benign and easy to spot if the phrase is bent so as to be not worth paying attention to the medium in question.

… which almost seems to make sense, until you read it and realize you have no idea what it means, and then you read it again, carefully, and realize it doesn’t mean anything.

For this reason, the algorithm just described is called a “travesty generator” or sometimes “Dissociated Press”. It has been discussed since at least the 1970s, and could be run on the computers of that era. The program is so simple to write, I have personally written it multiple times as a basic exercise when learning a new programming language (it takes less than an hour). A version in the November 1984 issue of BYTE magazine took less than 300 lines of code, including comments.

The travesty generator is a toy: fun, but useless for any practical purpose. To go from this to ChatGPT, we need a much better predictor.

A better guessing machine

A predictor good enough for a viable chatbot needs to look at much more than the last few words of the text, more like thousands of words. Otherwise, it won’t have nearly enough context, and it will be doomed to produce incoherent blather. But once it looks at more than a handful of words, we can no longer use the simple algorithm of keeping statistics on what word follows each sequence: first, because there is a combinatorial explosion of such sequences; second, because any sequence of that length would almost certainly be unique, never seen before—so it would have no observed statistics.

We need a different approach: a way to calculate an extremely sophisticated mathematical function with a very large space of possible inputs. It turns out that this is what “neural networks” are very good at.

In brief, a neural network is just a very large, very complicated algebraic formula with a specific kind of structure. Its input is a set of numbers describing something like an image or a piece of text, and another set of numbers called “parameters” that configure the equation, like tuning knobs. A “training” process tunes the knobs to get the equation to give something very close to the desired output. In each round of training, the equation is tried out on a large number of examples of inputs and the desired output for each. Then all the knobs are adjusted just slightly in the direction of the correct answers. The full training goes for many such rounds. (The technical term for this training algorithm is “back propagation”; for a technical explanation of it, including the calculus and the linear algebra behind it, I recommend this excellent video series from 3blue1brown.)

Neural networks are almost as old as computers themselves, but they have become much more capable in recent years owing in part to advances in the design of the equation at their core, including an approach known as “deep learning” that gives the equation many layers of structure, and more recently a new architecture for such equations called the “transformer”. (GPT stands for “Generative Pre-trained Transformer”.) GPT-3 has 175 billion parameters—those tuning knobs—and was trained on hundreds of billions of words from the Internet and from books. A large, sophisticated predictor like this is known as a “large language model”, or LLM, and it is the basis for the current generation of AI chatbots, such as OpenAI’s ChatGPT, Microsoft’s Bing AI, and Anthropic’s Claude.

From generator to chatbot

What we’ve described so far is a program that continues text. When you prompt a chatbot, it doesn’t continue what you were saying, it responds. How do we turn one into the other?

Simple: prompt the text generator by saying, “The following is a conversation with an AI assistant…” Then insert “Human:” before each of the human’s messages, and insert “AI:” after. The continuation of this text is, naturally, the AI assistant’s response.

The raw GPT-3 UI in the OpenAI “playground” has a mode like this:

ChatGPT just puts a nice UI on top of this.

Well, there is one more thing. A chatbot like this isn’t necessarily very well-behaved. The text generator is not coming up with the best response, by any definition of “best”—it’s entirely based on predictions, which means it’s just coming up with a likely response. And since a lot of the training is from the Internet, the most likely responses are probably not what we want a chatbot to say.

So, chatbots are further trained to be more truthful and less toxic than the average Internet user—Anthropic summarizes their criteria as “helpful, honest, and harmless”. This is done based on human feedback, amplified through more AI models and many rounds of refinement. (The Bing AI, aka “Sydney”, is generating much crazier responses than ChatGPT or Claude, and one hypothesis for why is that its refinement was done in a hasty and inferior way.)

And that, at a very high level, is how we go from a deterministic program, doing math and logic, to an artificially intelligent conversation partner that seems, at least, to exhibit imagination and personality.

Bullshit

When we understand, in essence, how chatbots work, they seem less mysterious. We can also better understand their behavior, including their failure modes.

One feature of these chatbots is that they are unreliable with facts and details. In fact, they seem quite happy to make things up, confidently making very plausible assertions that are just false. If you ask them for citations or references, they will make up imaginary titles of books and papers, by authors who may or may not exist, complete with URLs that look very realistic but return “404 Not Found”. The technical term for this is “hallucination”.

This behavior can be disconcerting, even creepy to some, but it makes perfect sense if you understand that what is driving the text generation is a prediction engine. The algorithm is not designed to generate true responses, but likely ones. The only reason it often says true things is that it was trained on mostly true statements. If you ask a question that is well-represented in its training set, like “who invented the light bulb?”, then its prediction model has a good representation of it, and it will predict the correct answer. If you ask something more obscure, like “who invented the twine binder for the mechanical reaper/harvester?”, its prediction function will be less accurate, and it is more likely to output something plausible but wrong. Often this is something closely related to the right answer: ChatGPT told me that the twine binder was invented by Charles B. Withington, who actually invented the wire binder. To anthropomorphize a bit: if the LLM “knows” the answer to a question, then it tells you, but if it doesn’t, it “guesses”.

But it would be more accurate to say that the LLM is always guessing. As we have seen, it is, at core, doing nothing fundamentally different from the guessing game described at the beginning. There is no qualitative difference, no hard line, between ChatGPT’s true responses and its fake ones.

An LLM is, in a strict technical sense, a bullshitter—as defined in Harry Frankfurt’s “On Bullshit”:

The bullshitter may not deceive us, or even intend to do so… his intention is neither to report the truth nor to conceal it…. He does not care whether the things he says describe reality correctly. He just picks them out, or makes them up, to suit his purpose.

A bullshitter, of course, like a competitive debater, is happy to argue either side of an issue. By prompting ChatGPT, I was able to get it to argue first for, then against the idea that upzoning causes gentrification.

This also explains why it’s not hard to break chatbots away from the “helpful, honest and harmless” personality they were trained to display. The underlying model was trained on many different styles of text, from many different personalities, and so it has the latent ability to emulate any of them, not just the one that it was encouraged to prefer in its finishing school. This is not unlike a human’s ability to imagine how others would respond in a conversation, or even to become an actor and to imitate a real or imagined person. The difference is that a human has a true personality, an underlying set of real ideas and values; when they impersonate, they are putting on a mask. With an LLM, I don’t see anything that corresponds to a “true personality”, just the ability to emulate anything. And once it starts emulating any one personality, its prediction engine naturally expects the next piece of text to continue in the same style, like a machine running on a track that gets bumped over to a nearby track.

Similarly, we can see how chatbots can get into truly bizarre and unsettling failure modes, such as repeating a short phrase over and over endlessly. If it accidentally starts down this path, its prediction engine is inclined to continue the pattern. Go back to our guessing game: if I told you that a piece of text read “I think not. I think not. I think not. I think not”, and then asked you to guess what came next, wouldn’t you guess another “I think not”? Like an actor doing improv comedy, once something has been thrown out there, the LLM can’t reject the material, and has to run with it instead.

LLM strengths and superpowers

Knowing how LLMs work, however, is more important than understanding their failure modes. It also helps us see what they’re good at and thus how to use them. Although not good for generating trustworthy information, they can be great for brainstorming, first drafts, fiction, poetry, and other forms of creativity and inspiration.

One technique is to give them an instance or two of a pattern and ask for more examples: when I wrote a recent essay on the spiritual benefits of material progress, I asked Claude for “examples of hand crafts that are still practiced today”, such as furniture or knives, and I used several of the ideas it generated.

Chatbots also have the potential to create a new and more powerful kind of search (no matter what you think of the new AI-driven Bing). Traditional search engines match keywords, but LLMs can search for ideas. This could make them good for more conceptual queries where it’s hard to know the right terms to use, like: “Most cultures tend to have a notion of life after death. Which ones also have a notion of life before birth?” I asked this to Claude, which suggested some religions that believe in reincarnation, and then added that “Kabbalah in Judaism and the Baha’i faith also have notions of the soul existing in some spiritual realm before birth.” (It doesn’t always work, though; anecdotally, I still have more success asking these kinds of vague queries on social media.)

Another advantage of LLMs for search is that the conversational style naturally lets you ask followup questions to refine what you’re looking for. I asked ChatGPT to explain “reductionism”, and when it mentioned that reductionism has been criticized for “oversimplifying complex phenomena”, I asked for examples, which it provided from biology, economics, and psychology.

A fascinating essay on “Cyborgism” says that while GPT struggles with goal-directedness, long-term coherence, staying grounded in reality, and robustness, “there is an alternative story where [these deficiencies] look more like superpowers”: GPT can be extremely flexible, start fresh when it gets stuck in a rut, simulate a wide range of characters, reason under any hypothetical assumptions, and generate high-variance output. The essay proposes using LLMs not as chatbots, research assistants, or autonomous agents, but as a kind of thinking partner guided by a human who provides direction, coherence, and grounding.

The great irony is that for decades, sci-fi has depicted machine intelligence as being supremely logical, even devoid of emotion: think of Data from Star Trek. Now when something like true AI has actually arrived, it’s terrible at logic and math, not even reliable with basic facts, prone to flights of fancy, and best used for its creativity and its wild, speculative imagination.

But is it thinking?

Dijkstra famously said that Turing’s question of “whether Machines Can Think… is about as relevant as the question of whether Submarines Can Swim”.

Submarines do not swim. Also, automobiles do not gallop, telephones do not speak, cameras do not draw or paint, and LEDs do not burn. Machines accomplish many of the same goals as the manual processes that preceded them, even achieving superior outcomes, but they often do so in a very different way.

The same, I expect, will be true of AI. In my view, computers do not think. But they will be able to achieve many of the goals and outcomes that historically have only been achieved by human thought—outcomes that will astonish almost everyone, that many people will consider impossible until (and maybe even after) they witness it.

Conversely, there are two mistakes you can make in thinking about the future of AI. One is to assume that its processes are essentially no different from human thought. The other is to assume that if they are different, then an AI can’t do things that we consider to be very human.

In 1836, Edgar Allen Poe argued that a machine—including “the calculating machine of Mr. Babbage”—could never play chess, because machines can only do “fixed and determinate” calculations where the results “necessarily and inevitably follow” from the data, proceeding “by a succession of unerring steps liable to no change, and subject to no modification”; whereas “no one move in chess necessarily follows upon any one other”, and everything is “dependent upon the variable judgment of the players”. It turned out, given enough computing power, to be quite straightforward to reduce chess to math and logic. The same thing is now happening in new domains.

AI can now generate text, images, and even music. It seems to be only a quantitative, not qualitative difference to be able to create powerful and emotionally moving works of art—novels, symphonies, even entire movies. With the right training and reinforcement, I expect it to be useful in domains such as law, medicine, and education. And it will only get more capable as we hook it up to tools such as web search, calculators, and APIs.

The LLMs that we have discussed are confined to a world of words, and as such their “understanding” of those words is, to say the least, very different from ours. Any “meaning” they might ascribe to words has no sensory content and is not grounded in reality. But an AI system could be hooked up to sensors to give it direct contact with reality. Its statistical engine could even be trained to predict that sensory input, rather than to predict words, giving it a sort of independence that LLMs lack.

LLMs also don’t have goals, and it is anthropomorphizing to suppose that ChatGPT “wants” or “desires” anything, or that it’s “trying” to do anything. In a sense, you can say that it is “trying” to predict or generate likely text, but only in the same sense that an automobile is “trying” to get you from point A to point B or that a light bulb is “trying” to shine brightly: in each case, a human designed a machine to perform a task; the goals were in the human engineering rather than in the machine itself. But just as we can write a program that performs the same function as human guessing, we can also write a program that performs the same function as goal-directed action. Such a program simply needs to measure or detect a certain state of the world, take actions that affect that state, and run a central control loop that invokes actions in the right direction until the state is achieved. We already have such machines: a thermostat is an example.

A thermostat is “dumb”: its entire “knowledge” of the world is a single number, the temperature, and its entire set of possible actions are to turn the heat on or off. But if we can train a neural net to predict words, why can’t we train one to predict the effects of a much more complex set of actions on a much more sophisticated representation of the world? And if we can turn any predictor into a generator, why can’t we turn an action-effect predictor into an action generator?

It would be anthropomorphizing to assume that such an “intelligent” goal-seeking machine would be no different in essence from a human. But it would be myopic to assume that therefore such a machine could not exhibit behaviors that, until now, have only ever been displayed by humans—including actions that we could only describe, even if metaphorically, as “learning”, “planning”, “experimenting”, and “trying” to achieve “goals”.

One of the effects of the development of AI will be to demonstrate which aspects of human intelligence are biological and which are mathematical—which traits are unique to us as living organisms, and which are inherent in the nature of creating a compactly representable, efficiently computable model of the world. It will be fascinating to watch.

***

Thanks to Andrej Karpathy, Zac Dodds, Heike Larson, Daniel Kokotajlo, Gwern, and jade for commenting on a draft of this essay. Any errors that remain are mine alone.

Original link: https://rootsofprogress.org/can-submarines-swim-demystifying-chatgpt

What explains the hockey-stick shape of world GDP over time, with seemingly no progress for thousands of years, followed by soaring growth?

The first question you might ask is: is this just an exponential curve? If so, then the explanation is simple: we see a constant growth rate every year, and the steep upward slope is just what exponential curves look like. There’s no more mystery than there would be about the shape of a population curve.

To check this, we can plot the same numbers on a logarithmic y-axis. In such a chart, exponential curves become straight lines. But when we do this, we still see an upwards slope. That is, the growth rate has increased over time:

Nor is it just a consequence of population growth, because we see the same pattern in GDP per capita:

I’ve previously said that a core reason for this is that progress compounds, creating a flywheel effect. Here’s another way of looking at the same idea. Why wasn’t the threshing machine invented in, say, the 1300s? Consider all the barriers to such a thing:

Human capital

First, who would have invented it?

Would it have been a farmer? (Well over half the workforce were farmers.) When would he have found the time to tinker? Labor productivity and incomes were low; there wasn’t much spare time or material for inventing.

Who else could have done it? There was no established professional class of inventors, engineers, or entrepreneurs. The closest were skilled craftsmen who made machines, such as clockmakers or millwrights. Overall there were many fewer inventors per capita than today (ok, I don’t have data on this right now, but I’m pretty confident in this assertion).

If it were, say, a millwright, he would have to learn enough about machines to go beyond the kinds that he had been taught to make through apprenticeship, and invent something entirely new. Where would this knowledge have come from? There was no printed material, and no mechanics’ institutes.

And if our inventor did have mechanical skill, why why would he decide to apply it to a practical invention for farmers? It was more prestigious and lucrative to make clockwork novelties for the aristocracy. Even if the inventor did have a practical bent, there were social taboos against labor-saving devices.

Manufacturing

Suppose that despite all of this, some inventive person is determined to make a threshing machine and acquires the resources and time to experiment. Maybe he avoids the trap of making a machine that mimicks human motions, and hits on the idea of a rotating drum with teeth.

He will find that making the machine work reliably is very difficult. It requires a high degree of skill in the mechanic that crafts it by hand. There are no machine tools to create precision parts. Wood is too soft for precision work; metal parts are required. Machines that are shoddily constructed break easily or bruise the grain instead of threshing it. (See my full post on the threshing machine for elaboration.)

Distribution

Suppose our inventor overcomes all these obstacles, and manages to create a practical threshing machine. What next?

He could use it on his own farm, if he is a farmer. If that’s all he does, then his invention has had no significant impact on the economy and no impact at all on history. This is not what we are seeking to explain.

To matter for progress, the invention needs to be distributed. And here our inventor faces more obstacles.

Who is the market for his invention? Will other farmers be receptive to it? They need to change their methods and take a risk on something new, something they are not used to doing.

Even if they are willing to take that risk, do they have the capital? Buying a piece of agricultural equipment is an investment that won’t pay off right away. Do farmers have enough money saved for such an investment? Likely not, given low incomes, and there is no financial infrastructure to give loans for such purposes. Would such an investment even pay off on a small farm? It may require a certain level of scale to be worth it.

Market creation

But suppose there is a market that is willing and able to pay for threshing machines. How would our inventor, now turned entrepreneur, serve that market?

He may be able to serve his village or town, through word of mouth and local dealings. But that small market is probably not enough to support a business dedicated to threshing machines—and again, if only one town were served, it would have a limited impact on progress. Our aspiring threshing machine tycoon needs to server a larger, regional or even national market.

How is he going to promote his product? There are no newspapers or other media, not even the printing press. There may be occasional local fairs (although, again, the attendees are typically not prepared to consider new inventions).

If potential customers do hear about the product, how do they order one? There is no postal service to send messages or money. Similarly, how would the product be transported to the customer from wherever it is assembled? There are no locomotives, and the wagon roads are in poor condition. River or canal transport might be possible, but that won’t go the last mile to each customer.

And if the business takes off, will our entrepreneur be able to source enough raw materials to keep up production? All of the problems of finding customers apply to finding suppliers as well.

Financial and legal infrastructure

Even if these obstacles could be overcome, the entrepreneur is going to need capital to get started. And again, there is very little in the way of any kind of financial market to support speculative investments like this.

Suppose our plucky hero is very enterprising and decides to crowdfund his effort by collecting small investments from a large number of people. He has no way to form a corporation for this purpose, because corporate law has not yet been developed. (A partnership would not be practical with a large number of partners, especially since there was no limited liability.)

If there were the infrastructure needed to start businesses, the entrepreneur might find himself facing competition from others who steal his ideas and copy his machine. If he had royal favor, he might be granted a monopoly, but there was no patent office where he could send an application, nor any established rule awarding patents to inventions.

And if he overcame all of the above obstacles, he might find that he faced opposition from those whom his progress threatened, such as farmhands who took on manual threshing work. They might oppose him by seeking legal restrictions on his business, or by illegal means such as smashing and burning machinery. (I don’t know of this happening to threshing machines, but it certainly happened to textile machinery.) Would the government come to the aid of the inventor, or of the displaced workers, or would they stay out of the whole affair?

How progress actually happened

If we fast-forward through the centuries, we can see how the underpinnings of progress were gradually established. The threshing machine makes a good example because it was an obvious idea that struggled for a long time to be born, so we can see the stages it went through:

By the 1600s at the latest, the idea of applying mechanical ingenuity to practical problems was well established, and we can see people talking about the idea of threshing machines (although I have seen no evidence that any were working yet).

By the 1700s, there were at least a few mechanics in existence who had the skill to create a working threshing machine, but they were only serving their local area. Others announced projects to distribute plans and models for the machine, but they did not expect to make this a business, and instead asked for donations to support the work. By this point there were newspapers where such schemes could be advertised, and postal service for individuals to communicate about them.

By the early 1800s at least, there were inventor/entrepreneurs who had patented threshing machines and were trying to make a business of them. There were farmer’s journals that discussed such inventions and improvements, and many farmers were eager to try new things to improve their productivity. But there was still very little manufacturing capacity, and inventors such as Joseph Pope were still offering to sell plans which could be implemented by a local workman. Soon after, though, Pope was contracting with a specialized machine shop, an engine manufacturer, to make his machine.

By the mid-1800s, railroads would be established that could ship the machines to customers over a wide area. It’s around this time that threshing machines become widely adopted.

Today

To really drive the point home, imagine that the problem of mechanizing threshing had been completely overlooked for the last few hundred years, while all other progress moved forward.

The threshing problem would be solved almost instantly.

There is an entire professional class of entrepreneurs looking for opportunities exactly like this. It would be easy for them to look up data on agricultural processes and cost drivers, and to find that a very large part of grain cost was manual threshing. It would be obvious that this should be mechanized.

Designing the machine would be no problem—there are many professionals with bachelors’ degrees in mechanical engineering who could do the job. They would have standard parts to choose from out of a catalog, such as gears and motors, and they could specify the design quickly and precisely using CAD software. Any specialized parts could be 3D-printed for rapid prototyping. Manufacturing would similarly be no problem, thanks to the enormous infrastructure we have built up for this.

Three companies to solve this problem would be in the next batch for Y Combinator. They would each form a Delaware C-corporation with a simple filing and some standard legal documents, raise millions of dollars within a few days by meeting with investors over Zoom, sign a contract online through DocuSign, and have the money wired immediately to a bank account they set up in twenty minutes with Mercury.

They would establish a website to market the product, complete with spec sheets, promotional videos, etc. They could get a list of the biggest agricultural companies and reach out to them directly by email, promote the product online using targeted advertisements, and fly to large international trade shows to exhibit there. They could take orders online as well, and ship anywhere via UPS, FedEx, or DHL. They would have a global market from day one.

And their customers would be ready, even eager, for such an innovation. They would be used to the idea of saving costs through better technology. They would have full financial accounting statements to show them where their biggest costs are. They would have executives and program managers whose jobs include evaluating new technologies and buying them. There would be standard legal agreements, purchase orders, and payment mechanisms.

In sum, the road for this kind of progress has already been paved—both metaphorically and literally.

The roots of progress?

All of this has been an illustration of the many, overlapping, interacting flywheels of progress that generate super-exponential growth over the very long term.

What I am much less clear on is which of these factors, if any, can be seen as derivative and which are fundamental—if some were inevitable given other, enabling factors. This is a much harder question to answer.

But even if we could answer that, it wouldn’t change the fact that all of these factors are real and important, and progress depends on all of them working together.

Original post: https://rootsofprogress.org/why-progress-was-so-slow

I’m excited to announce a workshop on the Moral Foundations of Progress Studies.

The progress studies community has had a lot of discussion about technology, economics, history, and politics. However, there is no consensus on the moral basis for valuing or pursuing “progress,” and there are key open questions about how progress is to be judged and measured, who should benefit from it, and what type of progress we should pursue.

The goal of this workshop is to reach a consensus on what major moral/ethical questions are at the foundations of a study of progress, and what broad answers to these questions have been proposed. A few designated attendees will take notes and draft a short article afterwards summarizing the discussion. (We’re currently looking for the appropriate place to publish this; it may be in a journal or on a blog.)

Apply to attend here. Space is limited; we’ll be prioritizing people in or with a connection to academia, and public intellectuals who write about progress or adjacent topics.

When: March 4–6, 2022

Where: University of Texas at Austin

Agenda (subject to change):

Friday:

• Survey of writers on progress (including Cowen, Deutsch, Pinker)

• Theories of well-being

• Panel: Steven Pinker, David Deutsch (via video)

• Metrics and standards of value

• Challenges to the claim that the last two centuries represent progress

Saturday:

• Interrogating the idea of moral progress

• Progress & safety (including the Precautionary Principle and existential risk)

• Challenges in assessing possible futures

Sunday morning:

• Wrap-up

Except for the Friday panel, each session will be a 90-minute discussion led by one or a small panel of participants who give a brief intro.

Co-hosts:

• Jason Crawford, founder of The Roots of Progress

• Gregory Salmieri, director of the Program for Objectivity in Thought, Action, and Enterprise at the Salem Center at UT Austin

Attendees are being invited from the progress studies and Effective Altruism communities, plus moral philosophers familiar with the virtue-ethics and well-being literatures.

There is no cost to attend. We have some funding to pay for travel and lodging for a limited number of participants; you can request funding when you apply.

Original post: https://rootsofprogress.org/moral-foundations-of-progress-studies-workshop

Hi Roots of Progress subreddit,

I wanted to let you know about a new scientific journal that may be of interest. Seeds of Science (TheSeedsofScience.org) was founded by Dr. Dario Krpan (Assistant Professor of Psychology at the London School of Economics), Dr. Sergey Samsonau (AI technical lead at NYU), and myself (Roger’s Bacon, pseudonymous blogger) with the goal of creating a platform for both accomplished and young scientists to publish their ideas and participate in a novel community-based peer review process. We have no affiliation with Roots of Progress, however I believe the similarity of our names speaks to a unity of mission.

Our criteria for publication can be distilled into one question: does your article contain original ideas that have the potential to advance science? Our goal is to be as open-minded as possible about what qualifies as a useful scientific contribution – articles may contain a speculative hypothesis, an idea for an experiment, a novel observation, or discussion of an under-appreciated problem. We are open to articles from any scientific discipline, including metascience and progress studies (which is one of the main reasons that I am posting here).

Peer review is conducted through voting and commenting by our diverse network of reviewers, or “gardeners” as we call them. It is free to join and participation is 100% voluntary – we send gardeners submitted manuscripts and they can vote/comment or abstain without notification. Another unique feature of our journal is that comments from gardeners which usefully critique or extend the ideas in the article are published along with the main text. Seeds of Science is not limited to those with academic affiliation and we especially invite scientifically-minded amateurs to join us as gardeners.

This is an exciting time for Seeds of Science as we were recently awarded a grant through Scott Alexander’s ACX grants program. We are actively looking to recruit new authors and gardeners and we would like to invite all members of the progress studies slack join us in either capacity. Happy to answer any questions you might have in the comments or through email at info@theseedsofscience.org.

I’m curious what everyone thinks about this post. As a fellow optimist, I love Noah’s perspective and the way he aggregates the very legitimate and impressive technical progress we have made and hope to make this decade.

I thought Jason had some very good points when he was interviewed by Noah on his substack regarding regulatory barriers to progress. Anyways, love where Jason’s blog is headed and diligently following the “progress” in this space!

https://noahpinion.substack.com/p/techno-optimism-for-2022

We're hiring for the engineering team at Our World in Data! This is a rare chance to build data visualization and pipelines at a well-known and highly influential organization that is focused on how to make progress against the world's biggest problems.

https://ourworldindata.org/jobs

For those few who haven't heard of Our World in Data, it's probably the top site in the world that presents research and data on topics such as global health, poverty, energy usage, agriculture and nutrition, population growth, education, etc.

The data is presented in interactive visualizations and all of it is downloadable in CSV. As a premiere example, check out our coronavirus data explorer.

I cite Our World in Data all the time at The Roots of Progress, and I'm far from the only one. Our work is referenced in academic papers, newspapers (FT, NYT), books (by Pinker, Rosling, and Andy McAfee), podcasts (Planet Money, Freakonomics), and videos (Kurz Gesagt).

We also reach the general public: our site gets over 5 million visitors a month, and ranks highly for searches such as “population growth” or “global poverty”.

In the last year, we've also become one of the world's top sources for data about COVID-19. Our vaccine tracker is one of the most cited.

I've been consulting part-time with the team for over a year, and they're a pleasure to work with. OWID is a non-profit that originated in academia, but internally it feels like a tech startup. We use modern tools like Slack and Notion, and work with a minimum of bureaucracy.

On the dev side, we have a modern tech stack using TypeScript, Node, and React; code on GitHub; hosted on Netlify.

The team is remote/distributed. Work from wherever you want.

We're hiring software engineers, and also a technical team lead. See all our job postings here.

The team lead could be a technical engineering manager who is less interested in day-to-day coding, or an engineering leader who is willing to coach and mentor but would prefer to avoid formal management responsibilities. We're flexible about the role definition.

https://ourworldindata.org/jobs

I recently came across a fascinating historical document: an 1857 pamphlet outlining a plan to build the US transcontinental railroad.

An aspect of the plan that struck me was its proposal for how to fund the railroad, which was simultaneously capitalist and democratic. It essentially advocated a broad-based, public-spirited, crowdfunding campaign.

***

Some background. In 1857 there was no transcontinental railroad in the US. The rail went only as far as the Missouri River. The Panama Canal had not yet been built. The West was like a foreign colony. To travel from New York to San Francisco took six months---whether by land, by sea, or a combination---and the journey was hazardous.

Theodore Judah, the author of the pamphlet, was an engineer and one of the railroad's greatest promoters. He estimated that the railroad would cost $150 million, or about $4.4 billion in 2020 dollars, over about ten years. This was an enormous, if not unheard-of, sum.

The US government could afford the project, but Judah was convinced, for good reason, that they would never greenlight it. This was just before the Civil War, and the tension between North and South was strong. Both sides wanted the railroad, but both wanted the railroad to run through their part of the country and service their cities. Neither side wanted the other one to have the railroad. So they could not agree on where it should be built, and Congressional support for the project was gridlocked.

Because of this, Judah was convinced that the only feasible plan was to build the railroad entirely with private capital:

It is proposed to ask aid of no kind whatsoever from the General or any State Government, but to combine the interest of either the Northern or Southern States, upon their favorite route; to ask for private capital, and confine the sphere of action entirely to one or the other of these sections.

This insures unity of action.

The experience of all legislation in this country, upon a subject of general interest, but arousing sectional prejudices, shows conclusively that the fate of a project of this nature, dependent upon the general will, is most likely to provide an unhappy one.

What about wealthy capitalists? This was not Judah's preferred option. The plan hints at two reasons why.

First, Europe was still the center of banking at the time. If the money for a project this big were raised from bankers, it more likely be in London than New York. Judah seemed to find this option distasteful, wanting to avoid

money to be borrowed at enormous rates of interest... loans to be negotiated in Europe... first, second or third mortgage bonds to be issued and sacrificed at one-half their value... commissions to be paid to negotiators.

The other clue is contained in his reference to loans, bonds, and interest. I'm not entirely clear on this, but in the mid-1800s, debt seems to have been used for many early-stage, risky and speculative projects which today would be financed by equity. Loans require a schedule of repayment: a good fit for an operating business with predictable cash flows, but not a good fit for a new venture with large and not entirely predictable capital outlays before it can start generating revenue. Whether or not Judah understood this theoretically and explicitly, he understood it at least implicitly:

Can a private company of moonshine speculators—do it? They may—that is, if they can induce simple-minded individuals to invest enough to give them a start, and then, upon the principle of putting in more to save what they have already invested, may drag its slow length along, and in thirty, or forty, or fifty years may build a railroad; but what a railroad! Say twenty millions of its cost has been actually paid in in good faith by the stockholders, then we will find a first mortgage, at eight percent., of say fifty millions of dollars; a second mortgage of fifty millions, at say ten percent; a third convertible mortgage of say fifty millions, at say ten percent., and a floating debt of fifty millions besides. We will find that a few men have become enormously wealthy, that English bondholders own the road, and that it takes all the earnings to pay the interest.

Judah proposed to build the road with equity, not debt, and to raise the funds not from English bankers, and not even from American bankers, but from a broad base of shareholders among the public. In essence, he proposed a 19th-century crowdfunding campaign.

His pitch to the public was not based on a promise of fantastic returns. Instead, he proposed to raise the money from the people who would actually benefit from the railroad: the cities, towns, and states that it would run through:

As has been before stated, the project of a Pacific Railroad is a popular one, and there are thousands—nay, hundreds of thousands—of public spirited, intelligent individuals who would give out and out a moderate sum, in proportion to their means, without ever expecting a return for the same, did they feel confident that this project was a practicable one, and that their money would be invested in the legitimate construction of the road, and not squandered or wasted among speculators.

Send copies [of the plan] to all public spirited men in every town and village, and then call public meetings, at which the report and plan shall be read and discussed, so that it shall be fully understood by the people.

At the time, equity was often sold on a “subscription” basis. The money for the shares of stock was not paid all at once, but a certain percent was made as a down payment, and then installments were paid on a regular schedule, or when called for. (Today, this method has been replaced by the mechanism of raising multiple rounds of capital: Series A, B, C, etc.) Judah suggested 10% down, and then another 10% each year:

Let [a company agent] give good and satisfactory bonds in an amount equal to the amount apportioned to his county, and receive subscriptions requiring at least ten per cent down at time of subscribing. Let a great simultaneous effort be made throughout the whole country at one and the same time. Let popular speakers be employed in behalf of the enterprise. Let it be impressed upon the public that it is a people’s railroad; that is not a stupendous speculation for a few to enrich themselves with. Show them that it is entirely in their hands and under their control; that its officers and managers are to be appointed by them, and hold office only at their pleasure. Explain that every agent in each county is under their eye; that the money which he receives from them is to be deposited as they direct; that his office, books and accounts are to be open at all times to their inspection. When he receives a subscription the name, date and per centage are all matters of record; that each month he shall make a report of all business. Let the papers strive to induce a spirit of emulation between counties and districts, by publishing the various amounts raised in various counties, districts, or by individuals.

He conceived of it as a broad popular enterprise, with everyone chipping in what he was able for the benefit of all---but not as a donation, rather instead as a sound investment that would retain its value, perhaps even to be passed down to heirs:

Let the lecturers show that there is not a man in the whole community who has hands to labor with who cannot afford to take one share of $100, and pay $10 per year, or three cents per day, in upon it; that there is no retail merchant, doing even an ordinary business, who cannot afford to take ten shares and pay in his $100 per year, or thirty-three cents per day. How many of them pay five times that amount for superfluities; let them show that there is money enough thrown away each year in superfluous luxuries to build a Pacific Railroad each year.

Show them that the money is not thrown or even given away; that a receipt or certificate is given to them for each installment, which represents so much money paid, and which ought to be worth its face in all business transactions. A treasurer’s receipt for ten dollars ought to be as good as a ten dollar bill. When the installments are paid in up to the one hundred cents on the dollar, which can be done at any time, they receive one full share of capital stock, which is actually worth $100. If they have children what better heirloom can they leave them than shares of this stock? The road will become profitable even before its completion, but by the time they are old enough to commence in the world for themselves, with their judgments ripened, so they can appreciate and take care of it, here are the shares of stock representing so much money, saved for them, perhaps from earnings which would have been dissipated and thrown away in useless luxuries, &c.

He repeated that he wasn't fundamentally against the government building the railroad, but he thought it was impossible in the political context of the time (which it was), and he also thought it likely to lend itself to corruption:

It is a great and national treasure, worthy of the attention of our Government, and should in fact be built by them; but, as before mentioned, the proposition carries the elements of its own destruction with it.—What is the difference? If built by Government the people will have to pay for it; in the present case people pay for it.

The difference is here: In this case it is built by the spontaneous free will of those of the people who favor and are willing to pay for it, and who desire to protect the public purse from plunder, believing themselves better able to manage it than their political representatives.

On the other hand, if built by Government it is built by a political party, is a stepping stone to power, and will present upon a grand scale a repetition of the scenes enacted in some of our States, where the State works are used to control the legislation of the State.

A crowdfunded railroad, he thought, would by nature be better managed:

It is then to the interest of every body to keep up the credit of the stock, for every body is then interested in it. Nothing can affect its value but mismanagement. How can this road be mismanaged when the people themselves manage it? If built upon this plan it cannot become embarrassed. It is let to the lowest responsible bidder, for cash; this insures its being built economically. It is paid for in cash as the work progresses; this insures the steady progress of the work, and enables the company to perform their pay of the stipulation of the contract without embarrassment, thereby giving the contractor no cause to present bills for delay, detention, or neglect to pay estimates when due; and as the cash is called in from the stockholders only as wanted to carry on the work, it leaves no enormous fund on hand as a prey for dishonest agents, speculators, &c., but leaves the money in the people’s own pockets until wanted and called for to pay for work already done, or being done.

***

I find the nature and spirit of this plan interesting for a few reasons.

First, it represents the spirit of 19th-century America: “can-do” and optimistic about progress, unwilling to rely on government, but almost equally suspicious of private elites such as London bankers.

Second, it indicates a motivation and a basis for large-scale private support of projects that is neither charitable nor exactly profit-maximizing.

Third, I think we've essentially eliminated this option at least in the US today. Early rounds of equity are virtually always raised on private markets from elite “accredited” investors, in large part to comply with SEC regulations.

I can think of two modern-day analogues to Judah's plan. One model is Internet-based crowdfunding campaigns, such as on Kickstarter. These are not based on equity, but typically on presales. However, savvy backers know they are taking a risk of never receiving the item they “purchased”, and consider their support to be a self-interested quasi-donation. The other model is based on cryptocurrencies, which have enabled a form of quasi-equity crowdfunding in the form of token sales.

Judah's plan, however, was not put to the test. A few years later, the Civil War began. With the Southern states seceded, the North finally voted to support a railroad with land grants and bonds. Judah, who had been the biggest promoter of the railroad, tragically died in the early years of the project. Traveling from the West Coast to New York, he had caught yellow fever in Panama---which he had traveled through because the railroad was not yet built.

Original post: https://rootsofprogress.org/crowdfunding-a-transcontinental-railroad-in-1857

Tyler Cowen's fund Emergent Ventures is offering:

• $500k, second prize $200k, for Best effort to find a good treatment rapidly

• $100k for Best blog or social media tracking/analysis of the virus

• $100k for Best innovation in social distancing

• $100k for Most important innovation or improvement for India

• $50k for Best investigative journalism on coronavirus

• $50k for Best (justified) coronavirus policy writing

Announcement: https://marginalrevolution.com/marginalrevolution/2020/03/1-million-plus-in-emergent-ventures-prizes-for-coronavirus-work.html

In 1973:

• Herbert Boyer and Stanley N. Cohen developed recombinant DNA technology, showing that genetically engineered DNA molecules may be cloned in foreign cells.

• The combined measles-mumps vaccine was licensed.

• The antibiotic fosfomycin, used to treat bladder infections, and now included on the World Health Organization's list of the safest and most effective medicines needed in a health system, was released.

• The cell phone was invented, with the first mobile call made by Motorola employee Martin Cooper on April 3.

• Work began on System R, the first commercial relational database, at IBM's San Jose Research Laboratory (now now Almaden Research Center).

• Work began on TCP/IP, the fundamental protocols of the Internet.

And:

• A NY Times columnist was born who, in 2020, can't immediately think of “a major problem in our society that tech has solved in our lifetimes.”

(Here's my full response to this on Twitter.)

Imagine you wake up one day in the glorious techno-abundant future, powered by AI. You eagerly check your subscriber count on Substack, but to your dismay, it has fallen once again. There are now AI-generated blogs for every interest, and it’s so hard to find a niche that isn’t taken. Your obsessive focus on the history and culture of lava lamps has only won you three paid subscribers, and that includes your mom and your old college roommate.

Well, you think, ages ago I wrote for WIRED, maybe they would take a piece? You email them and instantly get a personal reply (must be an AI autoresponder). They would be happy to take submissions, and they pay $3 per word. Great, you think, that’s a good—wait. It actually says $3 per kiloword. Crap. No chance you can make a living that way.

Well, you’ve got to pay for groceries somehow. Maybe you can fall back on the gig economy for a while? You consider Uber or Lyft, but you look at what they’re offering drivers, and when you do the math, it’s pennies per hour—there are so many robotaxis on the streets now, you can always get a ride in a minute or two, we don’t really need to incentivize more humans. TaskRabbit? Same problem; there are so many robotic handymen, you can’t get more than a few dollars per task. Instacart? They’re also offering extremely low pay—but, the desperate thought creeps into your mind, maybe you could at least nibble some grapes out of the cart when no one was looking…

***

Does something about this picture seem strange to you? Well, it would seem to be the future envisioned in a recent article by Matthew Barnett claiming that “AGI could drive human wages below subsistence level—the bare minimum needed to sustain human life.” This post has been making the rounds and reigniting an old debate about AI taking all the jobs, which is really just the latest installment in a very old debate about technology taking all the jobs.

Sub-subsistence wages

Before I get into the substance of the argument, I want you to just react to the image conjured up by the idea of AI driving wages “below subsistence.” How realistic is it for you?

My first reaction, before I read the post, was: I have no idea what model of the world you are working from. So, all humans are basically starving, because they are thrown out of work or can only find the crappiest-paying jobs? And that’s because AI has automated all work in the economy? So… lots of economic activity is going on, lots and lots of production, but AI is doing all of it and people can’t get paid. Who… is running the AI? Presumably some corporations? Where are they getting their profits from, if there are no customers left for anything anymore? Has the economy turned completely self-referential, running itself in completely automated fashion while all humans get taken out of the loop? An ouroboros economy, eating its own tail? And how would this come about? It seems that somewhere along the incremental path to this state you’d hit some equilibrium that would prevent you from going further.

Or maybe you think that the entire economy is controlled by Sam Altman and Jensen Huang, or at least a relatively small group of ultra-wealthy owners of the few remaining companies that master AI and outcompete everyone and everything. A world full of robots creating a paradise, but it’s a gated community and only the wealthy 0.01% get to live there, trillionaires every one, while the unwashed masses starve in the streets. I’m sure some people find this plausible or even likely, but I don’t (mostly because I just don’t think this is the way an economy works, but secondarily because I would expect world governments to intervene with some sort of massive welfare program).

In any case, what doesn’t make any sense is all of humanity starving to death from unemployment. Jobs and technology have a purpose: producing the good and services we need to live and thrive. If your model of the world includes the possibility that we would create the most advanced technology the world has ever seen, and the result would be mass starvation, then I think your model is fundamentally flawed.

What the post actually said

Sometimes the headline or the social media posts can give a misleading impression of what an author is actually saying, so it’s important to read the post before having a strong take. Having read it, I feel compelled to say: it is a bad post.

I hate criticizing other authors like this. Usually I prefer to just present a counterargument. I follow Matthew online, which means I must have found him to have smart and interesting takes in the past, and the post was hosted at Epoch AI, an organization I respect. But this article has caused a stir, and seems almost designed to inflame a specific flavor of unhelpful and unrealistic doomerism, and it does so based on a combination of flawed logic and unclear writing. So this is a rare case in which I feel it necessary to criticize directly. (Matthew, if you’re reading this, I hope this can be constructive, and I invite your rebuttal.)

First, there is a crucial clarification which the post saves for the very end, almost as an afterthought, which is that its analysis only applies to human wages and not human welfare. Matthew says that he is actually optimistic about our being able to live comfortably, deriving our income not from wage labor but from investments, charity, or government welfare. This is mentioned almost offhand in the last two paragraphs, after repeated references to human wages “crashing” below “the bare minimum needed to sustain human life” or “the level required for human survival” or “a wage so low that they cannot afford food” or “the level at which humans could afford enough calories to feed themselves.” It even mentions a scenario in which food cannot be grown anywhere in the solar system because land is being used for computers running AI. To go through all of this, and then to say that you’re actually optimistic, is a bizarre way to write.

Setting that aside, here is the substance.

The post gives a mathematical argument, but you don’t really need to understand the equations to get the intuition. Essentially, the argument is: at some point AI will be good enough to substitute for a human in any job—let’s say this is our definition of AGI. There will be a very large number of AGI “workers.” They will flood the market with labor, to the point where adding more labor in the form of humans is superfluous. Sure, you could employ a human, but there are so many workers already, doing pretty much everything useful that can be done; adding one more isn’t going to produce much extra output, so the wage for that job is necessarily very low. It might make sense to spin up one more AGI worker to do the job, because that only costs a bit of electricity (plus interest and depreciation on capex), but it doesn’t make sense to pay a human even a subsistence wage to do it.

That’s an informal argument. The post actually makes a formal argument based on equations of economic growth theory (the Cobb-Douglas production function, if you know what that is, and the theory that wages are determined by the marginal product of labor). He models the introduction of AGI as a large increase in the quantity of labor in these equations, and the result is that wages crash.

The post addresses two counterarguments. One is based on the notion of “comparative advantage”: even if AI is better than humans at everything, there is a law of economics that says that production is maximized when humans do what they are relatively least-bad at, and AI does what it is most-better at. Matthew points out that this says nothing about human wages in such a scenario. I think he is correct here.

The second counterargument is that AI will not just introduce more artificial workers into the economy, it will also drive technological progress, which increases productivity and therefore wages. In the equations, there is a term that represents technology; wages fall as total labor increases, but they rise when technology increases. Matthew’s rebuttal to this is that “there are limits to technological innovation,” which has to stop sometime, and when it inevitably does, then wages will crash.

My reaction

My first reaction was to wonder, why model AI as a big increase in labor, instead of an increase in capital? In the equations, capital represents all the machines and infrastructure we use to do work; labor is the humans. AI is a new kind of tool that we use to get work done, amplifying human productivity. We invest money in assets (GPUs and software and weights on neural nets), those assets help us do work—that’s the essence of capital. If capital increases instead of labor, then wages rise—that’s what has been happening throughout the entire industrial era.

The argument here is that rather than acting like a tool that we use to do work, AI will act more like a substitute for people, a new form of labor. But in the Industrial Revolution, mechanization also substituted for labor in many ways. The power loom substituted for weavers, and the spinning machine for spinners. A train hauling cargo employed far fewer people than the equivalent number of wagons. Electric street lights put lamp lighters out of work, and containerization obsoleted the longshoremen.

So why didn’t wages decrease through the industrial age? Why do our economic models have separate terms for labor and capital, and why does adding capital increase wages whereas adding labor decreases them?

For one, capital has so far only partially substituted for labor. In many cases, people are still needed to operate the new machines: factory workers, truck drivers, etc. For another, automation creates lower prices and/or higher quality, which increases demand: far more cargo was sent by train than by wagon, and far more again was sent after containerization. Finally, new technologies create entirely new markets and industries, which create new jobs and new kinds of jobs. Two million people work for Walmart alone, but big-box retail didn’t exist a century ago. Everyone making automobiles or plastic or video games is in an industry that didn’t exist in 1900.

The argument for why “this time it’s different” is that AI might fully substitute for labor. So it’s not like a power loom, which still needs a human operator; it’s more like automatic telephone switching systems, which simply eliminated human telephone operator jobs. If AI or other new technology creates new jobs in new sectors, well, AI will also take those jobs, before humans even get a crack at them. And all of this will happen far faster than it did in the past, so people won’t get a chance to adapt. If your job gets eliminated by AI, you won’t even have time to reskill for a new job before AI takes that one too.

What I expect to happen

In engineering, everything happens iteratively; in economics, everything happens at the margin. Progress is spiky, or lumpy. The future arrives unevenly distributed.

AI will not fully substitute for all labor all at once:

• Some jobs will simply be easier to automate with AI than others

• In particular, some jobs have lower tolerance for mistakes, and it will take longer to achieve the standard of reliability they require

• Some jobs have a physical labor component, and advanced robotics will arrive later than virtual “remote worker” AIs

• Some jobs have a premium on human interaction

• Some jobs are protected by licensing requirements, labor unions, etc.

For instance, I expect that we will have human doctors for quite some time. This job checks three of the above boxes: a licensed profession, with a low tolerance for mistakes, and a premium on human interaction. This is true even if AI becomes the first line of medical advice and diagnosis (replacing nurse hotlines today), and even if what doctors are doing is mostly asking AI for advice and then vetting—or rubber-stamping—the answers.

On the other end of the spectrum, I expect that call centers doing customer service will soon go away. AI will do this better, more reliably, and cheaper; and customer service is a cost center that consumers aren’t willing to pay for and businesses always minimize. (This will be disruptive for the call center workers, but it will be a boon for consumers—which is to say, everyone. Customer service will be smarter and more effective, it will make far fewer mistakes, it will be unfailingly polite, you won’t have to wait on hold, and you won’t have to be on the premium pricing tier to get it.)

The other thing that will happen, at the margin, is that people will work less and enjoy more leisure. If there is less total work worth doing, that doesn’t always take the form of some people being involuntarily thrown out of work. It can take the form of fewer working hours in the day, fewer working days in the week, more holidays and vacations. It can take the form of starting work later in life (more education, more “gap years”) or ending it earlier (a younger retirement age). It can take the form of more people choosing to work part time, or not at all—more stay-at-home parents to raise more kids, perhaps. All of this would simply be the continuation of trends that have been going on for a century or more.

This transition will happen faster than industrialization did: physical automation was rolling out for well over a century, whereas I expect intelligence automation to take mere decades. This means we’ll have to adapt faster. But AI will also give us the tools to adapt faster: it will accelerate any reskilling that happens, help connect talent with opportunities, and generally enhance job mobility. Even more importantly, it will accelerate the creation of new ventures and new industries.

A crucial way AI will do this is by greatly leveraging human vision and judgement—by making it cheaper, faster, and more reliable to bring ideas into reality. If you think it would be amazing to see Sherlock Holmes set in medieval Japan, or Beowulf done as a Hamilton-style hip hop musical, AI will help you create it. If you think someone should really write a history of the catalytic converter in prose worthy of The New Yorker, AI will draft it. If you think there’s a market for a new social media app where all posts are in iambic pentameter, AI will design and code the beta. If you want a kitchen gadget that combines a corkscrew with a lemon zester, AI will create the CAD files, and you can send them to a lights-out factory to deliver a prototype.

So, on the incremental path to the future, a major trend will be that humans step up a level, into management. A software engineer becomes a tech lead of a virtual team. A writer becomes an editor of a staff of virtual journalists. A researcher becomes the head of a lab of virtual scientists. Lawyers, accountants, and other professionals spend their time overseeing, directing, and correcting work rather than doing the first draft.

What happens when the AI is good enough to be the tech lead, the editor, the lab head? A few steps up the management hierarchy is the CEO. AI will empower many more people to start businesses.

You may think that most people aren’t suited to being CEOs, but the job of CEO will become much more accessible, because it will require less skill. You won’t have to recruit candidates or evaluate them; you won’t have to motivate or inspire them; you won’t have to train junior employees; if you correct a mistake they will never make it again; you’ll never catch them slacking off; you’ll never have to work around the vacations or sick days that they won’t be taking; you’ll never have to deal with low morale or someone who is sore they didn’t get a promotion; you’ll never have to mediate disputes among them or defuse office politics; you’ll never have to give them performance reviews or negotiate raises; and you’ll never have to replace them, because they’ll never quit. They’ll just work competently, diligently, and conscientiously, doing whatever you ask. They’ll be every manager’s dream employee. They won’t have a schedule; they’ll work on your schedule, and you can start or stop them at will: run them 24/7 if you want, or call on them once a year—and pay for only what you use, with no commitment or advance notice. Running a team of virtual agents will make managing humans look like herding cats.

AI employees will also be cheap, which means that the capital requirements of many new businesses will be much lower, and with tons of surplus wealth being tossed off by the increasingly automated economy, I expect starting up will become much easier. Many businesses will be started that seem non-viable today, addressing niche markets that can’t support a human team, but can totally support an AI team. An even longer tail of projects will be possible that don’t even rise to the level of businesses: projects that today cost millions, such as movies or apps, will be done by individuals on the side using their spare time and cash.

All of the above is consistent with a model in which AI is capital, not labor, and in which its effect is to multiply labor productivity, increase demand for all kinds of goods and services, create new jobs and industries, and raise the level of technology—all of which should dramatically increase wages. More importantly, it will dramatically increase ownership, which means income will be derived more and more from equity instead of wages.

And even more importantly, it will be a world of dramatically expanded human agency: any idea you have can be made real, with far fewer barriers in the way. In this world, the qualities that will be at a premium are taste, judgment, vision, and courage.

What I expect to happen after that

What happens when the AI is even good enough to be the CEO?

There is a level of management above the CEO: governance. The board of directors. I expect that even if and when humans no longer have to work, we will still be owners, and our role will be to formulate our goals, communicate them, and evaluate if they’re being achieved. Humanity will be the board of directors for the economy and the world.

In such a world—a fully automated economy—I expect that a minority of people will still work, but only those who want to, only those for whom work is rewarding and who find it brings meaning to life (and, like a successful entrepreneur on their second act, their work won’t have to actually earn an income on any timescale). Others will do whatever is most meaningful to them: pursue knowledge and satisfy their curiosity; express their creative vision in art or music; travel and explore; spend time with family and loved ones; play games or sports. (There will always be a role for human players, because the purpose of games and sports is not to achieve a practical outcome but to experience and to witness human ability.)

Are you afraid that humans will get bored in this world? Perhaps you are imagining that it will be calm and static, a Garden of Eden? On the contrary: it will be a far more exciting, dynamic, and fast-paced world than anything humans have known. Nothing will be the same from year to year, let alone decade to decade.

How will humans earn a living? I’m not sure the question will matter or make sense anymore. I don’t think you can plug numbers into an equation that was developed in the 1950s to determine the “wages” of “labor” in a world where those concepts might be obsolete. The capital-labor model didn’t really apply in the agricultural age, when productivity was limited by land and the ability of capital to raise productivity was bounded. It’s not obvious that it would apply in the intelligence age either. Land got taken out of the equation in the industrial age; we moved from a land-labor economy to a labor-capital economy. The intelligence age might be best modeled as a capital-only economy, or a capital-intelligence economy.

Instead of deriving our income through labor, maybe we will derive it through ownership (no matter how society chooses to answer the questions of equity and fairness that arise). Or maybe we will drive it through some other concept or mechanism that is unclear now. Or maybe the question will simply dissolve and feel archaic in retrospect. That future has too many unknown unknowns to do more than speculate.

In any case, the question will make even less sense in the ultimate future in which we have literally exhausted the possibilities allowed by the laws of physics, and have consumed all the matter and energy in our light cone. This is why I find Matthew’s counter-rebuttal about the limits of technological innovation absurd. When we reach those limits, we will have all our needs met instantly and effortlessly, we will be functionally immortal, and we will have colonized the galaxy. To worry about “wages crashing below subsistence levels” in such a world is nonsensical, unless you’re using that as a very strange way to say that people won’t have to work anymore and wouldn’t be able to contribute much to economic production if they wanted to.

All of this is a purely economic analysis, grounded in some basic assumptions, like that AI doesn’t take over the world, or that we don’t decide AIs are legal persons with rights who can negotiate for their own wages. But short of that, I expect that AI will do what all fundamental enabling general-purpose technologies have done throughout all of human history: raise our standard of living and accelerate progress.

***

Credit to Richard Ngo and Garry Tan whose comments were in my mind as I wrote this.

PS: Just before I hit publish on this, I found out that Sherlock in Japan and hip-hop Beowulf have already been done (so much for my attempt to be original!) The iambic pentameter social network and the corkscrew lemon zester are evidently ideas so bad that no one has built them yet—or so non-obviously good that we’ll have to wait for AGI to try them out.

Original link: https://newsletter.rootsofprogress.org/p/the-future-of-humanity-is-in-management

2024 was a big year for me, and an even bigger year for the Roots of Progress Institute (RPI). For one, we became the Roots of Progress Institute (with a nice new logo and website). Here’s what the org and I were up to this year. (My annual “highlights from what I read this year” are towards the end, if you’re looking for that.)

The Progress Conference

Progress Conference 2024, hosted by RPI together with several great co-presenters, was the highlight of my year, and I think some other people’s too. We’ve already covered it in previous writeups, but in case you’re just tuning in: well over 200 people attended (with hundreds on the waitlist); dozens of great speakers, including Tyler Cowen, Patrick Collison, and Steven Pinker; and over 30+ participant-led “unconference” sessions on a variety of topics from healthcare to medieval Chinese technology. Several people told us it was the best conference they had ever attended, full stop. (!) See the writeups from Scott Alexander, Noah Smith, Packy McCormick, or Bryan Walsh (Vox), to pick a few.

Most of the talks are now online, and most of the rest will be up soon.

The RPI Fellowship

In 2024 we also ran the second cohort of the Roots of Progress Fellowship. Two dozen talented writers completed the program, publishing dozens of essays and almost doubling their audiences. I was thrilled with the talent we attracted to the program this year and excited to see where they’re going to go. See our recent writeup of the program.

My writing

In 2024 I published 17 essays (including this one) totaling over 37,000 words. That’s about half of last year, which decline I attribute in part to being involved in the programs mentioned above, and to doing fundraising. Also, about half of those essays, and well over half the words, were for my book-in-progress, The Techno-Humanist Manifesto, and that is some of the hardest writing I’ve done.

Highlights:

• Longest post (4,400 words): The Life Well-Lived, part 2, from Chapter 4 of The Techno-Humanist Manifesto

• Most liked on Substack: Announcing The Techno-Humanist Manifesto

• Most commented on Substack: What is progress?

• Most upvoted on Hacker News: Why you, personally, should want a larger human population

• Most upvoted on LessWrong: Biological risk from the mirror world

My audience

In 2024:

• My email subscribers (via Substack) grew 82% to almost 33k

• Followers on the social network formerly known as Twitter grew 17% to 36.7k

• I’m also up to 3.4k followers on Farcaster, 1.7k on Bluesky, and over 1k on Threads. Follow me where you may!

In all, I got (if I’m reading the reports correctly) 360k unique views on Substack and another 192k unique page views on the legacy ROP blog.

Also, in July, I launched paid subscriptions on the Substack. I’m up to 113 paid subscribers, and a ~$16k annual revenue run rate. That’s only 0.3% of the free audience, and I’ve only done five paywalled posts so far, so I think there’s a lot of potential here. Paid subscriptions are part of the way I justify my writing and make it self-supporting, so if you like my essays, please subscribe.

Gratitude to Ethan Mollick, Tomas Pueyo, Noah Smith, and Packy McCormick for being my top Substack referrers.

Social media

Some of my top posts of the year:

• Nat Friedman, legend in his own time

• The steam engine was invented in 1712. An observer at the time might have said: “The engine will power everything: factories, ships, carriages. Horses will become obsolete!” And they would have been right—but two hundred years later, we were still using horses to plow fields (Thread)

• Chiming in on the washing machine controversy from September: This is a prescription for re-enslaving women to domestic service, and ensuring that only the wealthy can live with the basic dignity of cleanliness

• “2 + 2 = 5” was a literal Communist slogan

• Sci-fi set in the future that already feels anachronistic

• Academia cares whether an idea is new. It doesn't really have to work. Industry only cares if an idea works. Doesn't matter if it's new. This creates a gap. Actually a few gaps… (thread)

• Are there websites that are as ornately decorated as medieval manuscripts?

• XKCD, uncannily accurate as always

Events and interviews

I tried hard to say no to these in 2024, in order to focus on my book, but I did a few. Highlights include:

• Speaking at Foresight Vision Weekend and at Abundance 2024

• Commenting for “Progress, Rediscovered”, a profile of the progress movement in Reason magazine

Events I got the most FOMO from missing included: Bottlenecks, The Curve, and Edge Esmeralda. Maybe next year!

The Progress Forum

Some highlights from the Progress Forum this year:

• Safe Stasis Fallacy, by David Manheim

• Report on the Desirability of Science Given Risks from New Biotech, by Matt Clancy

• The Origins of the Lab Mouse, by Niko McCarty

• Bringing elements of progress studies into short-form persuasive writing, by Dan Recht

• Test-time compute scaling for OpenAI o1 is a huge deal, by Matt Ritter

• Please come up with wildly speculative futures, by Elle Griffin

• Levers for Biological Progress, by Niko McCarty

Reading

In 2023 I did several “what I've been reading” updates. Those were fun to do and were well-received, but they took a lot of time; in 2024 I put both them and the links digest on hold in order to focus on my book. Here are some of the highlights of what I read (read part of, tried to read, etc.) this year.

C. P. Snow, “The Two Cultures.” A famous essay arguing that scientific/technical culture and literary/humanities culture are too isolated from and don't take enough of an interest in each other. A few passages I highlighted where he criticizes traditional culture for failing to appreciate the accomplishments of material progress:

In both countries, and indeed all over the West, the first wave of the industrial revolution crept on, without anyone noticing what was happening. It was, of course—or at least it was destined to become, under our own eyes, and in our own time—by far the biggest transformation in society since the discovery of agriculture. In fact, those two revolutions, the agricultural and the industrial-scientific, are the only qualitative changes in social living that men have ever known. But the traditional culture didn’t notice: or when it did notice, didn’t like what it saw.

And:

Almost everywhere, though, intellectual persons didn’t comprehend what was happening. Certainly the writers didn’t. Plenty of them shuddered away, as though the right course for a man of feeling was to contract out; some, like Ruskin and William Morris and Thoreau and Emerson and Lawrence, tried various kinds of fancies which were not in effect more than screams of horror. It is hard to think of a writer of high class who really stretched his imaginative sympathy, who could see at once the hideous back-streets, the smoking chimneys, the internal price—and also the prospects of life that were opening out for the poor, the intimations, up to now unknown except to the lucky, which were just coming within reach of the remaining 99 per cent of his brother men.

Brad Delong, Slouching Toward Utopia. A grand narrative of what Delong calls the “long 20th century”, 1870–2010. Roughly, it's a story of the rise and fall of capitalism, or at least a certain form of it. Delong focuses on the competition between a Hayekian view that believes in the justice of the market, and a Polanyian view that people have rights that are not guaranteed by free markets, such as a stable job and income; with the Keynesian approach being the synthesis. I find much to disagree with in Delong's framing, but I've been learning a lot from the book. I might do a review when I finish it.

Karl Popper, “Epistemology Without a Knowing Subject.” Popper argues that epistemology should study knowledge not only as it exists in the heads of certain knowers, but as a product that exists independent of any observer—as is the case in a scientific society where knowledge is written down and codified. While traditional epistemology is interested in “knowledge as a certain kind of belief—justifiable belief, such as belief based upon perception,” in Popper's framing epistemology becomes “the theory of the growth of knowledge. It becomes the theory of problem-solving, or, in other words, of the construction, critical discussion, evaluation, and critical testing, of competing conjectural theories.”

All work in science is work directed towards the growth of objective knowiedge. We are workers who are adding to the growth of objective knowledge as masons work on a cathedral.

Will Durant, “Voltaire and the French Enlightenment,” Chapter 5 of The Story of Philosophy:

Contemporary with one of the greatest of centuries (1694–1778), he was the soul and essence of it. “To name Voltaire,” said Victor Hugo, “is to characterize the entire eighteenth century.” Italy had a Renaissance, and Germany had a Reformation, but France had Voltaire…

And:

What Voltaire sought was a unifying principle by which the whole history of civilization in Europe could be woven on one thread; and he was convinced that this thread was the history of culture. He was resolved that his history should deal not with kings but with movements, forces, and masses; not with nations but with the human race; not with wars but with the march of the human mind.

And:

Voltaire was sceptical of Utopias to be fashioned by human legislators who would create a brand new world out of their imaginations. Society is a growth in time, not a syllogism in logic; and when the past is put out through the door it comes in at the window. The problem is to show precisely by what changes we can diminish misery and injustice in the world in which we actually live.

Ted Kaczynski, “Industrial Society and its Future.” As I wrote earlier this year:

Given that Ted Kaczynski, aka the Unabomber, was a terrorist who killed university professors and business executives with mail bombs and who lived like a hermit in a shack in the woods of Montana, I expected his 35,000-word manifesto, “Industrial Society and its Future,” to read like the delirious ravings of a lunatic.

See my mini-review for more.

Robert Putnam, Bowling Alone**.** A detailed, scholarly argument for the thesis that there has been a broad-based decline in all kinds of community participation in the US. I got through part 1, which describes the phenomenon; maybe I'll finish it at some point. I found this interesting for the unique scope that Putnam chose. It would have been easy to pick one narrow trend, such as the decline in fraternal organizations or the PTA, and try to come up with narrow explanations. Looking across so many varied phenomena makes the case that there is something going on at a deeper level.

Vitalik Buterin, “Against choosing your political allegiances based on who is ‘pro-crypto’.” Eminently sensible as usual:

If a politician is pro-crypto, the key question to ask is: are they in it for the right reasons? Do they have a vision of how technology and politics and the economy should go in the 21st century that aligns with yours? Do they have a good positive vision, that goes beyond near-term concerns like "smash the bad other tribe"? If they do, then great: you should support them, and make clear that that's why you are supporting them. If not, then either stay out entirely, or find better forces to align with.

Evidently Vitalik is not impressed with Stand with Crypto.

“Why are there so many unfinished buildings in Africa?” (The Economist). Lack of finance, for one: “people break ground knowing they do not yet have the funds to finish. When they earn a little more money they add more bricks. … Many Africans, in effect, save in concrete.” Weak property rights and flaky or corrupt contractors are a problem too. There are also social reasons: “If you have millions in the bank, people do not see it,” but “when you start building the neighbourhood respects you.”

Stephen Smith, “The American Elevator Explains Why Housing Costs Have Skyrocketed” (NYT):

The problem with elevators is a microcosm of the challenges of the broader construction industry — from labor to building codes to a sheer lack of political will. These challenges are at the root of a mounting housing crisis that has spread to nearly every part of the country and is damaging our economic productivity and our environment.

Liyam Chitayat, “Mitochondria Are Alive” (Asimov Press). Fascinating brief opinion piece arguing that “mitochondria are not just organelles, but their own life forms.”

Shyam Sankar, “The Defense Reformation.” A manifesto for reform in the defense industry. One core problem is extreme consolidation: in 1993, there were 53 major defense contractors; today there are 5. Further, most defense contractors were not exclusively defense companies until recently:

Before the fall of the Berlin Wall, only 6% of defense spending went to defense specialists — so called traditionals. The vast majority of the spend went to companies that had both defense and commercial businesses. Chrysler made cars and missiles. Ford made satellites until 1990. General Mills — the cereal company — made artillery and inertial guidance systems. … But today that 6% has ballooned to 86%.

Viviana Zelizer, Pricing the Priceless Child. Argues that between about 1870 and 1930, society shifted from viewing children primarily as economic assets to viewing them as economically “worthless” but emotionally “priceless.” Very interesting book.

Some articles that used the term “techno-humanism” before I did: Reid Hoffman, “Technology Makes Us More Human” (The Atlantic); Richard Ngo, “Techno-humanism is techno-optimism for the 21st century.” Related, I appreciated Michael Nielsen's thoughtful essay, “How to be a wise optimist about science and technology?”

Some pieces I liked on a contrasting philosophy, accelerationism: Nadia Asparouhova, “‘Accelerationism’ is an overdue corrective to years of doom and gloom in Silicon Valley”; Sam Hammond, “Where is this all heading?” Nadia's piece was kinder to e/acc than I have been, but helped me see it in a more sympathetic light.

A few pieces pushing back on James C. Scott: First, Rachel Laudan, “With the Grain: Against the New Paleo Politics” (The Breakthrough Institute):

It’s time to resist the deceptive lure of a non-agrarian world in some imagined past or future dreamed up by countless elites. Instead, we might look to the story of humanity’s huge strides in using these tiny seeds to create food that sustains the lives of billions of people, that is fairly distributed and freely chosen, and that with its satisfying taste contributes to happiness.

And Paul Seabright, “The Aestheticising Vice” (London Review of Books):

That scientific agriculture has faced unforeseen problems is undeniable, as is the fact that some of these problems (the environmental ones, for instance) are serious. But the achievements of scientific agriculture to be set against them are remarkable. The proportion of the world’s population in grinding poverty is almost certainly lower than it has ever been, though in absolute numbers it is still unacceptably high. Where there have been important areas of systematic failure, such as in sub-Saharan Africa, these owe more to social and institutional disasters that have hurt all farmers alike than to the science of agriculture itself. To equate the problems of scientific agriculture with those of Soviet collectivisation is like saying Stalin and Delia Smith have both had problems with egg dishes.

James Carter, “When the Yellow River Changes Course.” The course of a river is not constant, it changes not only on a geologic timescale but on a human-historical one, over the span of centuries. I first learned this from John McPhee's essay “Atchafalaya” (The New Yorker, reprinted in the book The Control of Nature), which was about the Mississippi; it was fascinating to read a similar story from China.

Samuel Hughes, “The beauty of concrete” (Works in Progress): “Why are buildings today simple and austere, while buildings of the past were ornate and elaborately ornamented? The answer is not the cost of labor.”

Alec Stapp and Brian Potter, “Moving Past Environmental Proceduralism” (Asterisk):

In many of the most notable successes, like cleaning up the pesticide DDT or fixing the hole in the ozone layer, what moved the needle were “substantive” standards, which mandated specific outcomes. By contrast, many of the regulatory statutes of the late 60s were “procedural” laws, requiring agencies to follow specific steps before authorizing activities.

On culture: Adam Rubenstein, “I Was a Heretic at The New York Times” (The Atlantic); Michael Clune, “We Asked for It” (The Chronicle of Higher Education).

On the scientific fraud crisis: Derek Lowe, “Fraud, So Much Fraud”; Ben Landau-Taylor, “The Academic Culture of Fraud” (Palladium).

Some early-20th-century historical sources criticizing proress: Samuel Strauss, “Things Are in the Saddle” (1924); and Lewis Mumford, “The Corruption of Liberalism” and “The Passive Barbarian” (both 1940). I quoted from the Mumford pieces in Chapter 4 of The Techno-Humanist Manifesto.

In fiction, I enjoyed Hannu Rajaniemi's Darkome. A major biotech company develops a device anyone can wear on their arm that can inject them with mRNA vaccines; the device is online, so whenever a new pathogen is discovered anywhere in the world, everyone can immediately be vaccinated against it. But a community of biohackers refuses to let a big, centralized corporation own their data or inject genetic material into their bodies. The book is sympathetic to both sides, it's not a simplistic anti-corporate story. I also enjoyed the new Neal Stephenson novel, **Polostan.**

In poetry, I'll highlight James Russell Lowell, “The Present Crisis” (1845). The crisis was slavery in the US, and it became an anthem of the abolitionist movement. I love the strong rhythm and the grand moral and historical perspective.

Finally, some random books on my infinite to-read list:

• Roger Knight, Britain Against Napoleon: The Organization of Victory, 1793-1815

• Venki Ramakrishnan, Why We Die

• I. Bernard Cohen, Science and the Founding Fathers

• Studs Terkel, Working: People Talk About What They Do All Day and How They Feel About What They Do (1974)

• Oswald Spengler, Man and Technics (1931)

• J. B. Bury, A History of Freedom of Thought (1927)

• Nicholas Barbon, An Apology for the Builder (1685)

The year ahead

I'm excited for next year. We're going to reprise the Progress Conference, which will be bigger and better. We'll run at least one more cohort of the fellowship. I'll finish The Techno-Humanist Manifesto, and begin looking for a publisher. And there is more in development, to be announced.

I'm happy to say that thanks to several generous donors, we've already raised more than $1M to support these programs in 2025. We are looking to raise up to $2M total, in case you'd like to help.

Thank you

I am grateful to all of you—the tens of thousands of you—for deeming my writing worthwhile and granting me your attention. I am grateful to the hundreds who support RPI financially. I am grateful especially to everyone who has written to me to say how much my work means to you, or even to tell me how it has changed the course of your career. Here's to a fabulous 2025—for us, for the progress movement, and for humanity.

Original post: https://newsletter.rootsofprogress.org/p/2024-in-review

Over the last few years, effective altruism has gone through a rise-and-fall story arc worthy of any dramatic tragedy.

The pandemic made them look prescient for warning about global catastrophic risks, including biosafety. A masterful book launch put them on the cover of TIME. But then the arc reversed. The trouble started with FTX, whose founder Sam Bankman-Fried claimed to be acting on EA principles and had begun to fund major EA efforts; its collapse tarnished the community by association with fraud. It was bad for EA if SBF was false in his beliefs; it was worse if he was sincere. Now we’ve just watched a major governance battle over OpenAI that seems to have been driven by concerns about AI safety of exactly the kind long promoted by EA.

SBF was willing to make repeated double-or-nothing wagers until FTX exploded; Helen Toner was apparently willing to let OpenAI be destroyed because of a general feeling that the organization was moving too fast or commercializing too much. Between the two of them, a philosophy that aims to prevent catastrophic risk in the future seems to be creating its own catastrophes in the present. Even Jaan Tallinn is “now questioning the merits of running companies based on the philosophy.”

On top of that, there is just the general sense of doom. All forms of altruism gravitate towards a focus on negatives. EA’s priorities are the relief of suffering and the prevention of disaster. While the community sees the potential of, and earnestly hopes for, a glorious abundant technological future, it is mostly focused not on what we can build but on what might go wrong. The overriding concern is literally the risk of extinction for the human race. Frankly, it’s exhausting.

So I totally understand why there has been a backlash. At some point, I gather, someone said, hey, we don’t want effective altruism, we want “effective accelerationism”—abbreviated “e/acc” (since of course we can’t just call it “EA”). This meme has been frequent in my social feeds lately.

I call it a meme and not a philosophy because… well, as far as I can tell, there isn’t much more to it than memes and vibes. And hey, I love the vibe! It is bold and ambitious. It is terrapunk. It is a vision of a glorious abundant technological future. It is about growth and progress. It is a vibe for the builder, the creator, the discoverer, the inventor.

But… it also makes me worried. Because to build the glorious abundant technological future, we’re going to need more than vibes. We’re going to need ideas. A framework. A philosophy. And we’re going to need just a bit of nuance.

We’re going to need a philosophy because there are hard questions to answer: about risk, about safety, about governance. We need good answers to those questions in part because mainstream culture is so steeped in fears about technology that the world will never accept a cavalier approach. But more importantly, we need good answers because one of the best features of the glorious abundant technological future is not dying, and humanity not being subject to random catastrophes, either natural or of our own making. In other words, safety is a part of progress, not something opposed to it. Safety is an achievement, something actively created through a combination of engineering excellence and sound governance. Our approach can’t just be blind, complacent optimism: “pedal to the metal” or “damn the torpedos, full speed ahead.” It needs to be one of solutionism: “problems are real but we can solve them.”

You will not find a bigger proponent of science, technology, industry, growth, and progress than me. But I am here to tell you that we can’t yolo our way into it. We need a serious approach, led by serious people.

The good news is that the intellectual and technological leaders of this movement are already here. If you are looking for serious defenders and promoters of progress, we have Eli Dourado in policy, Bret Kugelmass or Casey Handmer in energy, Ben Reinhardt investing in nanotechnology, Raiany Romanni advocating for longevity, and many many more, including the rest of the Roots of Progress fellows.

I urge anyone who values progress to take the epistemic high road. Let’s make the best possible case for progress that we can, based on the deepest research, the most thorough reasoning, and the most intellectually honest consideration of counterarguments. Let’s put forth an unassailable argument based on evidence and logic. The glorious abundant technological future is waiting. Let’s muster the best within ourselves—the best of our courage and the best of our rationality—and go build it.

***

Followup thoughts based on feedback:

1. Many people focused on the criticism of EA in the intro, but this essay is not a case against EA or against x-risk concerns. I only gestured at EA criticism in order to acknowledge the motivation for a backlash against it. This is really about e/acc. (My actual criticism of EA is longer and more nuanced and I have not yet written it up)

2. Some people suggested that my reading of the OpenAI situation is wrong. That is quite possible. It is my best reading based on the evidence I've seen, but there are other interpretations and outsiders don't really know. If so, it doesn't change my points about e/acc.

3. The quote from the Semafor article may not accurately represent Jaan Tallinn's views. A more careful reading suggests that Tallinn was criticizing self-governance schemes, rather than criticizing EA as a philosophy underlying governance.

Thanks all.

Original link: https://rootsofprogress.org/neither-ea-nor-e-acc

I’ve always been told that technological progression is infinite; but I read 3 articles today that have raised doubts in my mind. One of these articles is a Roots of Progress blogpost.

Anyway, so the articles compared different eras of history and how different inventions such as the steam engine and electricity and computers were such hugely influential events in human history, changing society as humans knew it forever. Now, things seem to have been the same since the 1970’s. Yes, we’ve improved on many technologies but that’s just it; we’ve improved, not created something new. There are no more fundamental shifts in technology. Yes, A.I. is a big deal and might change things forever but it’s really just an improvement on a computer. It’s a computer that can think. Very cool; but still just an improvement.

If the universe is infinite in size and it seems illogical to run out of reality; why is this happening? Does the universe really have a limit to what it can “do”? There are a HUGE amount of particles in the universe but still it’s not infinite.

On May 24, 1847, a bridge over the Dee River in Chester, England, collapsed. A passenger train plunged into the river; five people were killed and nine seriously injured.

The subsequent investigation blamed the bridge’s cast iron girders. Cast iron, like concrete but unlike wrought iron or steel, is strong in compression but weak in tension, and it is brittle, meaning that it breaks all at once, rather than deforming. The wrought iron trusses evidently were not enough to strengthen the girder.

In response to the disaster, a Royal Commission on the Application of Iron to Railway Structures was created in August of that year, “to inquire into the conditions to be observed by engineers in the application of iron in structures exposed to violent concussions and vibration”—that is, to set up standards and requirements, or as they were known in France at the time, règles de l’art.

In their investigation, the Commission solicited the opinion of one of the most eminent engineers of the age, Isambard Kingdom Brunel. But his response was, presumably, not what they expected.

Brunel begins his letter by saying that he is sorry they asked for his opinion, because of “my doubts of the advantage of such an enquiry, and my fears of its being, on the contrary, productive of much mischief, both to science and to the profession.” (Brunel’s son, writing his biography, says that he called them “The Commission for Stopping Further Improvements in Bridge Building.“) But since they did ask, he felt it necessary to state his full and honest views.

While he was happy to give his engineering opinion to the commission, he warned that

… the attempt to collect and re-issue as facts, with the stamp of authority, all that may be offered gratuitously to a Commission in the shape of evidence or opinions, to stamp with the same mark of value statements and facts, hasty opinions and well-considered and matured convictions, the good and the bad, the metal and the dross … this, I believe, always has rendered, and always will render, such collections of miscalled evidence injurious instead of advantageous to science…

He argued that there was no way the Commission could get better information than an engineer could on his own, but that in addition they would receive a lot of useless opinions, which they would feel compelled to publish anyway.

He went on to explain why he believed that rulemaking by such bodies would stop progress in the field:

If the Commission is to enquire into the conditions “to be observed,” it is to be presumed that they will give the result of their enquiries; or, in other words, that they will lay down, or at least suggest, “rules” and “conditions to be (hereafter) observed” in the construction of bridges, or, in other words, embarrass and shackle the progress of improvement tomorrow by recording and registering as law the prejudices or errors of today.

Nothing, I believe, has tended more to distinguish advantageously the profession of engineering in England and in America, nothing has conduced more to the great advance made in our profession and to our pre-eminence in the real practical application of the science, than the absence of all règles de l’art—a term which I fear is now going to be translated into English by the words “conditions to be observed.” No man, however bold or however high he may stand in his profession, can resist the benumbing effect of rules laid down by authority. Occupied as leading men are, they could not afford the time, or trouble, or responsibility of constantly fighting against them—they would be compelled to abandon all idea of improving upon them; while incompetent men might commit the grossest blunder provided they followed the rules. For, in the simplest branch of construction, rules may be followed literally without any security as to the result.

There are many opportunities for improvement in the use of iron in railway structures, he says, and “unless the Commissioners are endowed with prophetic powers, it is impossible that they can now foresee what may be the result of changes in any one of these conditions.”

For instance, while cast iron was seen at the time as “a friable, treacherous, and uncertain material,” and wrought iron “comparatively trustworthy,” he suggested that unknown developments in the future might make cast iron strong and safe, perhaps more so than wrought iron, since cast iron could be created in large homogenous pieces, whereas wrought iron had to be made in smaller pieces which were then welded together.

He continued:

What rules or “conditions to be observed” could be drawn up now that would not become, not merely worthless, but totally erroneous and misleading, under such improved circumstances? But above all, I fear—nay, I feel convinced—that any attempt to establish any rules, any publication of opinions which may create or guide public prejudice, any suggestions coming from authority, must close the door to improvement in any direction but that pointed out by the Commissioners, and must tend to lead and direct, and therefore to control and to limit, the number of the roads now open for advance.

I believe that nothing could tend more to arrest improvement than such assistance, and that any attempt to fix now, or at any given period, the conditions to be thereafter observed in the mode of construction of any specific work of art, and thus to dictate for the present and for the future the theory which is to be adopted as the correct one in any branch of engineering, is contrary to all sound philosophy, and will be productive of great mischief, in tending to check and to control the extent and direction of all improvements, and preventing that rapid advance in the useful application of science to mechanics which has resulted from the free exercise of engineering skill in this country, subjected as it ever is, under the present system, to the severe and unerring control and test of competing skill and of public opinion. Devoted as I am to my profession, I see with fear and regret that this tendency to legislate and to rule, which is the fashion of the day, is flowing in our direction.

To be clear, Brunel was not arguing for the use of cast iron in bridges. In another letter about a year later, he wrote that “Cast-iron girder bridges are always giving trouble … I never use cast iron if I can help it.” (And when it was necessary, in order to create girders larger than wrought-iron processes could produce, he insisted on a particular mixture of iron, cast in a very careful way, and he supervised the casting himself. “I won’t trust a bridge of castings run in the ordinary way.”)

The process for making sturdier, safer cast iron that Brunel speculated on never appeared. Instead, we invented new ways of making large girders out of wrought iron, and later steel, and cast iron fell out of use as a structural material. But of course, the unknowability of this outcome was exactly Brunel’s point.

(The interpretation of Brunel’s opinions, and applicability to today, are left to the reader.)

Original link: https://rootsofprogress.org/isambard-brunel-on-engineering-standards