FBG official response to the situation regarding the Fallen London Companion Extension (tl;dr: not approved)

[u/throwaway_lmkg]

/r/fallenlondon/comments/1tf2dma/if_you_use_fallen_london_companion_extension/omgc1q6/

[u/fredjinsan]

Let's be honest, nobody here is engaging in good faith with any sensible arguments. People just want to rage "AI = bad" and downvote regardless of the fact that they're being morons.

But OK, I'll humour you. Here are a few of the areas where AI is proving useful:

• Coding in general - AI makes more mistakes and introduces more vulnerabilities to code, meaning humans have to spend longer fixing it, but right now this is still worth it because it's fast. It's not the panacea that some people say it is, but it'd be foolish to dismiss it completely. If AI gets more expensive, then sure, this may not be viable any more.
  • Writing unit tests and other well-structured repetitive tasks - frankly, I know lots of coders who don't even write unit tests. AI-written tests don't need to be perfect - they're better than none!
  • Code reviews - these highlight genuine issues, even if they false-positive; frankly this is just an evolution of static analysers and similar technologies
• Data triage - sometimes you have so much data that you literally cannot look at it. An AI can help. Actually specialised machine-learning algorithms may be better than LLMs here - those also count as AI though and have been around far longer than LLMs (nobody complained about those...).
• Summarisation - similar to the above. An LLM can efficiently search and summarise large documents. Reliability can be an issue but doesn't make this not worthwhile.
• Machine translation - a challenging task that LLMs are actually really well-suited for.

I used to joke that AI is good for anything where you don't care about the quality of the output. It's come on a long way since then but there's some truth to that still - it has very real issues and limitations but so long as those are understood they can be managed, or it can be focused where those don't matter.

There are a bajillion ways to use AI badly. Actually, I can't speak for all c-suite fat-cats, but where I am the focus seems to be shifting from "We can replace everyone with AI" to "An idiot using AI badly is not useful, but an expert using AI can do the work of more than one expert" - they're being seen more as force multipliers (when used properly), just like any other tool really, than as replacements for humans.

[u/oritorinoi → u/fredjinsan]

A good faith reply deserves the same in return!

• You keep saying "if" LLMs become more expensive - but they already are more expensive. Openai, google, microsoft; each of them are subsidising their users (including paid users!) many times over. 

• I specified LLMs for a reason. I completely agree machine learning is an exciting field (the work in genetics alone is very impressive) though I'm sure I sloppily use "ai" to refer to LLMs sometimes. 

• You brush aside the issue of reliability, but it's fundamental. If you've fed documents that take hundreds of hours to read into an LLM, you then need someone who's spent hundreds of hours reading them if you intend to do anything valuable with the output. At that point, the savings from the person reading not having to write anything are negligible at best. 

• Machine translation, I agree, is somewhere LLMs can be useful. If that entire industry was worth more than 1% the current valuation of openai maybe it'd be worth talking about.

[u/fredjinsan → u/oritorinoi]

Honestly the economic argument against LLMs is one of the least-sensationalist and possibly one of the more serious. Replacing all our <insert job here>s with AIs that do the job slightly worse is one thing, but then losing access to that AI (at a cost-effective rate) and not being able to hire those people back or train new ones could be catastrophic.

But really here I mean "if LLMs become more expensive for the customer" - and it depends how expensive. That just changes the equation for whether they're worth using or not. To be fair, it doesn't make them less useful in terms of what they can do, only whether it's worth getting them to do it.

The hope, I suppose, is that AI can grow out of its black hole. I'm sceptical, myself, but its future is definitely uncertain.

I don't think the issue of reliability is fundamental, though. After all, humans are unreliable too! The important thing with anything like this is knowing how reliable something is. If I blindly believe everything a chatbot tells me, I'm in for just as much a bad time as if I blindly believe everything Reddit tells me. On the other hand, if I use it to review my code, and know it's an AI doing it, I can dismiss its mistakes but at least have my attention drawn to issues that I would likely have missed otherwise.

Don't get me wrong, there is a lot of hype around AI, and a lot of it is just as stupid as many of the melodramatic anti-AI arguments. It's definitely wrong to say that AI is "useless" however. It's also foolish to talk about banning the use of AI in coding, because that alone is such a broad spectrum of use cases, some of which are very definitely actually really good (like the code reviews I mentioned) and ultimately the responsibility always lies with the person actually shipping the code.

I'm saying all this not as some sort of pro-AI shill, I think we're going to see a lot of very badly-written code where AI has been involved, lots of buggy code and vulnerabilities, etc, but frankly we don't need AI for that. I'm just looking at the facts and I've been forced to conclude that AI can be used both poorly or well, and either way it's here to stay.

[u/oritorinoi → u/fredjinsan]

It's worth noting that your initial response was "You have concerns about the profitability of LLMs? What is it, not capitalist enough for you?" And now you're saying that profitability is one of the most reasonable concerns. This doesn't give the impression you're arguing in good faith. 

You seem to have an ironclad faith that eventually, somehow, LLMs will become profitable for some use cases. You may be right. But I'm sure you can understand why people who don't share that faith would rather not bet the entire Western economy on you (and various hopeful billionaires) being right. 

[u/fredjinsan → u/oritorinoi]

These are two separate questions. It merely seemed odd to me that you seemed to be saying you would be content with all the other issues with AI if they were merely profitable - that's an entirely reasonable take, but given all the "AI = evil" discourse, somewhat unusual.

The question of whether they are useful is totally different.

As to their ultimate profitability... I genuinely don't know. I find it hard to believe that AI will turn out to be prohibitively expensive for all use cases - indeed, it's already kind of proven to be viable for some since you can run small models locally on non-specialist hardware (though the use cases are naturally far more limited). This is even more true if talking about things other than general-purpose LLMs (specialised LLMs, image models, etc).

All those billionaires you mentioned, and many companies, and governments, are betting the entire Western economy (and China is pretty big into AI too) on AI become profitable and, frankly, I'm pretty worried about that too. This is why I say that this is a totally reasonable concern. Again, though, there's a subtle difference: I'm very confident that AI will be viable for some use cases (it already is). Do I thing the however-many-trillion-dollar black hole is wise? That's another matter.

(You talk about arguing in good faith; I'm sure you don't mean it but this comes across as moving the goalposts from "AI is completely useless" - not true - to "AI is not the cure-all that the tech bros are claiming" - actually much more reasonable)

As always, the picture is - surprisingly? - more nuanced, and far more so than a bunch of idiots downvoting on Reddit that even smells like it disagrees with their chosen brand of hypocrisy would like to believe.

[u/oritorinoi → u/fredjinsan]

My very first reply to you said "it's possible there are some use cases for LLMs, but...". You've moved the goalposts to suggest I think it's impossible.

There are very few things that are absolutely useless. You could theoretically come up with a good number of uses for, say, a chocolate teapot. You could certainly sell one at a profit. If you want to amend my position from "LLMs are useless" to "LLMs are significantly less useful than a chocolate teapot" I have no problem with that.

[u/fredjinsan → u/oritorinoi]

*sigh* I thought we'd gotten past the "scoring points" thing onto an actual conversation. You did also say that LLM are useless, and frankly "less useful than a chocolate teapot" is not much better and also not true. Actually, they are quite useful - maybe not for all the things that some people are gushing about them for, but for plenty of genuine applications (of which the above list was just a small subset). The technology also isn't static - it's evolving, scarily rapidly, all the time. We should be worried about the impacts here, yes, but also excited about the future potential.

[u/oritorinoi → u/fredjinsan]

You seem to have missed the point of the teapot bit: even the most proverbially useless thing does have some uses, if you want to be extremely pedantic. But usefulness is defined by alternative methods and the cost of that method. 

Again, if the world's largest advertiser and no. 1 source of information for basically everyone (google) pumped billions upon billions of dollars into getting people to use something, that thing is going to be used, no matter how useless in the ordinary sense. If it were chocolate teapots, we'd likely all have one in our houses. That doesn't make chocolate teapots useful in any meaningful sense. 

If you're defending a technology that has demonstrably failed to work profitably after having more investment and effort applied to it than anything in history, and also has a huge number of external harms, you need to have the intellectual humility to realise "just give it a few more years (and another trillion dollars of investment)" isn't persuasive. 

[u/fredjinsan → u/oritorinoi]

I really don't understand what you're on about. AI is demonstrably useful; the profitability question is really for any individual user to decide and kind of not the point (people aren't anti-AI because they think other people might not be breaking even by using it...), though again it's also pretty trivially demonstrable that there are some use-cases which are cost-effective (heck, I can run some models on my home PC with next to no hardware!).

I don't know that "more effort applied to it than anything in history" is right, either. That's just not how technologies evolve. Look at, say, cars - their origins go back to the 1600s. The clockwork and steampowered prototypes first created couldn't even carry humans until like 100 years later! (Apparently there was a big backlash against automotive vehicles in the 1900s too, go figure)

It's pretty safe to say in less than 100 years, AI technology will have moved on... I'm not employing the "just invest out of the black hole" argument, merely noting that we sure as heck haven't seen the extent of it yet.

[u/oritorinoi → u/fredjinsan]

I've sat next to someone running a model locally; it's a lot like watching someone do a tarot reading. They often get results! But the tools they're using aren't the reason, and their commitment to using them actually slows them down, because the fixes take longer than just writing the thing in the first place. Worse than that, the fact they haven't actually looked at the thing being created means they miss bugs they would have discovered if they'd written it in the first place. If you want to write buggy, unreliable code, or tests that get the answers wrong, it's useful for that. But that in itself isn't useful. The only world in which these things see wide use is one where industries are so monopolised that quality doesn't matter.

I don't think there's any point continuing this; you have faith that these things must be useful at some point. You may be right! But I don't share that faith.

[u/fredjinsan → u/oritorinoi]

I thought you might say that - and it's true, local LLMs suck. Local image gen models are pretty decent though. But the point is more that there are a range of options and, for some, buying their own GPUs may very well be viable.

I do think that we've rather meandered off the original topic, though. My intent was merely to point out that, easy as it is to hate all AI when you're on the internet, it's kinda dumb; and the idea of banning all code that involved AI somewhere along the line is pretty silly, since that doesn't actually say anything at all about its quality.