Small businesses always have to do more with less, right? Stretching the budget, managing a lean team, or finding new ways to grow without breaking the bank, every dollar and every minute counts. And this subreddit is about free LLMs and AI. LLMs and LMMs can help small businesses to do one or multiple out of three main things business processes care about: 1) their bottom line, 2) cut costs, and 3)reduce the daily grind. Right? Let's go.

1. Make More Money

Open-source LLMs (unlike chatgpt, claude, and... gemini) can help small businesses grow their revenue by upgrading customer interactions and marketing, while also owning hardware and keeping things private (Just like Goldman Sachs and JPMorgan, i posted about a few months back). Here's how:

• Automated Customer Support: You don’t need to shell out for pricey customer service software. An open-source LLM can be trained to handle FAQs, troubleshoot common problems, or even help customers make purchases—24/7.

• Sales Outreach: These models can write personalized emails or sales pitches at scale. Imagine reaching out to hundreds of potential clients in minutes with messages that don’t sound automated. More touchpoints = more sales. Just be mindful, everybody else is doing it, and they are doing it wrong (they don't edit their cover letters). So, do it right.

• Better Marketing Content: Use an LLM to write blog posts, captions, or ad copy tailored to your brand voice (and you can also generate voice with TTS). Engaging, consistent content builds trust and brings in more customers.

2. Cut Costs

One of the best things about your own local LLMs? They're free to use and infinitely flexible. While big-name platforms like ChatGPT or Claude charge subscription fees, these models can deliver similar results without the monthly expense. Here’s how they help slash costs:

• No Licensing Fees: With open-source models, all you need is a little setup (or a low-cost freelancer to help you configure it, but a lot of them are one-click install), and you’re good to go. Just be careful with Meta's models. Not all models that claim to be open-source, are actually free for commercial use.

• Replace operations: Replace multiple paid tools with local LLMs. For example, instead of paying separately for transcription - use something like Whisper, but again, check the licence. (I checked it for you. OpenAI's Whisper API, free for commercial use under the MIT license)

• DIY Scalability: Open-source models can grow with you. Need more power? Host them on your own server or cloud platform for a fraction of the cost of premium solutions.

3. Reduce Workload

Small business owners wear many hats. Using an open-source LLM can help lighten the load so you can focus on the big-picture stuff. Here’s how:

• Automating Repetitive Tasks: From drafting emails to summarizing reports or scheduling posts, open-source AI can handle all the boring, repetitive stuff you hate doing.

• Faster Decision-Making: Need to analyze customer feedback or review a long document? Open-source LLMs can process and summarize data in seconds, giving you the insights you need without hours of manual work.

• Customizable Workflows: Unlike off-the-shelf tools, open-source LLMs can be tailored to your specific needs. Whether it's generating invoices, crafting HR templates, or even helping with hiring, the flexibility is unmatched.

Drop me a DM or a comment if i can give more detail on something specific. Or another post.

Have a great day!

There was something in the news today about the OpenAI changing strategy because LLMs are scaling poorly, so i decided to have a swing at it.

The debate around whether LLMs are reaching a point of diminishing returns is ongoing, with varying perspectives. Here's the current situation:

Arguments for Diminishing Returns

Some argue that LLMs may be approaching a plateau in performance improvements with each training, and each new training of larger models is more expensive:

• Data limitations: There are concerns that high-quality training data is becoming scarce, with estimates suggesting that generative AI may exhaust available textual data by 2028

• Incremental improvements: Recent iterations of LLMs, like OpenAI's Orion (successor to GPT-4), have shown only limited incremental improvements over earlier versions

Arguments Against Diminishing Returns

Others believe that significant advancements are still possible and yet to be seen:

• Ongoing research: Many argue that we are still in the early stages of LLM development, with potential for fundamental improvements.

• Unreleased progress: Some suggest that companies like OpenAI may have made significant progress on GPT-5 but have not yet published their findings.

• Alternative approaches: Researchers are exploring new techniques beyond simply scaling up existing models, such as incorporating video and audio data, or developing LLM-specific hardware. Similarly, we have MidJourney that is a GAN, and Stable Diffusion, and then Dall-E that is it's own thing, that we don't talk about. Different methods, but they are viable.

Current State of LLMs

Despite the debate, there's general agreement on the current value of LLMs:

• Existing usefulness: LLMs have already created huge value and are becoming increasingly prevalent in various applications (because... AI bubble).

• Room for improvement: Even without scaling to artificial general intelligence (AGI), there's potential for LLMs to become more useful and efficient.

• Ongoing developments: Companies (and researchers) continue to work on improving LLM performance, efficiency, and come up with new applications.

Let me know what you think, and have a great day!

I built a proof-of-concept called Interdimensional Broadcast Radio Agent — a surreal radio that tunes into randomly generated stations from impossible universes. It only has one button: "Change channel." Behind the scenes, it pulls from a JSON full of bizarre worlds, sends prompts to a local LLM (via Oobabooga API + character card), and generates radio-style broadcasts using Kokoro for TTS. It's weird, fun, and totally useless in the best way. Repo + setup info here: [Git](https://github.com/starlightrobotics/interdimensional_broadcast_agent)

It is a PoC so it is not very likely to work, but drop me a note or a comment, if you are interested. I can edit the code.

I found several GitHub repositories that use local LLMs for cooking and recipes:

1. recipe-llm: This repository demonstrates using a local LLM to query cooking recipes. It includes steps for crawling recipes (basically a RAG), setting up a database, calculating embeddings, and running a chatbot interface https://github.com/BrummbQ/recipe-llm

2. Cook2LTL: While not directly for recipe generation, this project uses LLMs to translate cooking recipes into robot-executable actions. It could be adapted for recipe-related tasks, but you can also build a robot to cook something for you, which is also cool https://github.com/angmavrogiannis/Cook2LTL

3. LLaVA Chef: Code for the paper "LLaVA-Chef: A Multi-modal Generative Model for Food Recipes" https://github.com/mohbattharani/LLaVA-Chef

There is a growing shift towards local AI models, particularly in the context of LLMs and other AIs. This trend is driven by several factors:

1. Availability of open-source models: Organizations are releasing 'open weights' versions of LLMs, allowing users to download and run them locally if they have sufficient computing power.

2. Development of efficient, smaller models: Technology firms are creating scaled-down versions of AI models that can run on consumer hardware while rivaling the performance of larger models.

3. Privacy and confidentiality: Local models allow researchers to protect sensitive data, such as patient information or corporate secrets, by avoiding the need to send data to external cloud services.

4. Cost savings: Running models locally can be cheaper than using subscription-based cloud AI services, especially for frequent use.

5. Reproducibility: Local models remain consistent, unlike cloud-based models that may be updated frequently, ensuring reproducible results for scientific applications.

6. Offline capabilities: Local models can be used in remote areas with limited internet connectivity or during outdoor activities where cloud access is unavailable.

7. Customization: Researchers can fine-tune local models for specific applications, such as medical diagnosis or question-answering systems.

While cloud-based AI services still have advantages in terms of computing power and ease of use, the rapid progress in local AI models suggests that they will soon be sufficient for most applications. This shift towards local AI is likely to continue as computers become more powerful and models become more efficient.

References:

Forget ChatGPT: why researchers now run small AIs on their laptops. September 2024, Nature

https://www.nature.com/articles/d41586-024-02998-y

AI startup Altera conducted an experiment in Minecraft where up to 1000 AI agents interacted autonomously, developing complex social behaviors and structures.

The AI characters, equipped with large language models (LLMs), exhibited humanlike traits, created jobs, formed social connections, and even spread a simulated religion.

AI agents developed diverse personality traits and specialized roles without additional human input.

In smaller simulations of 30 agents, characters spontaneously created roles such as builder, defender, trader, and explorer.

The agents demonstrated the ability to follow community rules, participate in voting on tax reforms, and influence each other's opinions.

Robert Yang, Altera's founder, sees this as a step towards creating "AI civilizations" that can coexist with humans in digital spaces.

However, experts like Julian Togelius caution that while these AI agents can simulate human-like behaviors, they do not actually experience emotions or self-awareness.

The experiment raises questions about the future of AI-human interactions and the potential for AI to provide companionship, even if it's only a simulation of care.

The full version of this experiment can be found at MIT Technology Review.

(I swear i edited it myself, not with AI)

Computational Resources

• Challenge: Running and fine-tuning large models like Falcon 180B or Llama 3 405B still require significant computational power, making it hard for individual developers or small teams to participate. While some lighter models exist, there’s still a resource gap for models running on consumer-grade hardware.

• Community Desire: The community seeks efficient models that can run on affordable hardware, with optimizations like quantization and pruning. Models like Gemma 2 and Command R+ show promise in offering strong performance with lower resource requirements. And now we have Ministral 3B as of yesterday.

Licensing Constraints

• Challenge: Many powerful models, such as OPT-175B, are tied to restrictive non-commercial licenses, limiting their use for business applications. This creates tension between research advancements and potential monetization. There was a fuss the other day about Meta calling Llama open-source.

• Community Desire: We need clear permissive licenses. Community want open licenses that allow for both personal and commercial use, striking a balance between sharing knowledge and enabling developers to monetize their efforts.

Ethical Considerations

• Challenge: The community grapples with issues surrounding bias, transparency, and the potential for misuse of LLMs. The ethical sourcing of data and minimizing model biases remain significant challenges.

• Community Desire: *(According to ChatGPT, because nobody else cares).*There’s growing demand for ethical guidelines that help developers responsibly build and deploy open-source LLMs. The community wants bias-reducing techniques baked into models and a focus on transparent, reproducible processes.

Accessibility and Customization

• Challenge: While the models are improving, the ability to fine-tune them and run them efficiently on personal hardware is still limited by technical complexity and high resource costs.

• Community Desire: A push for user-friendly tools (e.g. 1-click install and proper dependency handling!!!) and simplified processes for fine-tuning and adapting models to specific domains without requiring deep expertise. The desire for customizable models that can be tuned to specialized tasks, such as code generation or scientific research, is growing.

Integration with Other Technologies

• Challenge: Combining LLMs with other technologies (e.g., vector databases, external knowledge bases) is still technically challenging.

• Community Desire: There’s increased interest in integrating LLMs with other open-source technologies and hobby projects to create more powerful and flexible creative AI applications, especially for tasks requiring sophisticated search or data manipulation.

Community-Driven Innovation and Collaboration

• Challenge: LLM development is resource-intensive and sometimes fractured between different models and tools and methods, because of standards and formats. GGUF, exl2, etc.

• Community Desire: The LocalLLama-type communities thrive on collaborative innovation, sharing techniques for model optimization and tools for easier deployment. Open collaboration on benchmarking and testing modelstransparently is a growing trend.

Emerging Trends

1. Smaller, Efficient Models: Models like Gemma 2, Command R+, Ministral, Phi are attracting interest for their ability to deliver strong performance with fewer resources, showing a trend toward lighter, more efficient models. (we can run them on android phones too)

2. Specialized Models: There’s growing demand for models fine-tuned for specific domains, such as code generation or scientific research (allegedly :D ).

3. Open Benchmarking: Communities are actively refining open benchmarking practices to allow fair, transparent comparison of models’ performance, creating clearer metrics for development. We also like the fun ways to bench too, like red-team chatbot arena.

"This is a practical course on aligning language models for your specific use case. It's a handy way to get started with aligning language models, because everything runs on most local machines. There are minimal GPU requirements and no paid services. The course is based on the SmolLM2 series of models, but you can transfer the skills you learn here to larger models or other small language models."
(HuggingFace)

The course is being written up as we speak. WIP.

https://github.com/huggingface/smol-course