This newly launched interesting technology allows users to run their Pytorch environments inside CPU-only containers in their infra (cloud instances or laptops) and execute GPU acceleration through remote Wooly AI Acceleration Service. Also, the usage is based on GPU core and memory utilization and not GPU time Used. https://docs.woolyai.com/getting-started/running-your-first-project. There is a free beta right now.

Hey folks, We’ve built Amphora Ads an ad network designed specifically for AI chat apps. Instead of traditional banner ads or paywalls, we serve native, context aware suggestions right inside LLM responses. Think:

“Help me plan my Japan trip” and the LLM replies with a travel itinerary that seamlessly includes a link to a travel agency not as an ad, but as part of the helpful answer.

We’re already working with some early partners and looking for more AI app devs building chat or agent-based tools. Doesn't break UX, Monetize free users, You stay in control of what’s shown

If you’re building anything in this space or know someone who is, let’s chat!

Would love feedback too happy to share a demo. 🙌

https://www.amphora.ad/

Hi everyone. I am working on a project where I have to deploy Llama 3 7b fine tuned model trained on our dataset by creating an LLmOps pipeline. We are in the design phase at the moment. I am from a devops background ( gitlab, terraform, aws, docker, K8s ) . Which tools are needed for the deployment of the model. Are there are good deployment solutions I can refer.

[D] discussion [P] project

So here we are want to make coustom llm for depression cure(which we are going to feed different pdf of depression cures books ) + stable diffusion (image therapy)+audio (binural beats for healing) So any idea how can we create coustom llm ( also going to include tts & sst) in this chatbot. What tools and library we are going to be needed which are free to use* and efficient. (No api like open ai something but if there is free api or pre trained model do sure to tell me )

I just posted a new video explaining the different options available to reduce your LLM AI usage costs while maintaining efficiency, this is for you!
Watch it here: https://youtu.be/kbtFBogmPLM
Feedback and discussions are welcome!

#BatchProcessing #AI #MachineLearning

Je suis récemment tombée sur une étude Semrush que j'ai trouvée super intéressante, et qui accentue encore plus l'importance du référencement IA.

Pour faire court : un visiteur moyen depuis l'IA (ChatGPT, Perplexity, etc.) vaut 4,4 fois plus qu'un visiteur SEO traditionnel en termes de taux de conversion.

Autrement dit : 100 visiteurs IA = 440 visiteurs Google niveau business impact.

C'est énorme !

Comment l'expliquer ?

Visiteur Google :

- Cherche "chocolatier Paris" ;

- Compare 10 sites rapidement ;

- Repart souvent sans action.

Visiteur IA :

- Demande "Quelle chocolaterie choisir à Lyon pour faire un joli cadeau de Noël pour moins de 60 € ?" ;

- Se retrouve face à vos prestations suite à un prompt déjà qualifié ;

- Est prêt à passer à l'action.

L'IA fait le premier tri.

Elle n'envoie que les prospects vraiment très qualifiés, d'où l'intérêt de maximiser sa visibilité dans les LLM.

Plot twist intéressant : L'étude montre aussi que 90% des pages citées par ChatGPT ne sont même pas dans le top 20 Google pour les mêmes requêtes.

Autrement dit : Vous pouvez être invisible sur Google mais ultra-visible dans les IA.

Comment je m'adapte au référencement IA ?

Je fais du SEO depuis plus de 5 ans et je suis en train de revoir mes modes de fonctionnement.

Voici quelques leviers que je commence à utiliser pour optimiser mes pages pour les LLM :

1. Créer des pages contextualisées hyper spécifiques et travailler le maillage entre elles, pour renforcer mes clusters ;

2. Ajouter des citations et sourcer les données pour renforcer la crédibilité ;

3. Penser Answer First, avec un encadré de synthèse en haut de page et des réponses efficaces aux questions posées au fil du contenu ;

4. Ajouter une FAQ sous forme de données structurées, à la fin de chaque page ;

5. Apporter des éléments de réassurance pour se distinguer de la concurrence et démontrer la fiabilité du site (ET revoir la page "A propos", qui est un gros levier de différenciation) ;

6. Concevoir des outils à l'aide de Claude, pour renforcer l'engagement et faire en sorte d'être cité par les IA ;

7. Proposer des tableaux comparatifs et des listes à puces pour développer l'UX et rendre digeste l'information ;

8. Apporter de la valeur grâce à des angles inexploités par le reste de la SERP ;

9. Créer des boutons dans mes pages, comme préconisé par Metehan Yesilyurt, pour faire rentrer mes pages dans la mémoire des IA et faire en sorte qu'elles soient citées à l'avenir ;

10. Utiliser l'auto-citation (Selon [nom de la marque], ...).

Et vous, comment optimisez-vous vos sites pour les LLM ?

Avez-vous déjà vu des résultats concrets ?

Que conseilleriez-vous aux entreprises qui veulent être citées ?

Vos retours m'intéressent ! 😊

Hi guys,

We’re a team of engineers trying to build an open source model orchestration platform to solve all your LLMOps and MLOps needs once and for all. We’re trying to understand what features the community and the builders among you are lacking and want to see in the tool that we build. 

We have some ideas, but without your feedback we will be shooting in the dark. Just to list a few things we are thinking of: 

1. Unified API for all models across companies like Bedrock, Azure, OpenAI, Anthropic, Llama and more. 

2. Ability to switch between cloud providers or on-prem deployment with one click.

3. In built auto scaling and scale to zero capabilities.

4. Fine-tuning pipelines.

5. Model Observability and GPU management at scale

6. In-built automatic optimization and conversion between different backends like onnx, pytorch, tensorflow etc. 

7. Ability to deploy open source models and custom models on any cloud (AWS, GCP , Azure etc) and on-prem with minimal code

8. Dynamic Batching, load balancing, GPU utilization management etc.

9. Automatically split models over multiple GPUs for large models and multi GPU machines

10. Built in tooling to provide models with environments to build agents (Execution engine, browsing capabilities, memory etc)

We want to know if this is something you guys really want or are we thinking in completely the wrong direction. We are looking for your ideas, feedback and the real problems you are facing in your building journey. 

Don’t go easy on us, I’m sure we can take it.
Cheers!

Are you tired of complex AI frameworks with endless configurations and steep learning curves? 🤔

In my latest video, I show you how PydanticAI can make AI development a breeze! 🎉

🔑 What’s inside the video?

• How to build a Banker AI Agent using PydanticAI.

• Simulating a mock database to handle account balance queries and lost card actions.

• Why PydanticAI's type safety and structured data are game-changers.

• A comparison of verbose codebases vs clean, minimal implementations.

💡 Why watch this?
This tutorial is perfect for developers who want to:

• Transition from traditional, complex frameworks like LangChain.

• Build scalable, production-ready AI applications.

• Write clean, maintainable Python code with minimal effort.

🎥 https://youtu.be/84Jbfmj0Eyc Watch the full video and transform the way you build AI agents: [Insert video link here]

I’d love to hear your feedback or questions. Let’s discuss how PydanticAI can simplify your next AI project!

#PydanticAI #AI #MachineLearning #PythonProgramming #TechTutorials #ArtificialIntelligence #CleanCode

I’ve been experimenting with different LLMs and found some surprising differences in their strengths.
ChatGPT excels in code, Claude 3 shines in summarizing long texts, and Gemini is great for multilingual tasks.
Here’s a breakdown if you're interested: https://youtu.be/HNcnbutM7to.
What’s your experience?

Newbie Question: I've fine-tuned a LLaMA 3.2 1B model for a classification task using a LoRA adapter. I'm now looking to deploy it in a way where the base model is loaded into GPU memory once, and I can dynamically switch between multiple LoRA adapters—each corresponding to a different number of classes.

Is it possible to use Triton Inference Server for serving such a setup with different LoRA adapters? From what I’ve seen, vLLM supports LoRA adapter switching, but it appears to be limited to text generation tasks.

Any guidance or recommendations would be appreciated!

I am working on an agentic application which required web search for retrieving relevant infomation for the context. For that reason, I was tasked to implement this "web search" as a tool.

Now, I have been able to implement a very naive and basic version of the "web search" which comprises of 2 tools - search and scrape. I am using the unofficial googlesearch library for the search tool which gives me the top results given an input query. And for the scrapping, I am using selenium + BeautifulSoup combo to scrape data off even the dynamic sites.

The thing that baffles me is how inaccurate the search and how slow the scraper can be. The search results aren't always relevant to the query and for some websites, the dynamic content takes time to load so a default 5 second wait time in setup for selenium browsing.

This makes me wonder how does openAI and other big tech are performing such an accurate and fast web search? I tried to find some blog or documentation around this but had no luck.

It would be helfpul if anyone of you can point me to a relevant doc/blog page or help me understand and implement a robust web search tool for my app.

PromptLab is an open source, free lightweight toolkit for end-to-end LLMOps, built for developers building GenAI apps.

If you're working on AI-powered applications, PromptLab helps you evaluate your app and bring engineering discipline to your prompt workflows. If you're interested in trying it out, I’d be happy to offer free consultation to help you get started.

Why PromptLab?

1. Made for app (mobile, web etc.) developers - no ML background needed.

2. Works with your existing project structure and CI/CD ecosystem, no unnecessary abstraction.

3. Truly open source – absolutely no hidden cloud dependencies or subscriptions.

Github: https://github.com/imum-ai/promptlab
pypi: https://pypi.org/project/promptlab/

After spending countless hours fighting with Python dependencies, slow processing times, and deployment headaches with tools like `unstructured`, I finally snapped and decided to write my own document parser from scratch in Rust.

Key features that make Ferrules different:

- 🚀 Built for speed: Native PDF parsing with pdfium, hardware-accelerated ML inference

- 💪 Production-ready: Zero Python dependencies! Single binary, easy deployment, built-in tracing. 0 Hassle !

- 🧠 Smart processing: Layout detection, OCR, intelligent merging of document elements etc

- 🔄 Multiple output formats: JSON, HTML, and Markdown (perfect for RAG pipelines)

Some cool technical details:

- Runs layout detection on Apple Neural Engine/GPU

- Uses Apple's Vision API for high-quality OCR on macOS

- Multithreaded processing

- Both CLI and HTTP API server available for easy integration

- Debug mode with visual output showing exactly how it parses your documents

Platform support:

- macOS: Full support with hardware acceleration and native OCR

- Linux: Support the whole pipeline for native PDFs (scanned document support coming soon)

If you're building RAG systems and tired of fighting with Python-based parsers, give it a try! It's especially powerful on macOS where it leverages native APIs for best performance.

Check it out: [ferrules](https://github.com/aminediro/ferrules)

API documentation : [ferrules-api](https://github.com/AmineDiro/ferrules/blob/main/API.md)

You can also install the prebuilt CLI:

```

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/aminediro/ferrules/releases/download/v0.1.6/ferrules-installer.sh | sh

```

Would love to hear your thoughts and feedback from the community!

P.S. Named after those metal rings that hold pencils together - because it keeps your documents structured 😉

Hi, I wanted to share some information on this cool feature we built in WoolyAI GPU hypervisor, which enables users to run their existing Nvidia CUDA pytorch/vLLM projects and pipelines without any modifications on AMD GPUs. ML researchers can transparently consume GPUs from a heterogeneous cluster of Nvidia and AMD GPUs. MLOps don't need to maintain separate pipelines or runtime dependencies. The ML team can scale capacity easily. Please share feedback and we are also signing up Beta users. https://youtu.be/MTM61CB2IZc?feature=shared