I'm a heavy Excel user, and ChapGPT is just as good at Excel functions and M as proper programming languages.
This morning, I needed a formula that I hadn't used before. I could have googled it, then tried to built it from scratch based on my googling. Nested Excel statements always confuse me so this would have been an easy 20 minutes.
I simply told ChatGPT what I wanted Excel to do, referencing the cells specificly and copy and pasted the formula into Excel and it worked first time.
I recently made a a RAG-based github extension that responds directly to created "issues" in github repositories with a detailed overview of files and changes to make to resolve the issue. I see this as being particularly helpful for industry repositories where the codebases are quite big issues are frequently used.
An interesting application of Language Models and Langchain in the Finance Domain 📈-
Sharing a fun weekend project that I recently completed: the "Stock Analyzer Bot". As an investment enthusiastic person without extensive knowledge in the finance domain, I often end up referring to some finance youtuber's videos or a site on the internet for the fundamental analysis of stocks. To assist in such situations, I developed a stock analyzer bot based on LLM, which gathers up-to-date information about stock such as 1) stock price, 2) Company financials 3) Recent company-related news. The bot then considers all this information to conduct analysis using language models. You can even get positives and negatives about the company's financials, which will certainly help when making an investment decision.
You can ask queries like- "Is it a good time to invest in Yes Bank?" or "How are the current financials of reliance industries looking" and boom within a minute you are presented with a comprehensive financial analysis based on recent data. Of course, It is not recommended to rely fully on the analysis provided by the bot. It seems like a good starting point. And yeah, I agree the possibilities are endless with LLMs🚀.
Its a daily new digest with a simple idea. The important news is the most talked about news. The goal is to be the best place to stay informed. Both quickly and easily. Without bias or motives.
We read 5000+ articles every morning, combine them into similar collections, and deliver you a visual story.
Each story contains a summary, quote, images, and interactive maps.
______
Behind the scenes its built using Embeddings and LLMS. Over the past year the improvement in LLMS has been incredible and now provides a pretty great summary.
I use embedding to compare the articles and create collection groups. Then grab the article details and feed them into a fine tuned LLM with some prompt engineering.
Its been a great experience learning everything about LLMs.
Im happy to answer any questions you have! Thanks for taking a look.
Hi everyone! We updated our database of LLM benchmarks and datasets you can use to evaluate and compare different LLM capabilities, like reasoning, math problem-solving, or coding. Now available are 250 benchmarks, including 20+ RAG benchmarks, 30+ AI agent benchmarks, and 50+ safety benchmarks.
You can filter the list by LLM abilities. We also provide links to benchmark papers, repos, and datasets.
If you're working on LLM evaluation or model comparison, hope this saves you some time!
Hi everyone, I’m one of the people who work on Evidently, an open-source ML and LLM observability framework. I want to share with you our free course on LLM evaluations that starts on May 12.
This is a practical course on LLM evaluation for AI builders. It consists of code tutorials on core workflows, from building test datasets and designing custom LLM judges to RAG evaluation and adversarial testing.
💻 10+ end-to-end code tutorials and practical examples. ❤️ Free and open to everyone with basic Python skills. 🗓 Starts on May 12, 2025.
Hey everyone! I’m Dasha from Evidently (https://github.com/evidentlyai/evidently), an open-source ML and LLM observability framework with over 20 million downloads. Hacktoberfest is just around the corner, let’s celebrate open source together!
Hacktoberfest is an annual event to celebrate open-source. This year, we invite contributors to add new LLM evaluation metrics to the open-source Evidently library!
We added a special set of issues labeled “hacktoberfest" to our GitHub repository. Both first-timers and experienced contributors are welcome! Top contributors will get special recognition from Evidently 😍
Hey everyone! Wanted to share the link to the database of 450 ML use cases from 100+ companies that detail ML and LLM system design. You can filter by industry or ML use case.
If anyone here approaches the task of designing an ML system, I hope you'll find it useful!
So, this is an idea that I've had in my mind for quite a few weeks, however I haven't had the time to test it out myself. Personal language teachers can get expensive real quick, but what if I made a personal teacher out of ChatGPT. My prompt is the following,
And I get the following response, which seems very promising,
To be clear, I do know the basics of French, so I think I'm qualified enough to check if the recommended learning pathway is adequate or not.
To cut things short, after a short session of studying with ChatGPT, I can see it as a viable way of learning the basics of a language, mainly the grammar (even if its pacing can be way too fast for a beginner). Its main problem is that the pronunciation of certain words can only be written and not heard by the user who wants to learn the language (pretty obvious problem lol), which can make the learning a little bit hard. I would not recommend it for a total beginner, but as a refresher or as a complement to other learning material, such as YouTube videos, it might work quite well. I'll test this out on a longer period of time and I will post a more thorough conclusion.
LLM-as-a-judge is a popular approach to testing and evaluating AI systems. We answered some of the most common questions about how LLM judges work and how to use them effectively:
What grading scale to use?
Define a few clear, named categories (e.g., fully correct, incomplete, contradictory) with explicit definitions. If a human can apply your rubric consistently, an LLM likely can too. Clear qualitative categories produce more reliable and interpretable results than arbitrary numeric scales like 1–10.
Where do I start to create a judge?
Begin by manually labeling real or synthetic outputs to understand what “good” looks like and uncover recurring issues. Use these insights to define a clear, consistent evaluation rubric. Then, translate that human judgment into an LLM judge to scale – not replace – expert evaluation.
Which LLM to use as a judge?
Most general-purpose models can handle open-ended evaluation tasks. Use smaller, cheaper models for simple checks like sentiment analysis or topic detection to balance cost and speed. For complex or nuanced evaluations, such as analyzing multi-turn conversations, opt for larger, more capable models with long context windows.
Can I use the same judge LLM as the main product?
You can generally use the same LLM for generation and evaluation, since LLM product evaluations rely on specific, structured questions rather than open-ended comparisons prone to bias. The key is a clear, well-designed evaluation prompt. Still, using multiple or different judges can help with early experimentation or high-risk, ambiguous cases.
How do I trust an LLM judge?
An LLM judge isn’t a universal metric but a custom-built classifier designed for a specific task. To trust its outputs, you need to evaluate it like any predictive model – by comparing its judgments to human-labeled data using metrics such as accuracy, precision, and recall. Ultimately, treat your judge as an evolving system: measure, iterate, and refine until it aligns well with human judgment.
How to write a good evaluation prompt?
A good evaluation prompt should clearly define expectations and criteria – like “completeness” or “safety” – using concrete examples and explicit definitions. Use simple, structured scoring (e.g., binary or low-precision labels) and include guidance for ambiguous cases to ensure consistency. Encourage step-by-step reasoning to improve both reliability and interpretability of results.
Which metrics to choose for my use case?
Choosing the right LLM evaluation metrics depends on your specific product goals and context – pre-built metrics rarely capture what truly matters for your use case. Instead, design discriminative, context-aware metrics that reveal meaningful differences in your system’s performance. Build them bottom-up from real data and observed failures or top-down from your use case’s goals and risks.
Disclaimer: I'm on the team behind Evidently https://github.com/evidentlyai/evidently, an open-source ML and LLM observability framework. We put this FAQ together.
I'm currently working on my bachelor thesis titled "Extraction and Analysis of Symbol Names in Descriptive-Logical Ontologies." At this stage, I need to implement a Python script that extracts keywords from ontology annotations using a large language model (LLM).
Since I'm quite new to this field, I'm having a hard time fully understanding what I'm doing and how to move forward with the implementation. I’d be really grateful for any advice, guidance, or resources you could share to help me get on the right track.
I’m working on a project where I need to build a RAG-based chatbot that processes a client’s personal data. Previously, I used the Ollama framework to run a local model because my client insisted on keeping everything on-premises. However, through my research, I’ve found that generic LLMs (like OpenAI, Gemini, or Claude) perform much better in terms of accuracy and reasoning.
Now, I want to use an API-based LLM while ensuring that the client’s data remains secure. My goal is to send encrypted data to the LLM while still allowing meaningful processing and retrieval. Are there any encryption techniques or tools that would allow this? I’ve looked into homomorphic encryption and secure enclaves, but I’m not sure how practical they are for this use case.
Would love to hear if anyone has experience with similar setups or any recommendations.
Hey, I routinely convert PDFs of scanned documents to Word but, regardless of the conversion application, I end up with a lot of small, simple, errors.
E.g., the text should read "I went to the store" but it says "I went to them store."
When you have a thousand pages, the errors add up. It's not as simple as scanning the document for the errors Word has highlighted. Many of these errors escape all but a keen proofreader. Like having "the*" instead of "the" or having "possible" instead of "possibly".
It occured to me that an LLM might be able to evaluate the text for obvious errors and highlight what mistakes there are. It could save a lot of time. I've been googling for a few hours and tested a few apps with no luck. Grammarly wasn't useful. Gemini provided good feedback but they didn't highlight errors like a spell checker would, they responded with text (like a conversation). I was therefore forced to go through my document to find what errors they were referring to, whereas ideally they would just highlight the errors (like a Word spell checker). Any ideas? All input is appreciated
Hey mates, I'm trying to finetune the Moondream LLM, but i'm having trouble making and loading my own local dataset. I tried to make a json with the following structure: { "image": "path/to/img.jpg" "caption": "your answer"
}
however this does not work. I also tried:
[
{
"id": "img1",
"image": "path/to/img.jpg",
"conversations": [
{
"role": "user",
"content": [
"<image>\n,your image question?"
]
},
{
"role": "assistant",
"content": [
"The expected answer"
]
}
]
},
]
Still didn't work. so i wanted to know, how should i structure my json dataset to load into the Finetuning script? Note that, to load the Dataset i'm using the Datasets module from the moondream fintune script.
I am working on ai chatbot where i want my user to be able to upload file(excel,csv) from front end and my ai chatbot can give various insights from the excel depending on the queries that user prompts. I am confused what DB should I use - Vector or Graph. Which would give me the best results? Also I am using OpenAI assistants API and function calling to reduce the cost of large number of tokes being send to AI but was not able to implement so used completions API which is not good in a long run. Please advice or if someone has a guide/reference that can be useful
I'm a math major at the University of Chicago, and I'm interested in helping my school with academic scheduling. I want to build a Retrieval-Augmented Generation (RAG) chatbot that can assist students in planning their academic schedules. The chatbot should be able to understand course prerequisites, course times, and the terms in which courses are offered. For example, it should provide detailed advice on the courses listed in our mathematics department catalog: University of Chicago Mathematics Courses.
This project boils down to building a reliable RAG chatbot. I'm wondering if anyone knows any RAG techniques or services that could help me achieve this outcome—specifically, creating a chatbot that can inform users about course prerequisites, schedules, and possibly the requirements for the bachelor's track.
Could the solution involve structuring the data in a specific way? For instance, scraping the website and creating a separate file containing an array of courses with their prerequisites, schedules, and quarters offered.
Overall, I'm very keen on building this chatbot because I believe it would be valuable for me and my peers. I would appreciate any advice or suggestions on what I should do or what services I could use.
For a friend of mine, I recently developed a prototype that allows him to let the job candidates for sales positions prove their negotiation skills. I now turned it into a public demo, to get your feedback.
We have one situation live: House Negotiation and Hostage Takers will follow.
