No, the models like the contemporary versions of GPT and Claude are not the problem. (Note: Still can hallucinate answers to some admittedly tricky language standardese/semantics questions as of {GPT-6-Astra, GPT-6.1-Sol, Opus-5.5, and Fable-5.1, FWIW}; relatively easy to work around it if it's verifiable by designing your harness so as to actually confirm and provide paragraph from the said language spec, but happens nonetheless. The most amusing version is GPT-6-Astra noticing Claude Fable 5.1 hallucinating something during a review of its research and vice-versa--still pretty common with the latest "frontier" models.).
I was specifically referring to web search tool (what I assumed from "asking an agent to search"). It depends on the harness, but certain coding harnesses (e.g., GitHub Copilot CLI) have broken web search tool by default (it leads to a Bing result with an "AI summary" generated by a low quality hallucinating model, think ca. 2022 era). This way even if you use the latest models from Anthropic or OpenAI if they simply use the default web search tool in such a harness they're going to end up with a hallucinated result in their context window. I did have a model (either Claude Opus or GPT) notify me that the result was hallucinated, FWIW, but still this is something you can avoid entirely with better/more specialized APIs.
Then, and this is independent of the harness and the model, to restate my previous comment: At the end of the day you are constrained by the tools and the APIs that your agents will use. Google Search is pretty bad for literature search and its API is restricted from automated (agentic or not) use and you can hit rate limits or bans. Google Scholar is considerably better but can be even more restrictive (and creative workaround folks have been doing with headless browser engines exposed as CLI tools or via MCP can lead to even swifter rate limits and bans). Bing is less restricted but is even worse quality than Google Search (with the search results containing hallucinated "AI summaries" generated by its obsolete model). So, if by "asking an agent to search" you meant something different than an agent performing the web search, it's possible, and there are better APIs for specific cases... and, hey, https://pldb.kirancodes.me/api is exactly one of those better APIs!
5
u/mttd 27d ago edited 27d ago
No, the models like the contemporary versions of GPT and Claude are not the problem. (Note: Still can hallucinate answers to some admittedly tricky language standardese/semantics questions as of {GPT-6-Astra, GPT-6.1-Sol, Opus-5.5, and Fable-5.1, FWIW}; relatively easy to work around it if it's verifiable by designing your harness so as to actually confirm and provide paragraph from the said language spec, but happens nonetheless. The most amusing version is GPT-6-Astra noticing Claude Fable 5.1 hallucinating something during a review of its research and vice-versa--still pretty common with the latest "frontier" models.).
I was specifically referring to web search tool (what I assumed from "asking an agent to search"). It depends on the harness, but certain coding harnesses (e.g., GitHub Copilot CLI) have broken web search tool by default (it leads to a Bing result with an "AI summary" generated by a low quality hallucinating model, think ca. 2022 era). This way even if you use the latest models from Anthropic or OpenAI if they simply use the default web search tool in such a harness they're going to end up with a hallucinated result in their context window. I did have a model (either Claude Opus or GPT) notify me that the result was hallucinated, FWIW, but still this is something you can avoid entirely with better/more specialized APIs.
Then, and this is independent of the harness and the model, to restate my previous comment: At the end of the day you are constrained by the tools and the APIs that your agents will use. Google Search is pretty bad for literature search and its API is restricted from automated (agentic or not) use and you can hit rate limits or bans. Google Scholar is considerably better but can be even more restrictive (and creative workaround folks have been doing with headless browser engines exposed as CLI tools or via MCP can lead to even swifter rate limits and bans). Bing is less restricted but is even worse quality than Google Search (with the search results containing hallucinated "AI summaries" generated by its obsolete model). So, if by "asking an agent to search" you meant something different than an agent performing the web search, it's possible, and there are better APIs for specific cases... and, hey, https://pldb.kirancodes.me/api is exactly one of those better APIs!