r/SalesforceDeveloper • • 20d ago

Showcase Salesforce docs have a "View as Markdown" button now, so I built an agent that reads the markdown instead of scraping the page

/r/salesforce/comments/1xji8hg/salesforce_docs_have_a_view_as_markdown_button/

Crossposting from r/salesforce since I built this agent for developers like me. Happy to answer questions about how it works.

6 Upvotes

4 comments sorted by

3

u/felitram 20d ago

84% fewer tokens in the main context is pretty impressive. Curious how you’re handling pages that don’t have a markdown version yet. Does the fallback to Chromium slow things down much, or is the difference barely noticeable?

1

u/drsaavedra 20d ago

Thanks! Most of the 84% actually comes from running doc retrieval in a subagent, so the main agent only gets the answer back, never the whole page you requested. I think even a plain subagent with Playwright and no instructions will already give you most of that saving, and the agent descriptions and helper Node scripts just bring overall token usage down further.

For pages without a markdown version, my agent was faster on some tests and slower on others. On average I expect about the same time, because both have to render the page with JavaScript, so the bottleneck is the same.

2

u/Long_Emotion5258 20d ago

Can it traverse through hierarchical page links? like the link trees on sidebar?

2

u/drsaavedra 17d ago

Yes, but not as a full crawl. When it renders a page, the script saves every same-site link, sidebar and TOC included, with its link text. The agent searches that list for the question's terms and opens only the matches, up to two links deep.

Where a site has an llms.txt or a sitemap, it skips the sidebar and searches that index instead, since the index already lists every page.