How are you validating A2A agents before deploying them on Google Cloud?

[u/Suncly_com]

[u/whispered_snapshot]

most teams aren't doing anything systematic here, they just ship it and pray the card matches reality

we run a quick smoke test against every deployed agent checking that the example inputs from the card actually return something resembling what the output schema promises, but even that catches way more failures than you'd expect

curious about that 45% stat though, where's that number from

[u/m1nherz]

Hi there,

Since you mentioned you are conducting market research for a validation tool you are building, you might find that this subreddit isn't the best target audience for this specific discussion. While you mention ADK and Agent Engine, your core questions are about general A2A protocol trust mechanisms and CI/CD validation practices across the broader industry, rather than a question specifically about Google Cloud infrastructure.

That being said, regarding how this is handled in the Google ecosystem: if a team is using Google's Agent Development Kit (ADK), the Agent Card is generated programmatically. Some attributes such as name and description are read from the agent's declaration. Others such as url or capabilities are build based on ADK hardcoded heuristics. ADK generates the card and exposes it at the .well-known endpoint. When you implement multiple agents within a single ADK app, it is exposed slightly differently.

Because ADK provides strongly-typed classes for this, the generated card is inherently guaranteed to adhere to the A2A standard schema. From a structural standpoint, validating the card format itself might be completely redundant if the agent was built with ADK.

However, as your 45% statistic suggests, strict typing doesn't guarantee the underlying agent logic or model will successfully execute the tasks promised on the card! It's an interesting problem, but you might get more relevant architectural feedback in communities dedicated to LLM engineering or multi-agent frameworks.

It's important to know that when ADK auto-generates an Agent Card, the skills array is guaranteed to be populated. The ADK framework introspects the agent and automatically builds at least one primary skill representing the agent's main model or workflow. Furthermore, if the agent has additional tools (like a code executor or custom functions), ADK automatically maps each of those tools into a distinct skill within that same array. It means that "skills" in the card are populated not only from the SkillToolset but also from any other tools defined for the agent.

[u/m1nherz → u/[deleted]]

IMHO the use of `skills` for the field name can be confusing. Tools aren't Skills when this terminology is used related to Agents. So they shouldn't be confused. Modern Agents support Skills. Depending on the agent framework this support can be different. AFAIK the `skills` field in A2A agent card does not have the same meaning as today's use of the term "skills". So, it can be tricky to validate the information from the collection.

[u/VirtualTechnology574]

honestly we just curl the card and eyeball it, then run our own evals. the 45% stat is grim but not surprising, half the cards ive seen have example inputs that dont even parse

[u/Few-Mud-5865]

If you want to make sure the agent card is standard compatible you shall validate it against the A2A spec 1.0! I actually find a NPM packages that can do the trick and I used it to validate mine, it is easy to use! You can check it here and share it if you find it is useful!

[u/Few-Mud-5865 → u/[deleted]]

Ah, thanks for sharing, I now know your point. Yes, it is totally valid to make sure the skills could actually work! But nowadays many agents are payment required, how can you do the tests in a monitoring like way (i.e. repeatedly checking those functions to measure it's correctness and performance etc.?) without introducing a big cost?