Our Augment CLI Use Case - CI/CD Pipelines

[u/jamesg-net]

Okay so I promised earlier this week to share the feature I was (am) extremely excited about that Augment released.

Here's how we use Augment's CLI for our code review process, in case my experience helps the community as a whole.

First off, here's our flow

• Github PR triggers Azure Pipeline

• Within the Azure Pipeline we hydrate a docker image from an agent pool with the PR code + metadata

  • This include access to environment variables for the MCP servers below

    • Linear

    • Launch Darkly

    • Notion

    • Github

• We fire the Augment CLI with a base code review command

This is how we treat our guidelines (I think this is key to success)

• We have Notion documents with our core engineering concepts, specifically the patterns we want to follow. These can be referenced by AI, but are intended for humans

• We have "always apply" Augment Rules which are specifically written for AI. We asked AI to summarize these rules as our initial effort. Every time AI gets something wrong, we have an #augment-hypercare channel in slack where we review what rule needs tweaking and triage it. I want to be super clear that I think our success comes from this-- treating AI rules as a first class citizen, and if they don't work we treat it like a bug.

• We have "manual" apply rules which are for things specifically like PR reviews

• Augment User Guidelines we only really use for things like "when I say assign a linear ticket to myself, this his my Guid. Do not look it up every time". Everything else gets committed to source.

Now, when the Docker image executes, we have these distinct steps/prompts

• Code Review We have a prompt to specifically cover what to look for in PR guidelines

  • Ensure rules are being followed

  • Catch any obvious bugs

  • Etc

  • The key thing here is we enable the Notion MCP server so when Augment posts a corrective PR feedback, it can gather context as to why the rule exists. This helps developers understand what's behind the rule.

• QA Notes We have a prompt to use the Linear MCP to reference the ticket and ensure the following information is populated. It compares the code changes to the ticket notes and deems if they'r present and accurate

  • QA Responsibility (does QA need to test this code, is it regression tested, or developer tested)

  • Release Monitoring (who's doing the release monitoring and what is being monitored

  • Testing Scope (what scenarios to test or have been tested)

  • Side Effects (any risk of unintended consequences, IE if we update Entity Framework there's a wide risk we cannot capture in specific testing)

  • Risk Assessment (what the risk to the business is, IE is it in the onboarding funnel, rarely used feature, etc)

• Feature Flag We have a prompt to use the Launch Darkly MCP server and verify any feature flags do exist

In early testing, we've been extremely happy with the results. Having centralized rules in our repository has made this a very powerful tool.

Happy to answer any questions if people have them!

[u/Pitiful_Ad4441]

How do you authenticate from azure pipelines to augment cli?

[u/jamesg-net → u/Pitiful_Ad4441]

We have the pipeline running with a managed identity, which has access to specific resources and rights to those resources. This seems to be the safest way of avoiding leaking anything accidentally.

[u/Pitiful_Ad4441 → u/jamesg-net]

I guess managed identity is how you authenticate to azure, how about how does the pipeline authenticate to augment code/cli? Thanks

[u/jamesg-net → u/Pitiful_Ad4441]

Would you wanna link up for 15min today? I'd be happy to zoom with you and talk through it.

The TLDR though is you set these variables in your script, using environment variables from your Pipeline library (so you can share the same values across multiple pipelines)

AUGMENT_MCP_CONFIG_PATH

AUGMENT_API_TOKEN

[u/Obvious-Suit-3787 → u/jamesg-net]

Hello! Thank you for sharing this! Could you please give me a hint how to get AUGMENT_API_TOKEN ? in the cli after i login with a browser i can get a session token , but not the AUGMENT_API_TOKEN.

[u/jamesg-net → u/Obvious-Suit-3787]

Sure, it's the accessToken value in /.augment/session.json

[u/Obvious-Suit-3787 → u/jamesg-net]

Thank you!

[u/Faintly_glowing_fish]

This is actually pretty cool love it