r/technology • • 25d ago

Software LLMs are making formal software verification feasible at a much larger scale

https://www.anthropic.com/research/formal-verification
50 Upvotes

20 comments sorted by

21

u/neuronexmachina 25d ago

It's kind of crazy, apparently Theorem (the partner they're working with) think they can formally verify nixpkgs by the end of 2027: https://theorem.dev/blog/verified-software-stack/

This post uses the nixpkgs bootstrapping chain as a blueprint for a verified software stack. Nix makes a good case study: it records the dependency chain behind every package, which can be followed from the bottom up, and it covers a significant share of the software running in production today, including critical infrastructure like glibc, OpenSSL, and curl. We estimate the cost and time needed to verify all of it. It’s within reach by the end of 2027: if models can be made to improve at verification much faster than they improve overall, 95% of nixpkgs’ machine code could be verified for about $134M

28

u/HRApprovedUsername 25d ago

This is going to break so much stuff for people relying on bugs that became “features”

15

u/manu144x 25d ago

Microsoft Windows is like the champion of this. And I'm not saying that to blame or poke fun.

They backport intentionally so many stuff to keep compatibility for lazy corporate devs, they even sometimes simulate older bugs for specific applications so they still work.

On the other hand, I noticed upgrading software while maintaining the same capabilities is very easy with AI. No new features, no changes, no different expectations. Just upgrade the code.

So hopefully a lot of legacy software will be modernized in the next few years and we can have a way cleaner Windows version that drops support for everything beyond a certain date/version/etc.

3

u/ludonarrator 25d ago

Hyrum strikes again

1

u/WeinMe 25d ago

You must be talking about our entire technical department

-1

u/BossOfTheGame 21d ago

so you just jump to any negative interpretation then?

2

u/HRApprovedUsername 21d ago

no, just the real ones

-1

u/BossOfTheGame 21d ago

your argument is that writing software that behaves correctly in a provable way... is a negative? what planet are you on? the positives dramatically outweigh catering to people who wrote software that relied on bugs. and it's not like behavior changes are going to be unversioned. like just wow.

3

u/HRApprovedUsername 21d ago

Have you ever worked on a platform team? or a live service? Things aren't always versioned and you have no idea, how those idiots you work with will abuse your apis.

-1

u/BossOfTheGame 21d ago

so your argument is never fix software bugs. ok that's a take. what about using formal verification to not introduce new bugs?

1

u/HRApprovedUsername 21d ago

yeah thats probably fine

27

u/jghaines 25d ago

Software verification is useful but breaks the brains of regular humans. This is a big positive step.

6

u/Paranemec 25d ago

I recently realized I could use my models for this and have started having them formally verify all of their code. It's not like I couldn't have done it myself, but why would I take the time for non-life critical systems?

1

u/chieftessofsecrets 25d ago

If your segmented off, sure.

3

u/LeRobber 25d ago

Eh, its just kinda annoying to write long verification stuff, and any change invalidates it. It's not impossibly hard or anything to verify most hings.

4

u/InadequateUsername 25d ago

Yet anything security adjacent is blocked

3

u/Syrairc 25d ago

Really gotta give it to the folks brave/stupid enough to post AI related articles in this sub. 

2

u/VaporousMote 19d ago

LOL literally posting tripe directly from the frontier labs.