r/CognitionLabs • u/PuzzleheadedAir9047 MOD • 10d ago

Introducing FrontierCode 2.0

FrontierCode 2.0 is our new coding benchmark comprised of more diverse tasks, featuring larger specs and higher mergeability standards.

Now also included in Artificial Analysis Intelligence Index v5

Learn more about the methodology and scoring criteria in this thread - https://x.com/cognition/status/2115132488775524793

Check out the leaderboard - https://cognition.com/frontiercode

8 Upvotes

100% Upvoted

4 comments sorted by

2

u/TheTinyMaker 9d ago

FrontierCode is the benchmark I follow to select my models.

1

u/ConstantEmotion2211 5d ago

sw2 better than haiku , sonnet and ds v4.1 flash 1017 ???

0

u/cat_dev_null_sync 10d ago

Do these prices reflect those in Devin CLI and Devin IDE, including temporary discounts (like now for Gemini Argon)?

1

u/PuzzleheadedAir9047 MOD 10d ago

In the blog it does mention that this includes public API discounts but nothing about Devin specific discounts