I think the solution is a bunch of private trusted benchmarks, and averaging the... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

jvuygbbkuurx 65 days ago | parent | context | favorite | on: SWE-bench Verified no longer measures frontier cod...

I think the solution is a bunch of private trusted benchmarks, and averaging their announced results.

zephen 65 days ago [–]

> averaging their announced results.

Obligatory XKCD: https://xkcd.com/937/

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact