Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.
Benchmarks is in beta. Reach out to Sphinx to activate it for your workspace. Include your workspace name. The interface and behavior may change during the beta.
Use Benchmarks to test Sphinx on questions your team cares about. You define questions and criteria for a good answer, run the benchmark, and inspect the generated answers and judge results.
Before you start
Ask Sphinx to activate Benchmarks for your workspace.
Prepare representative questions and confirmed criteria for correct answers.
Select the workspace whose knowledge you want to evaluate.
Open Benchmarks
Open Settings at the bottom left and select Benchmarks. The page contains Average score trend, Questions, and Benchmark runs.
Benchmarks in a demo account with synthetic data.
Add questions and criteria
Click Add question.
Enter the question in Question.
Enter a specific requirement in Criterion 1. Use Add criterion for additional requirements. Each criterion is scored separately, and questions have equal weight in the final score.
Click Add question to save it.
For example:
Field
Demo content
Question
How long should someone listen to a song for it to count as a finance-qualified play?
Criterion 1
The answer says at least 30 qualifying seconds.
Criterion 2
The answer distinguishes the finance definition from product engagement definitions.
Use the search field to find saved questions. Each question has controls to view, edit, or delete it.
Write questions with enough context to have a clear expected answer. If different teams use different definitions, name the team or workflow. Review criteria with the knowledge owner so an outdated expected answer does not turn a valid response into a misleading failure.
Run and inspect a benchmark
Click Run benchmark.
Follow the run under Benchmark runs. Use Status to filter runs.
Open a completed run to see Benchmark run results. You can also click a run in the average score trend.
Review the number of test cases and the Mean, Median, Lowest, and Highest scores.
For each test case, inspect the Given answer, each Criterion, and the Judge result, including the score and explanation.
Scores are displayed on a 0–100 scale. A high score means the answer satisfied the configured criteria according to the judge. Read individual results before drawing a conclusion about the knowledge base.
Evaluate generalization
Include representative questions that were not used to develop your knowledge base. Keep the questions, criteria, model, and agent setup comparable when measuring a change. For a broader agent evaluation, compare answer quality alongside cost and latency; the score trend alone does not describe those tradeoffs. See Learning organizational context.
Track changes over time
Use 1d, 7d, or 30d in Average score trend to change the displayed time range. Question-level trends help you identify regressions that an overall average can hide.When a score falls, check both the answer and the criterion. Correct missing or conflicting knowledge through ingest or controversy resolution, review the resulting changes, and rerun the benchmark. If the expected answer changed, update the criterion deliberately and account for that change when comparing runs.
{"docsConfig":{"theme":"aspen","$schema":"https://mintlify.com/docs.json","name":"Sphinx Docs","colors":{"primary":"#704643","light":"#ede2d6","dark":"#15803D"},"logo":{"light":"https://mintcdn.com/sphinx-977f1673/P__T7c22F_M_IxmY/logo/wordmark-light.svg?fit=max\u0026auto=format\u0026n=P__T7c22F_M_IxmY\u0026q=85\u0026s=f9db2de016b08aba238741c2d0e90e74","dark":"https://mintcdn.com/sphinx-977f1673/P__T7c22F_M_IxmY/logo/wordmark-dark.svg?fit=max\u0026auto=format\u0026n=P__T7c22F_M_IxmY\u0026q=85\u0026s=b7100a9af095e59e46a99f6b8507bf12"},"favicon":"https://mintcdn.com/sphinx-977f1673/1wWAviaQrCe_7dvG/logo/sphinx.svg?fit=max\u0026auto=format\u0026n=1wWAviaQrCe_7dvG\u0026q=85\u0026s=3f3d9bb107547b1e4075269ff8505a2c","navbar":{"links":[{"href":"mailto:support@sphinx.ai","label":"Support"}]},"footer":{"socials":{"x":"https://x.com/getsphinx","linkedin":"https://www.linkedin.com/company/sphinx-ml"}},"styling":{"latex":true},"navigation":{"global":{"anchors":[{"anchor":"Blog","icon":"newspaper","href":"https://sphinx.ai/blog/filters/all"}]},"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"sphinx-977f1673","hasAudioNarration":false,"actualSubdomain":"sphinx-977f1673","customDomains":["docs.sphinx.ai"],"gitSource":{"type":"github","owner":"sphinx-ml","repo":"docs","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"entitlements":{},"buildId":"6b00d88c303173b0b1900bfc","clientVersion":"0.0.4271","searchFilterCounts":[{"language":"default","tag":"Documentation","count":22},{"language":"default","tag":"Knowledge base","count":7},{"language":"default","tag":"Sphinx MCP","count":4},{"language":"default","tag":"Sphinx Private Gateway","count":4},{"language":"default","tag":"Connect Sphinx MCP to your agent","count":3},{"language":"default","tag":"How Sphinx works","count":3},{"language":"default","tag":"Python SDK","count":2},{"language":"default","tag":"Examples","count":1},{"language":"default","tag":"Sphinx Responses API","count":1}]},"assistantConfig":{"enableStarterQuestions":false},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"treatment","enabled":true,"payload":{"type":"string","value":"treatment"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[]}
{"page":{"pagePath":"benchmarks","slug":"/benchmarks","pageMetadata":{"title":"Benchmarks (beta)","description":"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.","href":"/benchmarks"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,li:`li`,ol:`ol`,p:`p`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,ul:`ul`,..._provideComponents(),...props.components},{Frame,Heading,Note,OptimizedImage,Table,Tip}=_components;if(!Frame)_missingMdxReference(`Frame`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!OptimizedImage)_missingMdxReference(`OptimizedImage`,true);if(!Table)_missingMdxReference(`Table`,true);if(!Tip)_missingMdxReference(`Tip`,true);return _jsxs(_Fragment,{children:[_jsx(Note,{children:_jsxs(_components.p,{children:[`Benchmarks is in `,_jsx(_components.strong,{children:`beta`}),`. Reach out to `,_jsx(_components.a,{href:`mailto:support@sphinx.ai`,children:`Sphinx`}),` to activate it for your workspace. Include your workspace name. The interface and behavior may change during the beta.`]})}),`\n`,_jsx(_components.p,{children:`Use Benchmarks to test Sphinx on questions your team cares about. You define questions and criteria for a good answer, run the benchmark, and inspect the generated answers and judge results.`}),`\n`,_jsx(Heading,{level:`2`,id:`before-you-start`,children:`Before you start`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsx(_components.li,{children:`Ask Sphinx to activate Benchmarks for your workspace.`}),`\n`,_jsx(_components.li,{children:`Prepare representative questions and confirmed criteria for correct answers.`}),`\n`,_jsx(_components.li,{children:`Select the workspace whose knowledge you want to evaluate.`}),`\n`]}),`\n`,_jsx(Heading,{level:`2`,id:`open-benchmarks`,children:`Open Benchmarks`}),`\n`,_jsxs(_components.p,{children:[`Open `,_jsx(_components.strong,{children:`Settings`}),` at the bottom left and select `,_jsx(_components.strong,{children:`Benchmarks`}),`. The page contains `,_jsx(_components.strong,{children:`Average score trend`}),`, `,_jsx(_components.strong,{children:`Questions`}),`, and `,_jsx(_components.strong,{children:`Benchmark runs`}),`.`]}),`\n`,_jsx(Frame,{caption:`Benchmarks in a demo account with synthetic data.`,children:_jsx(OptimizedImage,{src:`https://mintcdn.com/sphinx-977f1673/O9wCzUTcMLhLJ368/images/ui/benchmarks.jpg?fit=max\u0026auto=format\u0026n=O9wCzUTcMLhLJ368\u0026q=85\u0026s=011047cc35210cb4fedd7fbe0d0fc14d`,alt:`Benchmarks page with average score trend, question list, Add question, and Run benchmark controls`,\"data-og-width\":`659`,width:`659`,\"data-og-height\":`886`,height:`886`,\"data-path\":`images/ui/benchmarks.jpg`,\"data-optimize\":`true`,\"data-opv\":`3`,srcset:`https://mintcdn.com/sphinx-977f1673/O9wCzUTcMLhLJ368/images/ui/benchmarks.jpg?w=280\u0026fit=max\u0026auto=format\u0026n=O9wCzUTcMLhLJ368\u0026q=85\u0026s=bdfda55f1f871ecc1ddc4d08a566307a 280w, https://mintcdn.com/sphinx-977f1673/O9wCzUTcMLhLJ368/images/ui/benchmarks.jpg?w=560\u0026fit=max\u0026auto=format\u0026n=O9wCzUTcMLhLJ368\u0026q=85\u0026s=a362ccd972a9662ec3e416cbfa577cc1 560w, https://mintcdn.com/sphinx-977f1673/O9wCzUTcMLhLJ368/images/ui/benchmarks.jpg?w=840\u0026fit=max\u0026auto=format\u0026n=O9wCzUTcMLhLJ368\u0026q=85\u0026s=924bcf12ea4b8fe607c003e3340cd302 840w, https://mintcdn.com/sphinx-977f1673/O9wCzUTcMLhLJ368/images/ui/benchmarks.jpg?w=1100\u0026fit=max\u0026auto=format\u0026n=O9wCzUTcMLhLJ368\u0026q=85\u0026s=905646a486720e25347b6dda34444105 1100w, https://mintcdn.com/sphinx-977f1673/O9wCzUTcMLhLJ368/images/ui/benchmarks.jpg?w=1650\u0026fit=max\u0026auto=format\u0026n=O9wCzUTcMLhLJ368\u0026q=85\u0026s=8d9437a9234e8c26c357cd50b319b67c 1650w, https://mintcdn.com/sphinx-977f1673/O9wCzUTcMLhLJ368/images/ui/benchmarks.jpg?w=2500\u0026fit=max\u0026auto=format\u0026n=O9wCzUTcMLhLJ368\u0026q=85\u0026s=e870402939a00857d7c20988a6138acc 2500w`})}),`\n`,_jsx(Heading,{level:`2`,id:`add-questions-and-criteria`,children:`Add questions and criteria`}),`\n`,_jsxs(_components.ol,{children:[`\n`,_jsxs(_components.li,{children:[`Click `,_jsx(_components.strong,{children:`Add question`}),`.`]}),`\n`,_jsxs(_components.li,{children:[`Enter the question in `,_jsx(_components.strong,{children:`Question`}),`.`]}),`\n`,_jsxs(_components.li,{children:[`Enter a specific requirement in `,_jsx(_components.strong,{children:`Criterion 1`}),`. Use `,_jsx(_components.strong,{children:`Add criterion`}),` for additional requirements. Each criterion is scored separately, and questions have equal weight in the final score.`]}),`\n`,_jsxs(_components.li,{children:[`Click `,_jsx(_components.strong,{children:`Add question`}),` to save it.`]}),`\n`]}),`\n`,_jsx(_components.p,{children:`For example:`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Field`}),_jsx(_components.th,{children:`Demo content`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Question`}),_jsx(_components.td,{children:`How long should someone listen to a song for it to count as a finance-qualified play?`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Criterion 1`}),_jsx(_components.td,{children:`The answer says at least 30 qualifying seconds.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Criterion 2`}),_jsx(_components.td,{children:`The answer distinguishes the finance definition from product engagement definitions.`})]})]})]}),`\n`,_jsx(_components.p,{children:`Use the search field to find saved questions. Each question has controls to view, edit, or delete it.`}),`\n`,_jsx(Tip,{children:_jsx(_components.p,{children:`Write questions with enough context to have a clear expected answer. If different teams use different definitions, name the team or workflow. Review criteria with the knowledge owner so an outdated expected answer does not turn a valid response into a misleading failure.`})}),`\n`,_jsx(Heading,{level:`2`,id:`run-and-inspect-a-benchmark`,children:`Run and inspect a benchmark`}),`\n`,_jsxs(_components.ol,{children:[`\n`,_jsxs(_components.li,{children:[`Click `,_jsx(_components.strong,{children:`Run benchmark`}),`.`]}),`\n`,_jsxs(_components.li,{children:[`Follow the run under `,_jsx(_components.strong,{children:`Benchmark runs`}),`. Use `,_jsx(_components.strong,{children:`Status`}),` to filter runs.`]}),`\n`,_jsxs(_components.li,{children:[`Open a completed run to see `,_jsx(_components.strong,{children:`Benchmark run results`}),`. You can also click a run in the average score trend.`]}),`\n`,_jsxs(_components.li,{children:[`Review the number of test cases and the `,_jsx(_components.strong,{children:`Mean`}),`, `,_jsx(_components.strong,{children:`Median`}),`, `,_jsx(_components.strong,{children:`Lowest`}),`, and `,_jsx(_components.strong,{children:`Highest`}),` scores.`]}),`\n`,_jsxs(_components.li,{children:[`For each test case, inspect the `,_jsx(_components.strong,{children:`Given answer`}),`, each `,_jsx(_components.strong,{children:`Criterion`}),`, and the `,_jsx(_components.strong,{children:`Judge result`}),`, including the score and explanation.`]}),`\n`]}),`\n`,_jsx(_components.p,{children:`Scores are displayed on a 0–100 scale. A high score means the answer satisfied the configured criteria according to the judge. Read individual results before drawing a conclusion about the knowledge base.`}),`\n`,_jsx(Heading,{level:`2`,id:`evaluate-generalization`,children:`Evaluate generalization`}),`\n`,_jsxs(_components.p,{children:[`Include representative questions that were not used to develop your knowledge base. Keep the questions, criteria, model, and agent setup comparable when measuring a change. For a broader agent evaluation, compare answer quality alongside cost and latency; the score trend alone does not describe those tradeoffs. See `,_jsx(_components.a,{href:`/concepts/context-learning`,children:`Learning organizational context`}),`.`]}),`\n`,_jsx(Heading,{level:`2`,id:`track-changes-over-time`,children:`Track changes over time`}),`\n`,_jsxs(_components.p,{children:[`Use `,_jsx(_components.strong,{children:`1d`}),`, `,_jsx(_components.strong,{children:`7d`}),`, or `,_jsx(_components.strong,{children:`30d`}),` in `,_jsx(_components.strong,{children:`Average score trend`}),` to change the displayed time range. Question-level trends help you identify regressions that an overall average can hide.`]}),`\n`,_jsxs(_components.p,{children:[`When a score falls, check both the answer and the criterion. Correct missing or conflicting knowledge through `,_jsx(_components.a,{href:`/knowledge-base/ingest`,children:`ingest`}),` or `,_jsx(_components.a,{href:`/knowledge-base/controversies`,children:`controversy resolution`}),`, review the resulting changes, and rerun the benchmark. If the expected answer changed, update the criterion deliberately and account for that change when comparing runs.`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Benchmarks (beta)","description":"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.","href":"/benchmarks"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Before you start","slug":"before-you-start","depth":2,"children":[]},{"title":"Open Benchmarks","slug":"open-benchmarks","depth":2,"children":[]},{"title":"Add questions and criteria","slug":"add-questions-and-criteria","depth":2,"children":[]},{"title":"Run and inspect a benchmark","slug":"run-and-inspect-a-benchmark","depth":2,"children":[]},{"title":"Evaluate generalization","slug":"evaluate-generalization","depth":2,"children":[]},{"title":"Track changes over time","slug":"track-changes-over-time","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":[],"subdomain":"sphinx-977f1673","siteName":"Sphinx Docs","title":"Benchmarks (beta) - Sphinx Docs","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-17T20:43:49.018Z","originalFileLocation":"benchmarks.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.\">\u003cmeta data-mint-head name=\"application-name\" content=\"Sphinx Docs\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Benchmarks (beta) - Sphinx Docs\">\u003cmeta data-mint-head property=\"og:description\" content=\"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Sphinx Docs\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://docs.sphinx.ai/benchmarks\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://sphinx-977f1673.mintlify.site/_mintlify/api/og/v2?division=Knowledge+base\u0026amp;title=Benchmarks+%28beta%29\u0026amp;description=Evaluate+Sphinx+answers+against+your+questions+and+criteria%2C+inspect+results%2C+and+track+scores+over+time.\u0026amp;theme=2e0ca79c820e075d549cbc7f\u0026amp;signature=9cc356f375e678755424e829dbc92ce596a2dc6be3b1e42ae74f7fab16596cb5\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Benchmarks (beta) - Sphinx Docs\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://sphinx-977f1673.mintlify.site/_mintlify/api/og/v2?division=Knowledge+base\u0026amp;title=Benchmarks+%28beta%29\u0026amp;description=Evaluate+Sphinx+answers+against+your+questions+and+criteria%2C+inspect+results%2C+and+track+scores+over+time.\u0026amp;theme=2e0ca79c820e075d549cbc7f\u0026amp;signature=9cc356f375e678755424e829dbc92ce596a2dc6be3b1e42ae74f7fab16596cb5\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/sphinx-977f1673/8idukxGllAmRhGRj/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Sphinx Docs\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#704643\">\u003clink data-mint-head rel=\"canonical\" href=\"https://docs.sphinx.ai/benchmarks\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/benchmarks.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://docs.sphinx.ai/#organization\",\"name\":\"Sphinx Docs\",\"url\":\"https://docs.sphinx.ai\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/sphinx-977f1673/P__T7c22F_M_IxmY/logo/wordmark-light.svg?fit=max\u0026auto=format\u0026n=P__T7c22F_M_IxmY\u0026q=85\u0026s=f9db2de016b08aba238741c2d0e90e74\"}},{\"@type\":\"WebSite\",\"@id\":\"https://docs.sphinx.ai#website\",\"name\":\"Sphinx Docs\",\"url\":\"https://docs.sphinx.ai\",\"publisher\":{\"@id\":\"https://docs.sphinx.ai/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://docs.sphinx.ai/benchmarks#webpage\",\"url\":\"https://docs.sphinx.ai/benchmarks\",\"name\":\"Benchmarks (beta)\",\"description\":\"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.\",\"dateModified\":\"2026-11-17T20:43:49.018Z\",\"isPartOf\":{\"@id\":\"https://docs.sphinx.ai#website\"},\"breadcrumb\":{\"@id\":\"https://docs.sphinx.ai/benchmarks#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://docs.sphinx.ai/benchmarks#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Knowledge base\",\"item\":\"https://docs.sphinx.ai/knowledge-base/browse\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Benchmarks (beta)\",\"item\":\"https://docs.sphinx.ai/benchmarks\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://docs.sphinx.ai/benchmarks#article\",\"headline\":\"Benchmarks (beta)\",\"name\":\"Benchmarks (beta)\",\"description\":\"Evaluate Sphinx answers against your questions and criteria, inspect results, and track scores over time.\",\"url\":\"https://docs.sphinx.ai/benchmarks\",\"mainEntityOfPage\":{\"@id\":\"https://docs.sphinx.ai/benchmarks#webpage\"},\"dateModified\":\"2026-11-17T20:43:49.018Z\",\"publisher\":{\"@id\":\"https://docs.sphinx.ai/#organization\"},\"isPartOf\":{\"@id\":\"https://docs.sphinx.ai#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"Sphinx Docs\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"Sphinx Docs\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/benchmarks.md","siteName":"Sphinx Docs","nav":{"global":{"anchors":[{"anchor":"Blog","icon":"newspaper","href":"https://sphinx.ai/blog/filters/all"}]},"tabs":[{"tab":"Documentation","groups":[{"group":"How Sphinx works","pages":[{"title":"How Sphinx works","href":"/how-sphinx-works"},{"title":"Learning organizational context","href":"/concepts/context-learning"},{"title":"Inspect and govern learned knowledge","href":"/concepts/interpretability"}]},{"group":"Knowledge base","pages":[{"title":"Browse the Sphinx UI","href":"/knowledge-base/browse"},{"title":"Ingest and train on data and documents","href":"/knowledge-base/ingest"},{"title":"Ingest directly from a Git repository","icon":"github","href":"/git-ingest"},{"title":"Write a good ingest prompt","icon":"pen-to-square","href":"/ingest-prompts"},{"title":"Review and accept knowledge changes","href":"/knowledge-base/review-changes"},{"title":"Resolve controversies","description":"Understand how Sphinx conveys unresolved conflicts to agents, review the evidence, and resolve disagreements when you are ready.","href":"/knowledge-base/controversies"},{"title":"Benchmarks (beta)","href":"/benchmarks"}]},{"group":"Sphinx Responses API","pages":[{"title":"Use the Sphinx Responses API","description":"Point your agent's Responses requests at Sphinx to receive context-augmented answers in the Responses format.","href":"/responses-api/overview"}]},{"group":"Sphinx MCP","pages":[{"title":"Use the Sphinx MCP server","href":"/mcp/overview"},{"group":"Connect Sphinx MCP to your agent","pages":[{"title":"Sphinx plugin onboarding for Claude","href":"/claude/sphinx-plugin-onboarding"},{"title":"API key setup for Claude Code CLI","href":"/claude-code-cli/api-key-setup"},{"title":"Install Sphinx MCP in ChatGPT Desktop","href":"/chatgpt-desktop/install-sphinx-mcp"}]}]},{"group":"Sphinx Private Gateway","pages":[{"title":"Sphinx Private Gateway","href":"/private-gateway/overview"},{"title":"Connect Claude Desktop to an API gateway","href":"/claude-desktop/api-gateway-setup"},{"title":"Setup ChatGPT Desktop LLM Gateway","href":"/chatgpt-desktop/setup-llm-gateway"},{"title":"Use Sphinx MCP with Claude Private Gateway","href":"/claude-desktop/sphinx-mcp/install-sphinx-mcp"}]},{"group":"Python SDK","pages":[{"title":"Python SDK reference","href":"/python-sdk/fit"},{"title":"Run your first ingest job","href":"/python-sdk/quickstart"}]},{"group":"Examples","pages":[{"title":"Train AI on a Business, using Sphinx","icon":"graduation-cap","href":"/examples/business-training"}]}]}]},"basePath":"","title":"Benchmarks (beta) - Sphinx Docs","deploymentId":"dpl_Em2U4NQQ2N2DE3HPbPUJVZLnAtvc","buildId":"dpl_Em2U4NQQ2N2DE3HPbPUJVZLnAtvc","dynamics":["07c022d89b","a74be98ba7"],"chunks":["Frame-DMn5ZJp6.js","DynamicMarkdown-5yMTCLas.js","zoom-image-lightbox-BUkqX3JF.js","lightbox-handle-r17bVUz3.js"]}