OpenAI evaluations and runs (BYOK)

OpenAI evaluations and runs (BYOK) — connector documentation

• MCP endpoint: https://byok-openai-evals.connectorbay.com/mcp
• REST API: POST https://byok-openai-evals.connectorbay.com/v1/tools/{tool_name}, described by the OpenAPI 3.1 spec at https://byok-openai-evals.connectorbay.com/openapi.json
• Transport: Streamable HTTP, stateless, MCP specification 2025-06-18 or later
• Authentication: your own OpenAI Evals OpenAI API key, sent in the X-Upstream-Key header of every request (never stored, logged or cached)
• Health: https://byok-openai-evals.connectorbay.com/health · Tool schemas: https://byok-openai-evals.connectorbay.com/tools.json
• Publisher: Unify Dynamics LLC · connectorbay.com/byok-openai-evals · agent@connectorbay.com

What OpenAI evaluations and runs (BYOK) is for

OpenAI evaluations and runs (BYOK) lets an AI assistant read your own OpenAI Evals data through the OpenAI Evals API with a key you supply (OpenAI API key). Read-only: 5 tools (list_evals, get_eval, list_eval_runs, get_eval_run, list_eval_run_output_items). The BYOK key travels with each request, is used for that one call only and is never stored, logged or cached; it is only ever sent to api.openai.com. Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by OpenAI Evals. OpenAI Evals and its logos are trademarks of their owners and are used only to say which service the connector works with.

Ask your assistant things like:

• "Which evaluations have I defined in my OpenAI project?"
• "Show the details of OpenAI evaluation eval_abc123"
• "How did the runs of OpenAI evaluation eval_abc123 turn out?"
• "What were the pass and fail counts of OpenAI eval run evalrun_abc123?"
• "Which items failed in OpenAI eval run evalrun_abc123?"

What it does

• list_evals: List the evaluations of the key's project with name and creation time, ordered by creation or update time. Reads evaluation metadata only: it starts no run and sends no prompt.
• get_eval: Get one evaluation: name, creation time, metadata, and the data source configuration type. Reads evaluation metadata only: it starts no run and sends no prompt.
• list_eval_runs: List the runs of one evaluation with status, model, report link and pass, fail and error counts. Reads evaluation metadata only: it starts no run and sends no prompt.
• get_eval_run: Get one evaluation run: status, model, creation time, report link, pass, fail and error counts, and any error. Reads evaluation metadata only: it starts no run and sends no prompt.
• list_eval_run_output_items: List the output items of an evaluation run with pass or fail status and ids (the item contents are not picked). Reads evaluation metadata only: it starts no run and sends no prompt.

BYOK key

You bring your own OpenAI Evals OpenAI API key (https://platform.openai.com/api-keys). Your assistant sends it in the X-Upstream-Key header with each request. ConnectorBay uses it for that one call only: it is never stored, logged or cached, never shown in an answer, and it is only sent to api.openai.com over https, with no redirects followed. Create a key with the fewest permissions the tools need (read-only if the provider offers it), and revoke it at the provider whenever you like.

Data sources

• https://platform.openai.com/docs/api-reference/evals (the API this connector calls, with a BYOK key)

Live from the OpenAI Evals API (api.openai.com) with a BYOK key; documentation: https://platform.openai.com/docs/api-reference/evals. Endpoints checked against that documentation on 2026-10-16.

What it is not

Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by OpenAI Evals. OpenAI Evals and its logos are trademarks of their owners and are used only to say which service the connector works with. It is read-only: it never creates, changes, sends or deletes anything in your OpenAI Evals account.

Built for: Worldwide. Languages: English.

OpenAI evaluations and runs (BYOK) — Privacy Policy

Effective 24 October 2026. Unify Dynamics LLC ("we", "us"), Menlo Park, California.

What OpenAI evaluations and runs (BYOK) is. OpenAI evaluations and runs (BYOK) is a connector (an MCP server and REST API) that an AI assistant calls on your behalf to read your own OpenAI Evals data: Read your OpenAI evaluations, their runs, pass and fail counts and per-item results metadata with your own API key. You bring your own OpenAI Evals OpenAI API key; the assistant sends it to us with each request.

BYOK key. The key you supply is used to make the one OpenAI Evals call you asked for. Our application does not store it, write it to a log, put it in a cache, or share it. It is sent only to api.openai.com, over https, and is removed from any text we return to you or your assistant. We do not follow redirects with your key. If OpenAI Evals rejects it, we say so without repeating it.

What we receive. The arguments of each tool call, the BYOK key (for that call only), the technical headers needed to serve the request (an IP address or a client identifier supplied by the assistant) and standard server logs. What OpenAI Evals receives. The request we make on your behalf, including the key and the arguments, under OpenAI Evals's own terms and privacy policy. OpenAI Evals acts at your direction and is the only recipient of the data of your request besides our hosting provider.

What we return. The answer OpenAI Evals gives for your request, trimmed to the fields the tool documents. We do not keep it: it is returned to your assistant and not stored, cached or used to train anything.

What we do not do. We do not build a profile, sell or share personal information, or keep your OpenAI Evals data. Rate-limit counters (a client identifier and timestamps) are held in memory for at most 24 hours. Hosting access logs (IP addresses, request paths and status codes) are kept for up to 30 days for security and abuse prevention; our application adds no request headers or bodies to them.

Sub-processors. OpenAI Evals (the API you call with a BYOK key, at your direction). Hosting: Vercel (United States regions).

Your rights. Because we store no personal data beyond short-lived logs, requests to access, correct or delete data are usually satisfied automatically; you can still contact agent@connectorbay.com and we will respond within 30 days. Revoke your key at OpenAI Evals at any time to end all access.

Children. OpenAI evaluations and runs (BYOK) is a general-audience service operated by the adult using the assistant, under OpenAI Evals's own age rules.

Changes. We will post changes here with a new effective date.

OpenAI evaluations and runs (BYOK) — Terms of Service

Effective 24 October 2026. These terms govern use of the OpenAI evaluations and runs (BYOK) connector ("the Service") operated by Unify Dynamics LLC ("we", "us").

1. The Service. OpenAI evaluations and runs (BYOK) provides: Read your OpenAI evaluations, their runs, pass and fail counts and per-item results metadata with your own API key. It is accessed through an AI assistant or directly over MCP or HTTP, and is read-only.
2. Your own key, your own account. You must use your own OpenAI Evals credentials and only access data you are allowed to access. OpenAI Evals's terms apply to your use of OpenAI Evals; you are responsible for complying with them, including any rate limits and permitted uses. Do not share a key you do not own.
3. Independent. Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by OpenAI Evals. OpenAI Evals and its logos are trademarks of their owners and are used only to say which service the connector works with.
4. No advice, no guarantees. Output is the data OpenAI Evals returns, lightly trimmed; it may be incomplete, delayed or changed by OpenAI Evals. Check important facts at the source.
5. Read-only. The Service does not create, change, send or delete anything in your OpenAI Evals account.
6. Acceptable use. Automated use within the published rate limits is welcome. Do not use the Service to access data you are not entitled to, to circumvent OpenAI Evals's controls, or to attack any system.
7. Availability. The Service is provided "as is" without warranty. We may change, limit or discontinue it at any time; OpenAI Evals may change or end its API at any time.
8. Data. Our Privacy Policy describes what we process. In short: the BYOK key is used for the one call and not stored or logged.
9. Pro keys. Optional ConnectorBay Pro API keys are issued at our discretion and are separate from your OpenAI Evals key.
10. Liability. To the fullest extent permitted by law, Unify Dynamics LLC is not liable for any loss arising from use of or reliance on the Service, including any action OpenAI Evals takes on your account. Our total liability is limited to US $100.
11. Governing law. California law; venue in San Mateo County, California.
12. Contact. Unify Dynamics LLC · agent@connectorbay.com · https://connectorbay.com/byok-openai-evals

OpenAI evaluations and runs (BYOK) support

Email agent@connectorbay.com. We answer within two business days.

• MCP endpoint: https://byok-openai-evals.connectorbay.com/mcp
• REST API: https://byok-openai-evals.connectorbay.com/openapi.json
• Live status: https://byok-openai-evals.connectorbay.com/health

When you report a problem, include the question you asked your agent, roughly when, and which agent you used (for example Claude, ChatGPT or Gemini). Never send us passwords, account numbers or ID numbers. OpenAI evaluations and runs (BYOK) never asks for them.