Opik Evaluation Ops (BYOK)

Opik Evaluation Ops (BYOK) — connector documentation

• MCP endpoint: https://byok-opik-evaluation-ops.connectorbay.com/mcp
• REST API: POST https://byok-opik-evaluation-ops.connectorbay.com/v1/tools/{tool_name}, described by the OpenAPI 3.1 spec at https://byok-opik-evaluation-ops.connectorbay.com/openapi.json
• Transport: Streamable HTTP, stateless, MCP specification 2025-06-18 or later
• Authentication: your own Opik Evaluation Ops Comet API key, sent in the X-Upstream-Key header and the Comet workspace name (shown in the Opik URL) in X-Upstream-Key-2 of every request (never stored, logged or cached)
• Health: https://byok-opik-evaluation-ops.connectorbay.com/health · Tool schemas: https://byok-opik-evaluation-ops.connectorbay.com/tools.json
• Publisher: Unify Dynamics LLC · connectorbay.com/byok-opik-evaluation-ops · agent@connectorbay.com

What Opik Evaluation Ops (BYOK) is for

Opik Evaluation Ops (BYOK) lets an AI assistant read your own Opik Evaluation Ops data through the Opik Evaluation Ops API with a key you supply (Comet API key). Read-only: 8 tools (list_evaluation_rules, get_evaluation_rule, list_annotation_queues, get_annotation_queue, list_optimizations, get_optimization, list_dataset_versions, list_experiment_feedback_score_names). The BYOK key travels with each request, is used for that one call only and is never stored, logged or cached; it is only ever sent to www.comet.com. Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by Opik Evaluation Ops. Opik Evaluation Ops and its logos are trademarks of their owners and are used only to say which service the connector works with.

Ask your assistant things like:

• "Which online evaluation rules are running in my Opik workspace?"
• "Show the Opik evaluation rule 0192-rule"
• "Which annotation queues are set up in my Opik workspace?"
• "Show the Opik annotation queue 0192-queue"
• "Which prompt optimizations ran in my Opik workspace and how much did they improve?"
• "Show the Opik optimization 0192-opt"

What it does

• list_evaluation_rules: List the online evaluation rules (LLM-as-judge, Python metric and queue-router rules) with type, enabled flag, sampling rate, trigger scope and the projects they apply to.
• get_evaluation_rule: Get one online evaluation rule by id: type, action, enabled flag, sampling rate, trigger scope and projects.
• list_annotation_queues: List the annotation queues with scope, project, instructions, feedback definitions, annotators per item and item count (reviewer names are not returned).
• get_annotation_queue: Get one annotation queue: scope, instructions, feedback definitions, annotators per item, lock timeout, item count and its automation settings (reviewer names are not returned).
• list_optimizations: List prompt optimization runs with dataset, objective, status, trial count, baseline and best scores and cost.
• get_optimization: Get one prompt optimization run by id with status, objective, trial count, baseline and best scores, durations and cost.
• list_dataset_versions: List the versions of one dataset with version name, hash, tags, item totals and how many items were added, modified or deleted.
• list_experiment_feedback_score_names: List the names and types of the feedback scores recorded on experiments, optionally for one project.

BYOK key

You bring your own Opik Evaluation Ops Comet API key (https://www.comet.com/api/my/settings/). Your assistant sends it in the X-Upstream-Key header with each request. ConnectorBay uses it for that one call only: it is never stored, logged or cached, never shown in an answer, and it is only sent to www.comet.com over https, with no redirects followed. Create a key with the fewest permissions the tools need (read-only if the provider offers it), and revoke it at the provider whenever you like.

Notices

Reads evaluation configuration and aggregate scores only. Traces, spans, dataset items, annotator names and prompt text are not returned.

Data sources

• https://github.com/comet-ml/opik/blob/main/sdks/code_generation/fern/openapi/openapi.yaml (the API this connector calls, with a BYOK key)

Live from the Opik Evaluation Ops API (www.comet.com) with a BYOK key; documentation: https://github.com/comet-ml/opik/blob/main/sdks/code_generation/fern/openapi/openapi.yaml. Endpoints checked against that documentation on 2026-10-16.

What it is not

Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by Opik Evaluation Ops. Opik Evaluation Ops and its logos are trademarks of their owners and are used only to say which service the connector works with. It is read-only: it never creates, changes, sends or deletes anything in your Opik Evaluation Ops account.

Built for: Worldwide. Languages: English.

Opik Evaluation Ops (BYOK) — Privacy Policy

Effective 24 October 2026. Unify Dynamics LLC ("we", "us"), Menlo Park, California.

What Opik Evaluation Ops (BYOK) is. Opik Evaluation Ops (BYOK) is a connector (an MCP server and REST API) that an AI assistant calls on your behalf to read your own Opik Evaluation Ops data: Read the online evaluation rules, annotation queues, prompt optimizations, dataset versions and feedback score names of your Opik workspace with your own Comet API key. You bring your own Opik Evaluation Ops Comet API key; the assistant sends it to us with each request.

BYOK key. The key you supply is used to make the one Opik Evaluation Ops call you asked for. Our application does not store it, write it to a log, put it in a cache, or share it. It is sent only to www.comet.com, over https, and is removed from any text we return to you or your assistant. We do not follow redirects with your key. If Opik Evaluation Ops rejects it, we say so without repeating it.

What we receive. The arguments of each tool call, the BYOK key (for that call only), the technical headers needed to serve the request (an IP address or a client identifier supplied by the assistant) and standard server logs. What Opik Evaluation Ops receives. The request we make on your behalf, including the key and the arguments, under Opik Evaluation Ops's own terms and privacy policy. Opik Evaluation Ops acts at your direction and is the only recipient of the data of your request besides our hosting provider.

What we return. The answer Opik Evaluation Ops gives for your request, trimmed to the fields the tool documents. We do not keep it: it is returned to your assistant and not stored, cached or used to train anything.

What we do not do. We do not build a profile, sell or share personal information, or keep your Opik Evaluation Ops data. Rate-limit counters (a client identifier and timestamps) are held in memory for at most 24 hours. Hosting access logs (IP addresses, request paths and status codes) are kept for up to 30 days for security and abuse prevention; our application adds no request headers or bodies to them.

Sub-processors. Opik Evaluation Ops (the API you call with a BYOK key, at your direction). Hosting: Vercel (United States regions).

Your rights. Because we store no personal data beyond short-lived logs, requests to access, correct or delete data are usually satisfied automatically; you can still contact agent@connectorbay.com and we will respond within 30 days. Revoke your key at Opik Evaluation Ops at any time to end all access.

Children. Opik Evaluation Ops (BYOK) is a general-audience service operated by the adult using the assistant, under Opik Evaluation Ops's own age rules.

Changes. We will post changes here with a new effective date.

Opik Evaluation Ops (BYOK) — Terms of Service

Effective 24 October 2026. These terms govern use of the Opik Evaluation Ops (BYOK) connector ("the Service") operated by Unify Dynamics LLC ("we", "us").

1. The Service. Opik Evaluation Ops (BYOK) provides: Read the online evaluation rules, annotation queues, prompt optimizations, dataset versions and feedback score names of your Opik workspace with your own Comet API key. It is accessed through an AI assistant or directly over MCP or HTTP, and is read-only.
2. Your own key, your own account. You must use your own Opik Evaluation Ops credentials and only access data you are allowed to access. Opik Evaluation Ops's terms apply to your use of Opik Evaluation Ops; you are responsible for complying with them, including any rate limits and permitted uses. Do not share a key you do not own.
3. Independent. Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by Opik Evaluation Ops. Opik Evaluation Ops and its logos are trademarks of their owners and are used only to say which service the connector works with.
4. No advice, no guarantees. Output is the data Opik Evaluation Ops returns, lightly trimmed; it may be incomplete, delayed or changed by Opik Evaluation Ops. Check important facts at the source.
5. Read-only. The Service does not create, change, send or delete anything in your Opik Evaluation Ops account.
6. Acceptable use. Automated use within the published rate limits is welcome. Do not use the Service to access data you are not entitled to, to circumvent Opik Evaluation Ops's controls, or to attack any system.
7. Availability. The Service is provided "as is" without warranty. We may change, limit or discontinue it at any time; Opik Evaluation Ops may change or end its API at any time.
8. Data. Our Privacy Policy describes what we process. In short: the BYOK key is used for the one call and not stored or logged.
9. Pro keys. Optional ConnectorBay Pro API keys are issued at our discretion and are separate from your Opik Evaluation Ops key.
10. Liability. To the fullest extent permitted by law, Unify Dynamics LLC is not liable for any loss arising from use of or reliance on the Service, including any action Opik Evaluation Ops takes on your account. Our total liability is limited to US $100.
11. Governing law. California law; venue in San Mateo County, California.
12. Contact. Unify Dynamics LLC · agent@connectorbay.com · https://connectorbay.com/byok-opik-evaluation-ops

Opik Evaluation Ops (BYOK) support

Email agent@connectorbay.com. We answer within two business days.

• MCP endpoint: https://byok-opik-evaluation-ops.connectorbay.com/mcp
• REST API: https://byok-opik-evaluation-ops.connectorbay.com/openapi.json
• Live status: https://byok-opik-evaluation-ops.connectorbay.com/health

When you report a problem, include the question you asked your agent, roughly when, and which agent you used (for example Claude, ChatGPT or Gemini). Never send us passwords, account numbers or ID numbers. Opik Evaluation Ops (BYOK) never asks for them.