Be the first to get access to new connectors. Get early access →
ConnectorBay Early access

Scorecard (BYOK) — connector documentation

  • MCP endpoint: https://byok-scorecard-ai.connectorbay.com/mcp
  • REST API: POST https://byok-scorecard-ai.connectorbay.com/v1/tools/{tool_name}, described by the OpenAPI 3.1 spec at https://byok-scorecard-ai.connectorbay.com/openapi.json
  • Transport: Streamable HTTP, stateless, MCP specification 2025-06-18 or later
  • Authentication: your own Scorecard Scorecard API key (starts with ak_), sent in the X-Upstream-Key header of every request (never stored, logged or cached)
  • Health: https://byok-scorecard-ai.connectorbay.com/health · Tool schemas: https://byok-scorecard-ai.connectorbay.com/tools.json
  • Publisher: Unify Dynamics LLC · connectorbay.com/byok-scorecard-ai · agent@connectorbay.com

What Scorecard (BYOK) is for

Scorecard (BYOK) lets an AI assistant read your own Scorecard data through the Scorecard API with a key you supply (Scorecard API key (starts with ak_)). Read-only: 10 tools (list_projects, list_testsets, get_testset, list_metrics, get_metric, list_metric_groups, list_runs, get_run, list_systems, get_system). The BYOK key travels with each request, is used for that one call only and is never stored, logged or cached; it is only ever sent to api2.scorecard.io. Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by Scorecard. Scorecard and its logos are trademarks of their owners and are used only to say which service the connector works with.

Ask your assistant things like:

  • "Which Scorecard projects do I have?"
  • "What testsets does my Scorecard project have?"
  • "Show Scorecard testset ts_01"
  • "Which metrics does my Scorecard project use?"
  • "Show Scorecard metric met_01"
  • "Which metric groups are defined in my Scorecard project?"

What it does

  • list_projects: List your Scorecard projects (oldest first) with id, name and description; paged by cursor.
  • list_testsets: List a project's testsets with name, description and which fields map to inputs, expected answers and metadata.
  • get_testset: Get one testset's name, description and field mapping.
  • list_metrics: List a project's evaluation metrics with type, output type, judge model, passing threshold and guidelines; paged.
  • get_metric: Get one metric's definition: evaluation type, guidelines, judge model and passing threshold.
  • list_metric_groups: List a project's metric groups (named sets of metrics) with their member metric ids.
  • list_runs: List a project's evaluation runs (newest first) with testset, system version, record and score counts and status.
  • get_run: Get one run's testset, system version, metric ids, record and score counts and status.
  • list_systems: List the systems (the AI products under test) of a project with production version and all version names.
  • get_system: Get one system with its production version and the list of versions.

BYOK key

You bring your own Scorecard Scorecard API key (starts with ak_) (https://app.scorecard.io/settings). Your assistant sends it in the X-Upstream-Key header with each request. ConnectorBay uses it for that one call only: it is never stored, logged or cached, never shown in an answer, and it is only sent to api2.scorecard.io over https, with no redirects followed. Create a key with the fewest permissions the tools need (read-only if the provider offers it), and revoke it at the provider whenever you like.

Data sources

Live from the Scorecard API (api2.scorecard.io) with a BYOK key; documentation: https://docs.scorecard.io/api-reference/overview. Endpoints checked against that documentation on 2026-10-16.

What it is not

Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by Scorecard. Scorecard and its logos are trademarks of their owners and are used only to say which service the connector works with. It is read-only: it never creates, changes, sends or deletes anything in your Scorecard account.

Built for: Worldwide. Languages: English.