Be the first to get access to new connectors. Get early access →
ConnectorBay Early access

LangSmith (BYOK) — connector documentation

  • MCP endpoint: https://byok-langsmith.connectorbay.com/mcp
  • REST API: POST https://byok-langsmith.connectorbay.com/v1/tools/{tool_name}, described by the OpenAPI 3.1 spec at https://byok-langsmith.connectorbay.com/openapi.json
  • Transport: Streamable HTTP, stateless, MCP specification 2025-06-18 or later
  • Authentication: your own LangSmith LangSmith API key (personal access token or service key), sent in the X-Upstream-Key header of every request (never stored, logged or cached)
  • Health: https://byok-langsmith.connectorbay.com/health · Tool schemas: https://byok-langsmith.connectorbay.com/tools.json
  • Publisher: Unify Dynamics LLC · connectorbay.com/byok-langsmith · agent@connectorbay.com

What LangSmith (BYOK) is for

LangSmith (BYOK) lets an AI assistant read your own LangSmith data through the LangSmith API with a key you supply (LangSmith API key (personal access token or service key)). Read-only: 12 tools (list_projects, get_project, list_datasets, get_dataset, list_examples, query_runs, get_run, list_feedback, list_annotation_queues, list_prompt_repos, list_workspaces, get_workspace_stats). The BYOK key travels with each request, is used for that one call only and is never stored, logged or cached; it is only ever sent to api.smith.langchain.com. Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by LangSmith. LangSmith and its logos are trademarks of their owners and are used only to say which service the connector works with.

Ask your assistant things like:

  • "Which LangSmith projects have the most runs?"
  • "Show the statistics of LangSmith project 3f1c2a9e-8b7d-4c1e-9a55-0d6f2b7a1c10"
  • "What datasets do I have in LangSmith?"
  • "How many examples are in LangSmith dataset 7a2e5d10-4b3c-4e8f-b1a9-6c0d9f3e2a11?"
  • "Show some examples from LangSmith dataset 7a2e5d10-4b3c-4e8f-b1a9-6c0d9f3e2a11"
  • "Show the failed LLM runs of LangSmith project 3f1c2a9e-8b7d-4c1e-9a55-0d6f2b7a1c10 from the last day"

What it does

  • list_projects: List tracing projects (sessions) in the workspace with start time, run count, error rate, latency percentiles, token totals and cost. Stats such as run_count are only filled when include_stats is true.
  • get_project: Get one tracing project with its start time, run count, error rate, latency, tokens and cost.
  • list_datasets: List datasets with type, example count, number of experiments and modification time.
  • get_dataset: Get one dataset's type, description, example count, experiment count and dates.
  • list_examples: List examples of a dataset with inputs, outputs and metadata, most recent first.
  • query_runs: Query the runs (traces and spans) of one project, newest first, with status, type, timing, tokens, cost and error. Supports LangSmith's filter expression and a cursor for the next page. Does not return inputs or outputs.
  • get_run: Get one run (a trace or span) with name, type, status, timing, error, token counts, cost, tags and trace id.
  • list_feedback: List feedback records (scores and comments) filtered by run, project or feedback key.
  • list_annotation_queues: List annotation queues with type, reviewer settings and size.
  • list_prompt_repos: List prompt hub repositories with owner, tags, commit count, downloads and last update. Total count is in total.
  • list_workspaces: List the workspaces (tenants) the key can access with name, role and whether it is personal.
  • get_workspace_stats: Count the current workspace's projects, datasets, prompt repositories, annotation queues, deployments, dashboards and evaluators.

BYOK key

You bring your own LangSmith LangSmith API key (personal access token or service key) (https://smith.langchain.com/settings). Your assistant sends it in the X-Upstream-Key header with each request. ConnectorBay uses it for that one call only: it is never stored, logged or cached, never shown in an answer, and it is only sent to api.smith.langchain.com over https, with no redirects followed. Create a key with the fewest permissions the tools need (read-only if the provider offers it), and revoke it at the provider whenever you like.

Data sources

Live from the LangSmith API (api.smith.langchain.com) with a BYOK key; documentation: https://api.smith.langchain.com/openapi.json. Endpoints checked against that documentation on 2026-10-16.

What it is not

Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by LangSmith. LangSmith and its logos are trademarks of their owners and are used only to say which service the connector works with. It is read-only: it never creates, changes, sends or deletes anything in your LangSmith account.

Built for: Worldwide. Languages: English.