Be the first to get access to new connectors. Get early access →
ConnectorBay Early access
Discover
Langfuse evaluators, rules and annotation queues (your keys) icon

Langfuse evaluators, rules and annotation queues (your keys)

Read your Langfuse evaluators, evaluation rules, annotation queues, dashboards and LLM connection names (EU cloud region) with your own project API keys

Unify Dynamics LLC

Tools10read-only
Sign-inBYOKbring your own key
Key keptNeverused for one call only
PriceFreeoptional Pro limits
ProviderLangfuse Evaluatorsunofficial, not affiliated
Works withMCP · RESTany agent

Try asking

Real questions, and the tools your agent calls to answer them.

Which evaluators are set up in my Langfuse project?
Tools called
list_evaluators
Show Langfuse evaluator ev_abc123.
Tools called
get_evaluator
How many versions does Langfuse evaluator ev_abc123 have?
Tools called
list_evaluator_versions
Which Langfuse evaluation rules are enabled and at what sampling rate?
Tools called
list_evaluation_rules
Show Langfuse evaluation rule rule_1.
Tools called
get_evaluation_rule
What annotation queues exist in Langfuse?
Tools called
list_annotation_queues

About

Langfuse evaluators, rules and annotation queues (your keys) lets an AI assistant read your own Langfuse Evaluators data through the Langfuse Evaluators API with a key you supply (Langfuse public key (starts with pk-lf-)). Read-only: 10 tools (list_evaluators, get_evaluator, list_evaluator_versions, list_evaluation_rules, get_evaluation_rule, list_annotation_queues, get_annotation_queue, list_annotation_queue_items, list_dashboards, list_llm_connections). The BYOK key travels with each request, is used for that one call only and is never stored, logged or cached; it is only ever sent to cloud.langfuse.com. Unofficial connector by Unify Dynamics LLC: not affiliated with, endorsed by or sponsored by Langfuse Evaluators. Langfuse Evaluators and its logos are trademarks of their owners and are used only to say which service the connector works with.

Sources: Live from the Langfuse Evaluators API (cloud.langfuse.com) with a BYOK key; documentation: https://api.reference.langfuse.com/. Endpoints checked against that documentation on 2026-10-16.

Privacy

The publisher, Unify Dynamics LLC, states that tool inputs are processed in memory and not retained.

Data not collectedNo account, no profile, no stored prompts or results.

What it can do

10 tools, grouped by job. Every one is read-only or compute-only.

OpenAPI
Read your data10 tools
  • List evaluatorsList evaluators (LLM-as-judge and code) with name, type, status, current version and dates.
  • Get evaluatorGet one evaluator by id: name, type, status, pause reason, current version and dates.
  • List evaluator versionsList the versions of one evaluator, newest first, with version number and creation time.
  • List evaluation rulesList evaluation rules (which live data gets scored) with enabled flag, sampling rate and evaluator assignments.
  • Get evaluation ruleGet one evaluation rule by id with enabled flag, sampling rate, filters and evaluator assignments.
  • List annotation queuesList annotation queues (human review queues) with name, description and the score configs used.
  • Get annotation queueGet one annotation queue by id: name, description and score config ids.
  • List annotation queue itemsList the items of an annotation queue (traces, observations or sessions to review) with status and completion time.
  • List dashboardsList custom dashboards (unstable API) with name, description and dates; the widget definitions are not returned.
  • List llm connectionsList the LLM provider connections configured for evaluators: provider, adapter and base URL (secrets are not returned).

Information

PublisherUnify Dynamics LLC
CategoryDeveloper tools
LanguagesEnglish
Built forWorldwide
CompatibilityAny MCP client · any agent · REST
Version0.1.0
MCP endpointhttps://byok-langfuse-evaluators.connectorbay.com/mcp
REST APIopenapi.json
SupportGet help · Status
CountryWorldwide
FunctionOther
LayerDeep connectors

More like this

More in Other

TalentLMS (BYOK)

Read your TalentLMS users, courses, categories, groups, branches, course progress and domain usage with your own API key

Live · MCP + REST10 tools

Lucca (BYOK)

Read your Lucca HR directory (employees, departments, establishments, legal units) and office work locations with your own API key

Live · MCP + REST8 tools

Unified API (BYOK)

Read employees, groups, locations, time off and ATS jobs from the HR or recruiting systems connected to your Unified account, with your own key (no pay, ID or birth data)

Live · MCP + REST6 tools

Vacation Tracker (BYOK)

Read departments, locations, holidays, labels, leave types, leave requests, users and leave balances from your Vacation Tracker workspace with your own API key

Live · MCP + REST8 tools

Knit (BYOK)

Read leave types, leave balances, pay periods, termination reasons, departments and ATS reasons and tags from the HR system linked to your Knit account, with your own keys

Live · MCP + REST8 tools

Kombo (BYOK)

Read teams, locations, absence types, time off balances and ATS stages from the HR system linked to your Kombo account, with your own keys

Live · MCP + REST11 tools

Get Langfuse evaluators, rules and annotation queues (your keys)

Free · BYOK · Read-only

BYOK: bring your own keyThis connector uses your own Langfuse Evaluators Langfuse public key (starts with pk-lf-). Create one at https://cloud.langfuse.com, with the fewest permissions it needs (read-only if the provider offers it). Your assistant sends it in the X-Upstream-Key header with every request. It is used for that one call only: never stored, logged or cached, and only ever sent to Langfuse Evaluators. Revoke it at the provider whenever you like. Unofficial connector: not affiliated with Langfuse Evaluators.