Enable AI agents to interact with the Atla API for state-of-the-art LLMJ evaluation.
An MCP server implementation providing a standardized interface for LLMs to interact with the Atla API for state-of-the-art LLMJ evaluation.
Learn more about Atla here. Learn more about the Model Context Protocol here.
evaluate_llm_response
: Evaluate an LLM's response to a prompt using a given evaluation criteria. This function uses an Atla evaluation model under the hood to return a dictionary containing a score for the model's response and a textual critique containing feedback on the model's response.evaluate_llm_response_on_multiple_criteria
: Evaluate an LLM's response to a prompt across multiple evaluation criteria. This function uses an Atla evaluation model under the hood to return a list of dictionaries, each containing an evaluation score and critique for a given criteria.To use the MCP server, you will need an Atla API key. You can find your existing API key here or create a new one here.
We recommend using
uv
to manage the Python environment. See here for installation instructions.
Once you have uv
installed and have your Atla API key, you can manually run the MCP server using uvx
(which is provided by uv
):
ATLA_API_KEY=<your-api-key> uvx atla-mcp-server
Having issues or need help connecting to another client? Feel free to open an issue or contact us!
For more details on using the OpenAI Agents SDK with MCP servers, refer to the official documentation.
pip install openai-agents
import os
from agents import Agent
from agents.mcp import MCPServerStdio
async with MCPServerStdio(
params={
"command": "uvx",
"args": ["atla-mcp-server"],
"env": {"ATLA_API_KEY": os.environ.get("ATLA_API_KEY")}
}
) as atla_mcp_server:
...
For more details on configuring MCP servers in Claude Desktop, refer to the official MCP quickstart guide.
claude_desktop_config.json
file:{
"mcpServers": {
"atla-mcp-server": {
"command": "uvx",
"args": ["atla-mcp-server"],
"env": {
"ATLA_API_KEY": "<your-atla-api-key>"
}
}
}
}
You should now see options from atla-mcp-server
in the list of available MCP tools.
For more details on configuring MCP servers in Cursor, refer to the official documentation.
.cursor/mcp.json
file:{
"mcpServers": {
"atla-mcp-server": {
"command": "uvx",
"args": ["atla-mcp-server"],
"env": {
"ATLA_API_KEY": "<your-atla-api-key>"
}
}
}
}
You should now see atla-mcp-server
in the list of available MCP servers.
Contributions are welcome! Please see the CONTRIBUTING.md file for details.
This project is licensed under the MIT License. See the LICENSE file for details.
Up-to-date documentation for your coding agent. Covers 1000s of public repos and sites. Built by ref.tools
A comprehensive MCP server for tooling interactions(40+) and resource accessibility(60+) plus many useful prompts to interact with Algorand Blockchain.
Tools to query latest Maven dependency information
Manage Google Apps Script projects, including creation, editing, deployment, and execution. Requires Google Cloud credentials for authentication.
Arbitrary code execution and tool-use platform for LLMs by Riza
Server for advanced AI-driven video editing, semantic search, multilingual transcription, generative media, voice cloning, and content moderation.
Model Kontext Protocol Server for Kubernetes that allows LLM-powered applications to interact with Kubernetes clusters through native Go implementation with direct API integration and comprehensive resource management.
Query and analyze your Opik logs, traces, prompts and all other telemtry data from your LLMs in natural language.
A specialized MCP gateway for LLM enhancement prompts and jailbreaks with dynamic schema adaptation. Provides prompts for different LLMs using an enum-based approach.
Clojure linter