Enable AI agents to interact with the Atla API for state-of-the-art LLMJ evaluation.
An MCP server implementation providing a standardized interface for LLMs to interact with the Atla API for state-of-the-art LLMJ evaluation.
Learn more about Atla here. Learn more about the Model Context Protocol here.
evaluate_llm_response
: Evaluate an LLM's response to a prompt using a given evaluation criteria. This function uses an Atla evaluation model under the hood to return a dictionary containing a score for the model's response and a textual critique containing feedback on the model's response.evaluate_llm_response_on_multiple_criteria
: Evaluate an LLM's response to a prompt across multiple evaluation criteria. This function uses an Atla evaluation model under the hood to return a list of dictionaries, each containing an evaluation score and critique for a given criteria.To use the MCP server, you will need an Atla API key. You can find your existing API key here or create a new one here.
We recommend using
uv
to manage the Python environment. See here for installation instructions.
Once you have uv
installed and have your Atla API key, you can manually run the MCP server using uvx
(which is provided by uv
):
ATLA_API_KEY=<your-api-key> uvx atla-mcp-server
Having issues or need help connecting to another client? Feel free to open an issue or contact us!
For more details on using the OpenAI Agents SDK with MCP servers, refer to the official documentation.
pip install openai-agents
import os
from agents import Agent
from agents.mcp import MCPServerStdio
async with MCPServerStdio(
params={
"command": "uvx",
"args": ["atla-mcp-server"],
"env": {"ATLA_API_KEY": os.environ.get("ATLA_API_KEY")}
}
) as atla_mcp_server:
...
For more details on configuring MCP servers in Claude Desktop, refer to the official MCP quickstart guide.
claude_desktop_config.json
file:{
"mcpServers": {
"atla-mcp-server": {
"command": "uvx",
"args": ["atla-mcp-server"],
"env": {
"ATLA_API_KEY": "<your-atla-api-key>"
}
}
}
}
You should now see options from atla-mcp-server
in the list of available MCP tools.
For more details on configuring MCP servers in Cursor, refer to the official documentation.
.cursor/mcp.json
file:{
"mcpServers": {
"atla-mcp-server": {
"command": "uvx",
"args": ["atla-mcp-server"],
"env": {
"ATLA_API_KEY": "<your-atla-api-key>"
}
}
}
}
You should now see atla-mcp-server
in the list of available MCP servers.
Contributions are welcome! Please see the CONTRIBUTING.md file for details.
This project is licensed under the MIT License. See the LICENSE file for details.
Knowledge graph-based persistent memory system
Dynamic and reflective problem-solving through thought sequences
The only platform you need to get paid - all payments in one place, invoicing and accounting reconciliations with Adfin.
ALAPI MCP Tools,Call hundreds of API interfaces via MCP
MCP to interface with multiple blockchains, staking, DeFi, swap, bridging, wallet management, DCA, Limit Orders, Coin Lookup, Tracking and more.
Marketing insights and audience analysis from Audiense reports, covering demographic, cultural, influencer, and content engagement analysis.
Query and analyze your Axiom logs, traces, and all other event data in natural language
Query Onchain data, like ERC20 tokens, transaction history, smart contract state.
Read and write access to your Baserow tables.
MCP Server that connects AI agents to Chargebee platform.