Documentation Index

Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt Use this file to discover all available pages before exploring further.

Galileo MCP Server

Learn how to integrate Galileo's Model Context Protocol (MCP) server with AI-enabled IDEs like Cursor and VS Code

The Galileo Model Context Protocol (MCP) server enables seamless integration between AI-powered IDEs, such as Cursor, or VS Code with GitHub Copilot, and Galileo's evaluation and observability platform.

With MCP, you can access Galileo's capabilities directly from your development environment, including:

• Creating and managing datasets
• Running experiments
• Setting up prompt templates
• Getting signals on Log streams
• Integrating Galileo with your code

Prerequisites

Before you begin, ensure you have the following:

Install an AI-enabled IDE such as [Cursor](https://cursor.sh) or [VS Code](https://code.visualstudio.com) with AI capabilities Generate your Galileo API key from the [API keys page](https://app.galileo.ai/settings/api-keys)

Configure your IDE

The Galileo MCP server works with both Cursor and VS Code. Follow the steps below for your IDE:

Install the [GitHub Copilot](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot) extension if you haven't already

  <Step title="Open MCP settings">
    Open the Command Palette (`Ctrl + Shift + P` on Windows/Linux, or `Cmd + Shift + P` on Mac) and search for **"MCP: Open User Configuration"**
  </Step>

  <Step title="Add the Galileo MCP server configuration">
    Copy and paste the configuration below. Replace `YOUR-API-KEY` with your actual Galileo API key.

    ```json VSCode MCP Configuration highlight={3-9} theme={null}
    {
      "servers": {
        "galileo_mcp_server": {
          "url": "https://api.galileo.ai/mcp/http/mcp",
          "headers": {
            "Galileo-API-Key": "YOUR-API-KEY",
            "Accept": "text/event-stream"
          }
        }
      },
      "inputs": []
    }
    ```

    This assumes you are using [app.galileo.ai](https://app.galileo.ai).

    If you're using a self-hosted Galileo deployment, replace the `https://api.galileo.ai/mcp/http/mcp` url with your deployment URL. The format of this URL is based on your console URL, replacing `console` with `api` and appending `/mcp/http/mcp`.

    For example:

    * if your console URL is `https://console.galileo.example.com`, the MCP url would be `https://api.galileo.example.com/mcp/http/mcp`
    * if your console URL is `https://console-galileo.apps.mycompany.com`, the MCP url would be `https://api-galileo.apps.mycompany.com/mcp/http/mcp`
  </Step>

  <Step title="Reload VS Code">
    Reload VS Code by opening the Command Palette and running **"Developer: Reload Window"** for the changes to take effect
  </Step>
</Steps>

Open the Cursor command palette (`Ctrl + Shift + P` on Windows/Linux, or `Cmd + Shift + P` on Mac), then choose "Open MCP Settings".

  <Step title="Open the MCP server configuration file">
    Click on **Add Custom MCP** to open the `mcp.json` MCP configuration file, or **New MCP Server** if you already have other MCP servers configured.
  </Step>

  <Step title="Add the Galileo MCP server configuration">
    Copy and paste the configuration below. Replace `YOUR-API-KEY` with your actual Galileo API key.

    ```json Cursor MCP Configuration highlight={3-9} theme={null}
    {
      "mcpServers": {
        "galileo_mcp_server": {
          "url": "https://api.galileo.ai/mcp/http/mcp",
          "headers": {
            "Galileo-API-Key": "YOUR-API-KEY",
            "Accept": "text/event-stream"
          }
        }
      }
    }
    ```

    This assumes you are using [app.galileo.ai](https://app.galileo.ai).

    If you're using a self-hosted Galileo deployment, replace the `https://api.galileo.ai/mcp/http/mcp` url with your deployment URL. The format of this URL is based on your console URL, replacing `console` with `api` and appending `/mcp/http/mcp`.

    For example:

    * if your console URL is `https://console.galileo.example.com`, the MCP url would be `https://api.galileo.example.com/mcp/http/mcp`
    * if your console URL is `https://console-galileo.apps.mycompany.com`, the MCP url would be `https://api-galileo.apps.mycompany.com/mcp/http/mcp`
  </Step>

  <Step title="Save and restart">
    Save the configuration and restart Cursor for the changes to take effect
  </Step>
</Steps>

The configuration is the same for both Cursor and VS Code. Make sure to replace `YOUR-API-KEY` with your actual Galileo API key from the [API keys page](https://app.galileo.ai/settings/api-keys).

Verify your setup

Once configured, you can verify your MCP setup by asking your AI assistant in your IDE:

```text Example Query theme={null} Can you show me how to add Galileo logging to my agent bot? ```

Help me create a synthetic dataset for customer support queries

How do I integrate Galileo with LangChain?

Your AI assistant should now be able to access Galileo's capabilities and respond with information from your Galileo account.

Tools

The Galileo MCP server provides powerful tools that you can access through natural conversation with your AI assistant. Simply ask questions or make requests, and the AI will use these tools to help you.

Generate synthetic datasets or upload your own data to test and evaluate your AI applications. The tool supports creating datasets with various types of queries including general queries, prompt injections, off-topic content, and toxic content scenarios.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  Create a dataset with 50 customer service queries about billing issues
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Generate a dataset of 30 chatbot queries, including some prompt injection attempts
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Make a dataset with product recommendation queries and include off-topic questions
  ```
</CodeGroup>

Track the progress of your dataset generation and preview the generated content. You'll see the first 10 rows of data along with generation status and progress updates.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  Check the status of my dataset that's currently generating
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Show me the preview of dataset abc-123
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Is my customer service dataset ready yet?
  ```
</CodeGroup>

Build reusable prompt templates that you can use across all your projects. Set up model configurations, temperature settings, and other parameters for consistent prompt behavior.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  Create a prompt template called "Friendly Assistant" for customer support responses
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Make a prompt template for summarizing technical documentation with lower temperature
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Set up a chat template for a code review assistant
  ```
</CodeGroup>

Get complete guidance on setting up and running Galileo experiments, including dataset preparation, metrics configuration, and integration with your existing code. Available for Python and other supported languages.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  How do I set up a Galileo experiment in Python?
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Show me how to run an experiment with my RAG application
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Guide me through creating an experiment for my agentic workflow
  ```
</CodeGroup>

Analyze your application's Log streams to identify issues, patterns, and opportunities for improvement. Get specific recommendations based on your logged data.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  What issues do you see in my production log stream?
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Fetch the most recent signals from Galileo and propose fixes for them in my code
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Get insights about my chatbot log stream and suggest improvements
  ```
</CodeGroup>

Get step-by-step integration guides for adding Galileo observability to your OpenAI applications. Automatically log prompts, responses, model parameters, and token usage with minimal code changes.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  How do I add Galileo logging to my OpenAI application?
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Show me how to integrate Galileo with OpenAI in TypeScript
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Help me set up Galileo observability for my GPT-4 chatbot
  ```
</CodeGroup>

Get complete integration instructions for adding Galileo to your LangChain applications. Capture full traces of your chains, agents, and tools with automatic logging.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  How do I integrate Galileo with my LangChain application?
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Show me how to add Galileo tracing to my LangChain agent
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Help me log my LangChain RAG pipeline with Galileo
  ```
</CodeGroup>

Find relevant information, code examples, API references, and implementation guides across all Galileo documentation. Get direct links to the pages you need.

**What you can ask:**

<CodeGroup>
  ```text Query wrap theme={null}
  How do I set up data logging in Galileo?
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Find documentation about custom metrics
  ```
</CodeGroup>

<CodeGroup>
  ```text Query wrap theme={null}
  Search for examples of agentic AI evaluation
  ```
</CodeGroup>

Example use cases

Create a synthetic dataset

Ask your AI assistant:

```text Query wrap theme={null} Create a synthetic dataset with 20 customer service queries about product returns. Include both general queries and off-topic queries. ```

The MCP server will guide you through the dataset creation process and provide a dataset ID to track progress.

Get integration help

Ask your AI assistant:

```text Query wrap theme={null} How do I integrate Galileo with my LangChain application in Python? ```

The MCP server will provide complete integration code examples and setup instructions.

Get Signals

Ask your AI assistant:

```text Query wrap theme={null} Fetch the most recent signals from Galileo and propose fixes for them in my code ```

The MCP server will analyze your Log stream and suggest improvements.

Troubleshooting

* Check that the MCP server URL is set to `https://api.galileo.ai/mcp/http/mcp` * Ensure the `Accept` header is set to `text/event-stream` * Restart your IDE MCP connection after making configuration changes * Confirm your API key is properly set in the configuration * Check that your API key has not expired * Generate a new API key from the [API keys page](https://app.galileo.ai/settings/api-keys)

Next steps

Learn how to log your first trace with Galileo Set up and run experiments to evaluate your AI applications Learn about Galileo integrations with third-party frameworks