Tiktoken MCP

Count tokens using OpenAI's tiktoken library.

GitHub

tiktoken-mcp

An MCP (Model Context Protocol) server that provides token counting functionality using OpenAI's tiktoken library.

Features

  • 🔢 Token Counting: Count tokens for any text using various tiktoken encodings
  • 💰 Cost Estimation: Estimate token costs for popular AI models (GPT-4, Claude, etc.)
  • 🚀 Easy to Use: Run directly with npx - no installation required
  • 🔧 Claude Desktop Integration: Seamlessly integrates with Claude Desktop app

Installation

You can run this MCP server directly using npx:

npx tiktoken-mcp

Or install it globally:

npm install -g tiktoken-mcp

Prerequisites

  • Node.js 16 or higher
  • Python 3.6 or higher
  • pip (Python package manager)

The tiktoken Python library will be automatically installed when you first run the package.

Available Tools

count_tokens

Count tokens in text using various tiktoken encodings.

Parameters:

  • text (required): The text to count tokens for
  • model (optional): The encoding model to use. Options: 'cl100k_base' (default), 'p50k_base', 'p50k_edit', 'r50k_base'

estimate_cost

Estimate token costs for various AI models.

Parameters:

  • text (required): The text to estimate cost for
  • model (required): The model to estimate cost for. Options: 'gpt-4', 'gpt-4-turbo', 'gpt-3.5-turbo', 'claude-3-opus', 'claude-3-sonnet', 'claude-3-haiku'

Claude Desktop Configuration

To use this MCP server with Claude Desktop, add the following to your Claude configuration file:

macOS

Location: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows

Location: %APPDATA%\Claude\claude_desktop_config.json

Configuration

{
  "mcpServers": {
    "tiktoken": {
      "command": "npx",
      "args": ["tiktoken-mcp"]
    }
  }
}

After adding this configuration, restart Claude Desktop. You should then be able to use the token counting tools in your conversations.

Example Usage in Claude

Once configured, you can use the tools like this:

  • "Count the tokens in this text: 'Hello, world!'"
  • "Estimate the cost of this prompt for GPT-4"
  • "How many tokens are in this paragraph using the cl100k_base encoding?"

Example Output

Token Counting:

Token count: 13
Model: cl100k_base
First 100 token IDs: 9906, 11, 1917, 0, 498, 1097, 701, 30, 220, 17, ...

Cost Estimation:

Token count: 150
Model: gpt-4
Estimated input cost: $0.004500
Estimated output cost: $0.009000
Total cost (if used as both input and output): $0.013500

How It Works

This MCP server uses Python's tiktoken library under the hood to accurately count tokens. It supports multiple encoding models:

  • cl100k_base: Used by GPT-4, GPT-3.5-turbo, and newer models
  • p50k_base: Used by Codex models
  • p50k_edit: Used by edit models
  • r50k_base: Used by GPT-3 models

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Issues

If you encounter any problems, please file an issue on the GitHub repository.

License

MIT

Serveurs connexes