A unified framework for integrating various language models and embedding providers to generate text completions and embeddings.
✨ AiCore is a comprehensive framework for integrating various language models and embedding providers with a unified interface. It supports both synchronous and asynchronous operations for generating text completions and embeddings, featuring:
🔌 Multi-provider support: OpenAI, Mistral, Groq, Gemini, NVIDIA, and more
🤖 Reasoning augmentation: Enhance traditional LLMs with reasoning capabilities
📊 Observability: Built-in monitoring and analytics
💰 Token tracking: Detailed usage metrics and cost tracking
⚡ Flexible deployment: Chainlit, FastAPI, and standalone script support
🛠️ MCP Integration: Connect to Model Control Protocol servers via tool calling
pip install git+https://github.com/BrunoV21/AiCore
or
pip install git+https://github.com/BrunoV21/AiCore.git#egg=core-for-ai[all]
or
pip install core-for-ai[all]
from aicore.llm import Llm
from aicore.llm.config import LlmConfig
import os
llm_config = LlmConfig(
provider="openai",
model="gpt-4o",
api_key="super_secret_openai_key"
)
llm = Llm.from_config(llm_config)
# Generate completion
response = llm.complete("Hello, how are you?")
print(response)
from aicore.llm import Llm
from aicore.llm.config import LlmConfig
import os
async def main():
llm_config = LlmConfig(
provider="openai",
model="gpt-4o",
api_key="super_secret_openai_key"
)
llm = Llm.from_config(llm_config)
# Generate completion
response = await llm.acomplete("Hello, how are you?")
print(response)
if __name__ == "__main__":
asyncio.run(main())
more examples available at examples/ and docs/exampes/
LLM Providers:
Embedding Providers:
Observability Tools:
MCP Integration:
To configure the application for testing, you need to set up a config.yml
file with the necessary API keys and model names for each provider you intend to use. The CONFIG_PATH
environment variable should point to the location of this file. Here's an example of how to set up the config.yml
file:
# config.yml
embeddings:
provider: "openai" # or "mistral", "groq", "gemini", "nvidia"
api_key: "your_openai_api_key"
model: "text-embedding-3-small" # Optional
llm:
provider: "openai" # or "mistral", "groq", "gemini", "nvidia"
api_key: "your_openai_api_key"
model: "gpt-o4" # Optional
temperature: 0.1
max_tokens: 1028
reasonning_effort: "high"
mcp_config_path: "./mcp_config.json" # Path to MCP configuration
max_tool_calls_per_response: 3 # Optional limit on tool calls
config examples for the multiple providers are included in the config dir
from aicore.llm import Llm
from aicore.config import Config
import asyncio
async def main():
# Load configuration with MCP settings
config = Config.from_yaml("./config/config_example_mcp.yml")
# Initialize LLM with MCP capabilities
llm = Llm.from_config(config.llm)
# Make async request that can use MCP-connected tools
response = await llm.acomplete(
"Search for latest news about AI advancements",
system_prompt="Use available tools to gather information"
)
print(response)
asyncio.run(main())
Example MCP configuration (mcp_config.json
):
{
"mcpServers": {
"search-server": {
"transport_type": "ws",
"url": "ws://localhost:8080",
"description": "WebSocket server for search functionality"
},
"data-server": {
"transport_type": "stdio",
"command": "python",
"args": ["data_server.py"],
"description": "Local data processing server"
},
"brave-search": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-brave-search"
],
"env": {
"BRAVE_API_KEY": "SUPER-SECRET-BRAVE-SEARCH-API-KEY"
}
}
}
}
You can use the language models to generate text completions. Below is an example of how to use the MistralLlm
provider:
from aicore.llm.config import LlmConfig
from aicore.llm.providers import MistralLlm
config = LlmConfig(
api_key="your_api_key",
model="your_model_name",
temperature=0.7,
max_tokens=100
)
mistral_llm = MistralLlm.from_config(config)
response = mistral_llm.complete(prompt="Hello, how are you?")
print(response)
To load configurations from a YAML file, set the CONFIG_PATH
environment variable and use the Config
class to load the configurations. Here is an example:
from aicore.config import Config
from aicore.llm import Llm
import os
if __name__ == "__main__":
os.environ["CONFIG_PATH"] = "./config/config.yml"
config = Config.from_yaml()
llm = Llm.from_config(config.llm)
llm.complete("Once upon a time, there was a")
Make sure your config.yml
file is properly set up with the necessary configurations.
AiCore includes a comprehensive observability module that tracks:
Key metrics tracked:
from aicore.observability import ObservabilityDashboard
dashboard = ObservabilityDashboard(storage="observability_data.json")
dashboard.run_server(port=8050)
Reasoner Augmented Config
AiCore also contains native support to augment traditional Llms with reasoning capabilities by providing them with the thinking steps generated by an open-source reasoning capable model, allowing it to generate its answers in a Reasoning Augmented way.
This can be usefull in multiple scenarios, such as:
To leverage the reasoning augmentation just introduce one of the supported llm configs into the reasoner field and AiCore handles the rest
# config.yml
embeddings:
provider: "openai" # or "mistral", "groq", "gemini", "nvidia"
api_key: "your_openai_api_key"
model: "your_openai_embedding_model" # Optional
llm:
provider: "mistral" # or "openai", "groq", "gemini", "nvidia"
api_key: "your_mistral_api_key"
model: "mistral-small-latest" # Optional
temperature: 0.6
max_tokens: 2048
reasoner:
provider: "groq" # or openrouter or nvidia
api_key: "your_groq_api_key"
model: "deepseek-r1-distill-llama-70b" # or "deepseek/deepseek-r1:free" or "deepseek/deepseek-r1"
temperature: 0.5
max_tokens: 1024
A Hugging Face Space showcasing reasoning-augmented models
Instant summaries of Git activity
🌐 Live App
📦 GitHub Repository
Graph representation of codebases for enhanced retrieval
For complete documentation, including API references, advanced usage examples, and configuration guides, visit:
This project is licensed under the Apache 2.0 License.
Enables AI agents and assistants to interact with Visual Studio Code through the Model Context Protocol.
Explore and interact with Swagger/OpenAPI specifications, allowing for browsing endpoints and retrieving details on API operations.
Interact with Gibson projects to create/update projects, explain database/API interactions, and write code within your IDE.
Access WordPress development rules and best practices from the WordPress LLM Rules repository. It dynamically creates tools for each rule and caches content using Cloudflare Durable Objects.
Securely execute shell commands with whitelisting, resource limits, and timeout controls for LLMs.
A TypeScript library for integrating MCP with tools like LangChain and Zod, providing helpers for schema conversion and event streaming.
Integrates Ollama's local LLM models with MCP-compatible applications. Requires a local Ollama installation.
A Neo4j-based MCP server providing persistent memory and contextual assistance for Claude Code.
Provides preset prompt templates as tools to assist clients like Cursor and Windsurf in executing various tasks.
Extracts images from files, URLs, or base64 strings and converts them to base64 for LLM analysis.