cosmos3-inference

Guide users through running Cosmos3 inference — offline batch generation, online serving with Ray and Gradio, parallelism options, input formats, sampling…

Cosmos3 Inference

When to use this skill

Use when a user wants to generate images or videos with Cosmos3
Use when a user asks about inference parameters, input formats, or parallelism
Use when a user wants to set up online serving (Ray Serve, Gradio)
Use when a user asks about prompt engineering or upsampling
For environment or import errors, hand off to cosmos3-env-troubleshoot

Path convention

All paths below are relative to the cosmos3 package root (../../../ from this skill file). All uv run / python commands should also be run from there.

Where to find answers

User question	Go to
How do I run inference? (single-GPU, multi-GPU)	`README.md` § Inference
Which model should I use? (Nano vs Super, memory, shift)	`README.md` § Models
Which modality? (t2i, t2v, i2v, examples)	`README.md` § Modalities
What parallelism preset? (latency vs throughput)	`README.md` § Inference
What input fields are available? (prompt, vision_path, num_frames, ...)	`docs/inference.md` § Sample Arguments
What are the default parameter values?	`cosmos_framework/inference/defaults/<model_mode>/sample_args.json` (per-modality JSON)
How do I use custom defaults?	`docs/inference.md` § Custom Defaults
How do I override a parameter? (precedence)	`docs/faq.md` § How do I override a default parameter?
What is the shift parameter?	`docs/faq.md` § What is the `shift` parameter?
How many frames can I generate? (resolution caps)	`docs/faq.md` § How many frames can I generate?
How do I start Ray Serve / Gradio / submit requests?	`docs/faq.md` § How do I run online inference with Ray?
How do I upsample short prompts?	`docs/faq.md` § Prompt upsampling
How do I use the low-level API? (examples/)	`examples/inference.py` (model API) / `examples/inference_pipeline.py` (pipeline API)
All CLI flags	`uv run --all-extras --group=cu130 python -m cosmos_framework.scripts.inference --help`

Things not obvious from the docs

Path resolution: relative paths in input JSON files are resolved relative to the JSON file's directory, not the working directory.
Seed: always pass --seed for reproducible results. Without it, a random seed is used each time.
Resume: interrupted runs can be resumed by re-running the same command — existing outputs are skipped automatically.
--keep-going: continues processing remaining samples after a per-sample failure (e.g. guardrail rejection). Used in online serving by default.
Unique names: every sample in a run must have a unique name field, or the script will error.

Related skills

Skill	When to use
`../cosmos3-setup/SKILL.md`	Installation and environment setup
`../cosmos3-codebase-nav/SKILL.md`	Finding files, parameters, and configs in code
`../cosmos3-env-troubleshoot/SKILL.md`	Debugging environment and runtime errors