cosmos3-codebase-nav

bởi nvidia

Điều hướng codebase của gói Cosmos3 để tìm nơi lưu trữ các tham số, cấu hình, giá trị mặc định, tập lệnh và tài liệu. Sử dụng khi người dùng hỏi "X ở đâu trong…

npx skills add https://github.com/nvidia/cosmos-framework --skill cosmos3-codebase-nav

Tải ZIP GitHub

491

Dùng thử kỹ năng nàyĐược tài trợ

Cosmos3 Codebase Navigation

When to use this skill

Use this skill when an agent is navigating the Cosmos3 package
Use this skill to answer "where is X", "how do I find the config for Y", or any file-location question
Use this skill when the user opens or edits cosmos3 files and needs orientation

Path convention

All paths below are relative to this file's location (.agents/skills/cosmos3-codebase-nav/). The repo is laid out as:

cosmos_framework/ — main training package (data, model, trainer, callbacks, checkpoint, utils, …).
cosmos_framework/configs/base/experiment/ — vfm (generator) experiment SKUs referenced by [train.train_policy].experiment in the recipe TOMLs.
cosmos_framework/configs/base/reasoner/experiment/ — vlm (reasoner) experiment SKUs.
cosmos_framework/inference/ — inference subpackage (args, model, inference engine, defaults, Ray serving, common helpers).
cosmos_framework/scripts/ — top-level entry-point scripts (train, inference, eval, export_model, convert_model_to_dcp, upsample_prompts, caption_from_video, captions_to_sft_jsonl, action_policy_server, …). Invoked as python -m cosmos_framework.scripts.<name>.
examples/toml/sft_config/<recipe>.toml + examples/launch_sft_<recipe>.sh — paired SFT recipes (training entry-point input). The shell sources examples/_sft_launcher_common.sh, which forwards into cosmos_framework.scripts.train --sft-toml=....
cosmos_framework/configs/toml_config/ — pydantic schemas (sft_config.py) and helpers that validate the recipe TOML at load time.

Quick Reference

Where parameters and defaults live

What you're looking for	File
Sampling params (num_steps, guidance, shift, fps, etc.)	`../../../cosmos_framework/inference/args.py` → `SamplingArgs`, `SamplingOverrides`
Per-modality default values	`../../../cosmos_framework/inference/defaults/<mode>/sample_args.json`
Setup params (parallelism, checkpoints, model path)	`../../../cosmos_framework/inference/args.py` → `OmniSetupArgs`, `OmniSetupOverrides`
Common args base classes	`../../../cosmos_framework/inference/common/args.py` → `ArgsBase`, `OverridesBase`
Ray serving parallelism presets	`../../../cosmos_framework/inference/ray/configs/latency.yaml`, `../../../cosmos_framework/inference/ray/configs/throughput.yaml`
Feature flags	`../../../cosmos_framework/utils/flags.py`
Prompt upsampler system prompt	`../../../cosmos_framework/inference/defaults/prompt_upsampler.txt`
Video captioner system prompt	`../../../cosmos_framework/inference/defaults/video_captioner.txt`
SFT recipe TOMLs (paired with `examples/launch_sft_*.sh`)	`../../../examples/toml/sft_config/<recipe>.toml`
SFT pydantic schema (validates the recipe TOML)	`../../../cosmos_framework/configs/toml_config/sft_config.py`
Training experiment SKUs (vfm)	`../../../cosmos_framework/configs/base/experiment/`
Training experiment SKUs (vlm / reasoner)	`../../../cosmos_framework/configs/base/reasoner/experiment/`
Example inputs	`../../../inputs/omni/t2i.json`, `../../../inputs/omni/t2v.json`, `../../../inputs/omni/i2v.json`, …

Available modality modes for defaults: text2image, text2video, image2video, image2image, video2video, forward_dynamics, inverse_dynamics, wam.

Config defaults resolution chain

When a user runs inference, default parameter values are resolved in this order:

cosmos_framework/inference/defaults/<mode>/sample_args.json     # 1. Per-modality JSON defaults (num_steps, guidance, shift, fps, etc.)
        ↓
_load_modality_defaults() in cosmos_framework/inference/args.py # 2. Loaded and cached at import time
        ↓
SamplingArgs / SamplingOverrides                      # 3. Pydantic models with field-level validation
        ↓
OmniSampleOverrides.build_sample()                    # 4. Merges user overrides → final resolved args
        ↓
_RESOLUTION_SHIFT_DEFAULTS[model_size, resolution]    # 5. Model+resolution shift override (if user didn't set shift)
        ↓
CLI flags (--guidance, --shift, etc.)                 # 6. User overrides from command line

The _RESOLUTION_SHIFT_DEFAULTS table in ../../../cosmos_framework/inference/args.py (on OmniSampleOverrides) overrides the default shift based on model size and resolution, unless the user explicitly specified --shift.

Mode	Default file	Key defaults
`text2image`	`../../../cosmos_framework/inference/defaults/text2image/sample_args.json`	`num_frames=1`, `guidance=6.0`, `shift=10.0`
`text2video`	`../../../cosmos_framework/inference/defaults/text2video/sample_args.json`	`num_frames=189`, `guidance=6.0`, `shift=10.0`
`image2video`	`../../../cosmos_framework/inference/defaults/image2video/sample_args.json`	`num_frames=189`, `guidance=6.0`, `shift=10.0`

Action and video2video modes also have defaults under cosmos_framework/inference/defaults/{image2image,video2video,forward_dynamics,inverse_dynamics,policy}/sample_args.json.

Users can also supply a custom defaults file per-request via the defaults_file field in sample arguments (see ../../../docs/inference.md).

Where to make changes

Task	Edit
Change a built-in default value	`../../../cosmos_framework/inference/defaults/<mode>/sample_args.json`
Add a new CLI parameter	`SamplingArgs` + `SamplingOverrides` in `../../../cosmos_framework/inference/args.py`, then add to each `sample_args.json`
Change parallelism presets	`../../../cosmos_framework/inference/ray/configs/latency.yaml` or `throughput.yaml`
Add a new script	`../../../cosmos_framework/scripts/` — follow `inference.py` as the pattern

Key entry points

Entry point	How to run
Batch inference	`python -m cosmos_framework.scripts.inference`
Training	`python -m cosmos_framework.scripts.train --sft-toml=examples/toml/sft_config/<recipe>.toml`
Online serving (Ray)	`python -m cosmos_framework.inference.ray.serve`
Submit to Ray server	`python -m cosmos_framework.inference.ray.submit`
Gradio UI	`python -m cosmos_framework.inference.ray.gradio`
Prompt upsampling	`python -m cosmos_framework.scripts.upsample_prompts`
Model export (HF)	`python -m cosmos_framework.scripts.export_model`
DCP conversion	`python -m cosmos_framework.scripts.convert_model_to_dcp`
Diffusers conversion	`python -m cosmos_framework.scripts.convert_model_to_diffusers`
Video captioning	`python -m cosmos_framework.scripts.caption_from_video`
Captions → SFT JSONL	`python -m cosmos_framework.scripts.captions_to_sft_jsonl`
Action policy server (LIBERO HTTP)	`python -m cosmos_framework.scripts.action_policy_server_libero`
Action policy server (RoboLab WS)	`python -m cosmos_framework.scripts.action_policy_server_robolab`

Documentation

Doc	Covers
`../../../AGENTS.md`	Commands, rules, key file locations (read this first)
`../../../README.md`	Overview, quickstart, examples
`../../../docs/setup.md`	Installation, environment, checkpoints
`../../../docs/code_structure.md`	Repo layout and per-subpackage tour of `cosmos_framework/`
`../../../docs/inference.md`	Sample args, default values, custom defaults
`../../../docs/training.md`	SFT / post-training workflow
`../../../docs/faq.md`	FAQ, tips, and troubleshooting