compileiq-booster-pack

compileiq-booster-pack

Try curated .acf candidates before running a full CompileIQ search. A Booster Pack is a zip of ACFs that NVIDIA validated against a specific workload family. They are not guaranteed speedups; treat every candidate as workload-specific and validate it on your own benchmark.

Authoritative narrative: docs/booster_packs.md, docs/flashinfer_booster.md.

When

Available packs (today)

The public release shape is documented in docs/booster_packs.md. Read each candidate's compiler_stages from its pack manifest and use every listed stage. There is no runtime download API today. Don't invent one.

Steps

0. Pre-flight: the O0/O3 ACF-injection canary (mandatory first step)

The most common silent failure when applying ACFs is a framework cache (Triton, Helion, FlashInfer's flashinfer_cubin/flashinfer_jit_cache, NVCC build cache) serving a stale binary that ignored the ACF. The Debug pack has two ACFs with predictable, opposite-direction signatures:

• ptxas_opt0.acf: forces unoptimized PTXAS compilation. Applied → expect a measurable regression (often 2-10x slower) vs. baseline.
• ptxas_opt3.acf: forces the default PTXAS optimization level. Applied → expect to match baseline (the no-ACF default is already -O3).

# Baseline
T_BASE_MS=$(./run-benchmark.sh)

# O0 must regress
PTXAS_OPTIONS="--apply-controls=booster-pack-debug/ptxas_opt0.acf" T_O0_MS=$(./run-benchmark.sh)

# O3 must match baseline
PTXAS_OPTIONS="--apply-controls=booster-pack-debug/ptxas_opt3.acf" T_O3_MS=$(./run-benchmark.sh)

python -c "
import sys
base, o0, o3 = $T_BASE_MS, $T_O0_MS, $T_O3_MS
if o0 < base * 1.05:
    print('FAIL: O0 did not regress; ACF is NOT reaching PTXAS. Fix the cache-bust.')
    sys.exit(1)
if abs(o3 - base) / base > 0.05:
    print(f'WARN: O3 differs from baseline by >5%; baseline may not be -O3 or framework caching differs.')
print('PASS: ACF injection is wired up correctly.')
"

If this fails, stop. Fix the cache-bust before trying any real pack candidate:

• Triton: export TRITON_ALWAYS_COMPILE=1, unique TRITON_CACHE_DIR per eval.
• Helion: export HELION_SKIP_CACHE=1.
• FlashInfer: confirm flashinfer_cubin and flashinfer_jit_cache packages are absent (docs/flashinfer_booster.md:56-64).
• Raw nvcc: clean the build dir between candidates.

1. Download

Browse https://github.com/NVIDIA/CompileIQ/releases, find the latest tag matching booster-packs-*, and download the relevant pack zip plus the top-level booster-pack-catalog.json.

BOOSTER_TAG="$(gh release list -R NVIDIA/CompileIQ --limit 100 --json tagName,isDraft \
  --jq '.[] | select(.isDraft == false) | select(.tagName | startswith("booster-packs-")) | .tagName' \
  | head -n 1)"
echo "Using $BOOSTER_TAG"
gh release download "$BOOSTER_TAG" -R NVIDIA/CompileIQ -p 'booster-pack-helion.zip' -p 'booster-pack-catalog.json' -D ./packs
unzip ./packs/booster-pack-helion.zip -d ./packs
cat ./packs/booster-pack-helion/booster-pack-manifest.json

Always read the per-pack manifest before applying: it lists the intended workload, compiler version, GPU target, validation context, and known caveats. For a reproducible rerun, set BOOSTER_TAG to the exact tag printed above.

2. Apply one ACF at a time

Apply exactly one ACF per run. If it fails to compile, hangs, crashes, returns wrong answers, or regresses, reject that candidate and move to the next.

3. Validate every candidate

• Compare against a known-good reference (correctness, not just speed).
• Test multiple input shapes when shape matters.
• Use compile and runtime timeouts to bound runaway candidates.
• Run multiple performance trials if the benchmark is noisy.
• Record the reproducibility checklist below (one row per candidate).

4. Reproducibility log

For every candidate you accept or reject, append a row to booster-pack-log.csv with:

• ACF filename (and sha256)
• Manifest / release version
• Benchmark command
• GPU model + driver version
• CTK version
• nvcc and ptxas paths + versions
• Framework version or commit
• Input shape
• Baseline result (mean ± std)
• Candidate result (mean ± std)
• Correctness status
• Decision: KEPT or REJECTED:<reason>

This is the same checklist docs/flashinfer_booster.md:135-148 recommends. The scripts/apply_one_acf.sh helper does most of this automatically.

Self-test

bash scripts/apply_one_acf.sh --self-test

Dry-runs a "baseline vs baseline" comparison (no ACF applied to either side) and confirms the helper correctly reports "NOT a real improvement". Catches misconfigured script invocations before they pollute the reproducibility log.

Gotchas

• Pack name is not a hard boundary. Helion Pack helps some FlashInfer cases (docs/booster_packs.md:34); test before assuming.
• Booster Packs are not search-space inputs. Don't try to feed an ACF through PtxasSearchSpace(...); packs are already-generated .acf candidate bundles, not inputs to PtxasSearchSpace or NvccSearchSpace.
• Force recompilation. If you can't prove a recompile happened between candidates, don't trust the measurement. See the cache-bust hints under the pre-flight canary.

Next

• If no pack candidate helps your workload, go to compileiq-run-search for a full CompileIQ search over PtxasSearchSpace().
• For attention workloads specifically, also see compileiq-search-space (variant="att").