tools

作者: astronomer

在編寫、編輯或審查此儲存庫中的命令列工具和輔助腳本(位於 bin/ 目錄內)時使用。涵蓋工具存放位置、參數解析等內容…

npx skills add https://github.com/astronomer/astronomer --skill tools

Writing tools in bin/

Repo tooling (setup scripts, one-off utilities, anything a human or CI invokes directly) lives in bin/ and follows a few rules so tools are discoverable, self-documenting, and safe to run by accident.


Critical Rules

  1. Tools live in bin/ as executable scripts: a shebang (#!/usr/bin/env python3 for Python) plus chmod +x. Python tools run via uv run bin/<tool>.py.
  2. Every tool parses arguments with argparse (or the language equivalent) so --help works and every argument is self-documenting. Parse arguments as the first thing main() does.
  3. --help and insufficient/invalid arguments must do no work. They print usage and exit before any side effect. argparse gives this for free as long as parsing happens before any side-effecting code.
  4. A tool must not perform a destructive or state-mutating operation by default. Merely running it (or running it to read --help) must not create/delete Kubernetes objects, write/delete files, call external services, or change the active context.

Non-destructive by default

The failure mode to design against: someone runs bin/some-tool.py (or bin/some-tool.py --help) expecting it to be inert or to print help, and instead it mutates whatever ambient context it finds — the current kube context, the current directory, a live cluster.

The rule that prevents it: do not give a safe-looking default to any argument that determines where a mutation lands (a namespace, a cluster, a path, a target host). Make those arguments required with no default, so a bare or accidental invocation aborts before doing anything.

With argparse, a required=True argument with no default means:

  • tool (no args) → prints usage to stderr and exits non-zero, before main() reaches any side effect.
  • tool --help → prints help and exits 0.
  • tool --namespace foo ... → runs, because the caller was explicit about the target.

Worked example: bin/setup-forgejo-ca.py

This script creates and deletes Kubernetes Secrets in a cluster. It originally defaulted its namespaces (astronomer, git-forgejo). Running bin/setup-forgejo-ca.py --help to read the help text would instead have run the whole thing against the reader's current kube context — a potentially destructive surprise.

The fix was to make the namespaces required, with no defaults:

def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--platform-namespace", required=True, help="...")
    parser.add_argument("--forgejo-namespace", required=True, help="...")
    return parser.parse_args()


def main() -> None:
    args = parse_args()  # aborts here on a bare run or --help, before any kubectl
    ...  # cluster mutations only happen after this line

Now --help and a bare run both abort before touching the cluster, and any real run has to name its target namespaces on purpose.

Automation still works

Making the target arguments required does not break automated callers — it just moves the intent to the caller, where it belongs. The automated invocation passes the values explicitly. For example, the git-sync-private-ca scenario's pre_helm_scripts entry names the namespaces:

pre_helm_scripts:
  - bin/setup-forgejo-ca.py --platform-namespace astronomer --forgejo-namespace git-forgejo

Checklist for a new or edited tool

  • Lives in bin/, is executable, has the right shebang.
  • Uses argparse; --help works and does nothing else.
  • Arguments that decide where a mutation lands are required with no default.
  • Side effects run only after arguments parse successfully.
  • Fails loudly on error (non-zero exit), and is idempotent (safe to re-run) where practical.
  • Callers (CI, scenario manifests, other scripts) pass the required arguments explicitly.

來自 astronomer 的更多技能

airflow-state-store
astronomer
Persists task and asset state across retries and DAG runs using Airflow 3.3's AIP-103 key/value stores (`task_state_store`, `asset_state_store`) and the…
creating-openlineage-extractors
astronomer
為不支援的Airflow運算子及複雜血緣場景設計的自訂OpenLineage提取器。提供兩種方法:直接在你擁有的運算子中加入OpenLineage方法(建議做法),或為無法修改的第三方運算子建立自訂提取器。提取器在三個時間點攔截運算子執行:執行前取得靜態血緣、成功後取得執行階段決定的輸出、以及選擇性地在失敗後取得部分血緣。可透過airflow.cfg或環境變數註冊提取器...
debugging-dags
astronomer
針對失敗的 Airflow DAG 進行系統性根本原因分析與修復,並提供結構化的調查流程。引導完成四個階段的診斷步驟:識別失敗、提取錯誤細節、收集背景資訊,以及提供可行的修復步驟。將失敗分為四種類型(資料、程式碼、基礎設施、相依性),以聚焦調查並建議適當的修正方式。提供可直接使用的 CLI 指令,用於日誌擷取、執行比較、任務清除與 DAG...
delegating-to-otto
astronomer
驅動 Astronomer 的 Otto 代理
deploying-airflow
astronomer
部署 Airflow DAG 和專案。當使用者想要部署程式碼、推送 DAG、設定 CI/CD、部署到生產環境,或詢問部署策略時使用…
deploying-go-sdk-bundles
astronomer
建置、打包並部署已編譯的 Airflow Go SDK 套件,以便 ExecutableCoordinator 能執行它們。當使用者想要編譯 Go 任務套件、要求…時使用。
testing-dags
astronomer
針對Airflow DAG的反覆測試-除錯-修復循環,提供全面的失敗診斷。從af runs trigger-wait <dag_id>開始執行DAG並等待完成,無需預先檢查。失敗時,使用af runs diagnose獲取完整的失敗摘要,並透過af tasks logs檢查特定任務的錯誤細節。支援自訂配置、超時設定與重試機制;能處理成功、失敗及超時情境,並提供清晰的回應解讀。快速驗證功能亦已就緒...
tracing-downstream-lineage
astronomer
追蹤下游資料血緣,在修改資料表或DAG前評估變更影響。透過原始碼搜尋、檢視相依性及BI工具連線,識別目標資料表或DAG的直接消費者。建立完整的相依性樹狀圖,繪製從資料表到儀表板再到機器學習模型的所有下游影響。依關鍵性(關鍵、高、中、低)分類相依性,以優先處理利害關係人溝通與測試。產出包含風險評估、受影響範圍的影響報告。