tools

作者: astronomer

在编写、编辑或审查此仓库中的命令行工具和辅助脚本(位于bin/目录下)时使用。涵盖工具的存放位置、参数解析等内容……

npx skills add https://github.com/astronomer/astronomer --skill tools

Writing tools in bin/

Repo tooling (setup scripts, one-off utilities, anything a human or CI invokes directly) lives in bin/ and follows a few rules so tools are discoverable, self-documenting, and safe to run by accident.


Critical Rules

  1. Tools live in bin/ as executable scripts: a shebang (#!/usr/bin/env python3 for Python) plus chmod +x. Python tools run via uv run bin/<tool>.py.
  2. Every tool parses arguments with argparse (or the language equivalent) so --help works and every argument is self-documenting. Parse arguments as the first thing main() does.
  3. --help and insufficient/invalid arguments must do no work. They print usage and exit before any side effect. argparse gives this for free as long as parsing happens before any side-effecting code.
  4. A tool must not perform a destructive or state-mutating operation by default. Merely running it (or running it to read --help) must not create/delete Kubernetes objects, write/delete files, call external services, or change the active context.

Non-destructive by default

The failure mode to design against: someone runs bin/some-tool.py (or bin/some-tool.py --help) expecting it to be inert or to print help, and instead it mutates whatever ambient context it finds — the current kube context, the current directory, a live cluster.

The rule that prevents it: do not give a safe-looking default to any argument that determines where a mutation lands (a namespace, a cluster, a path, a target host). Make those arguments required with no default, so a bare or accidental invocation aborts before doing anything.

With argparse, a required=True argument with no default means:

  • tool (no args) → prints usage to stderr and exits non-zero, before main() reaches any side effect.
  • tool --help → prints help and exits 0.
  • tool --namespace foo ... → runs, because the caller was explicit about the target.

Worked example: bin/setup-forgejo-ca.py

This script creates and deletes Kubernetes Secrets in a cluster. It originally defaulted its namespaces (astronomer, git-forgejo). Running bin/setup-forgejo-ca.py --help to read the help text would instead have run the whole thing against the reader's current kube context — a potentially destructive surprise.

The fix was to make the namespaces required, with no defaults:

def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--platform-namespace", required=True, help="...")
    parser.add_argument("--forgejo-namespace", required=True, help="...")
    return parser.parse_args()


def main() -> None:
    args = parse_args()  # aborts here on a bare run or --help, before any kubectl
    ...  # cluster mutations only happen after this line

Now --help and a bare run both abort before touching the cluster, and any real run has to name its target namespaces on purpose.

Automation still works

Making the target arguments required does not break automated callers — it just moves the intent to the caller, where it belongs. The automated invocation passes the values explicitly. For example, the git-sync-private-ca scenario's pre_helm_scripts entry names the namespaces:

pre_helm_scripts:
  - bin/setup-forgejo-ca.py --platform-namespace astronomer --forgejo-namespace git-forgejo

Checklist for a new or edited tool

  • Lives in bin/, is executable, has the right shebang.
  • Uses argparse; --help works and does nothing else.
  • Arguments that decide where a mutation lands are required with no default.
  • Side effects run only after arguments parse successfully.
  • Fails loudly on error (non-zero exit), and is idempotent (safe to re-run) where practical.
  • Callers (CI, scenario manifests, other scripts) pass the required arguments explicitly.

来自 astronomer 的更多技能

airflow-state-store
astronomer
Persists task and asset state across retries and DAG runs using Airflow 3.3's AIP-103 key/value stores (`task_state_store`, `asset_state_store`) and the…
creating-openlineage-extractors
astronomer
针对不受支持的Airflow运算符及复杂血缘场景的自定义OpenLineage提取器。提供两种方案:建议在自有运算符中直接添加OpenLineage方法,或为无法修改的第三方运算符创建自定义提取器。提取器在三个执行节点进行拦截:执行前获取静态血缘、成功后获取运行时输出、可选在失败后获取部分血缘。通过airflow.cfg或环境变量注册提取器...
debugging-dags
astronomer
针对失败的Airflow DAG进行系统性根因分析与修复,提供结构化调查工作流。引导完成四步诊断流程:识别故障、提取错误详情、收集上下文信息、提供可操作的修复步骤。将故障分为四类(数据、代码、基础设施、依赖),以聚焦调查并建议适当的修复方案。提供即用型CLI命令,用于日志检索、运行对比、任务清除及DAG...
delegating-to-otto
astronomer
Drives Astronomer's Otto agent (`astro otto`) as a delegated sub-agent for Airflow, dbt, and data-engineering work. Use when the user explicitly asks to "use…
deploying-airflow
astronomer
部署Airflow DAG和项目。当用户想要部署代码、推送DAG、设置CI/CD、部署到生产环境,或询问部署策略时使用…
deploying-go-sdk-bundles
astronomer
编译、打包并部署已编译的Airflow Go SDK包,以便ExecutableCoordinator能够运行它们。当用户想要编译Go任务包时使用,询问…
testing-dags
astronomer
针对Airflow DAG的迭代式测试-调试-修复循环,提供全面的故障诊断。首先使用af runs trigger-wait <dag_id>运行DAG并等待完成,无需预检。失败时,使用af runs diagnose获取全面的故障摘要,并通过af tasks logs查看特定任务的错误详情。支持自定义配置、超时和重试次数;处理成功、失败和超时场景,并给出清晰的响应解读。提供快速验证功能...
tracing-downstream-lineage
astronomer
追踪下游数据血缘,在修改表或DAG前评估变更影响。通过源代码搜索、视图依赖和BI工具连接识别目标表或DAG的直接消费者,构建完整的依赖树,映射从表到仪表盘再到机器学习模型的所有下游影响。按关键性(关键、高、中、低)对依赖进行分类,以优先安排利益相关者沟通和测试。生成包含风险评估、受影响...的影响报告。