setting-up-astro-project

作者: astronomer

使用依赖项、连接和环境设置初始化并配置Astro/Airflow项目。通过astro dev init搭建完整的项目结构,包括DAGs、插件、测试和配置文件的目录。通过requirements.txt和packages.txt管理Python和操作系统级依赖项,支持自定义Dockerfile以应对复杂配置。在airflow_settings.yaml中以声明方式配置连接、变量和连接池,并提供用于环境导出/导入的命令...

npx skills add https://github.com/astronomer/agents --skill setting-up-astro-project

Astro Project Setup

This skill helps you initialize and configure Airflow projects using the Astro CLI.

To run the local environment, see the managing-astro-local-env skill. To write DAGs, see the authoring-dags skill. Open-source alternative: If the user isn't on Astro, guide them to Apache Airflow's Docker Compose quickstart for local dev and the Helm chart for production. For deployment strategies, use the deploying-airflow skill.


Initialize a New Project

astro dev init

Don't pass --airflow-version or --runtime-version unless the user explicitly asks for a specific pin. Plain astro dev init resolves to the latest Astro Runtime — that's the right default. Specifying a version risks pinning to a stale value from training data. If the user wants to know what was installed, read the generated Dockerfile afterward instead of guessing.

Creates this structure:

project/
├── dags/                # DAG files
├── include/             # SQL, configs, supporting files
├── plugins/             # Custom Airflow plugins
├── tests/               # Unit tests
├── Dockerfile           # Image customization
├── packages.txt         # OS-level packages
├── requirements.txt     # Python packages
└── airflow_settings.yaml # Connections, variables, pools

Adding Dependencies

Python Packages (requirements.txt)

apache-airflow-providers-snowflake==5.3.0
pandas==2.1.0
requests>=2.28.0

OS Packages (packages.txt)

gcc
libpq-dev

Custom Dockerfile

For complex setups (private PyPI, custom scripts):

FROM quay.io/astronomer/astro-runtime:12.4.0

RUN pip install --extra-index-url https://pypi.example.com/simple my-package

After modifying dependencies: Run astro dev restart


Configuring Connections & Variables

airflow_settings.yaml

Loaded automatically on environment start:

airflow:
  connections:
    - conn_id: my_postgres
      conn_type: postgres
      host: host.docker.internal
      port: 5432
      login: user
      password: pass
      schema: mydb

  variables:
    - variable_name: env
      variable_value: dev

  pools:
    - pool_name: limited_pool
      pool_slot: 5

Export/Import

# Export from running environment
astro dev object export --connections --file connections.yaml

# Import to environment
astro dev object import --connections --file connections.yaml

Validate Before Running

Parse DAGs to catch errors without starting the full environment:

astro dev parse

Related Skills

  • managing-astro-local-env: Start, stop, and troubleshoot the local environment
  • authoring-dags: Write and validate DAGs (uses MCP tools)
  • testing-dags: Test DAGs (uses MCP tools)
  • deploying-airflow: Deploy DAGs to production (Astro, Docker Compose, Kubernetes)

来自 astronomer 的更多技能

airflow-state-store
astronomer
Persists task and asset state across retries and DAG runs using Airflow 3.3's AIP-103 key/value stores (`task_state_store`, `asset_state_store`) and the…
creating-openlineage-extractors
astronomer
针对不受支持的Airflow运算符及复杂血缘场景的自定义OpenLineage提取器。提供两种方案:建议在自有运算符中直接添加OpenLineage方法,或为无法修改的第三方运算符创建自定义提取器。提取器在三个执行节点进行拦截:执行前获取静态血缘、成功后获取运行时输出、可选在失败后获取部分血缘。通过airflow.cfg或环境变量注册提取器...
debugging-dags
astronomer
针对失败的Airflow DAG进行系统性根因分析与修复,提供结构化调查工作流。引导完成四步诊断流程:识别故障、提取错误详情、收集上下文信息、提供可操作的修复步骤。将故障分为四类(数据、代码、基础设施、依赖),以聚焦调查并建议适当的修复方案。提供即用型CLI命令,用于日志检索、运行对比、任务清除及DAG...
delegating-to-otto
astronomer
Drives Astronomer's Otto agent (`astro otto`) as a delegated sub-agent for Airflow, dbt, and data-engineering work. Use when the user explicitly asks to "use…
deploying-airflow
astronomer
部署Airflow DAG和项目。当用户想要部署代码、推送DAG、设置CI/CD、部署到生产环境,或询问部署策略时使用…
deploying-go-sdk-bundles
astronomer
编译、打包并部署已编译的Airflow Go SDK包,以便ExecutableCoordinator能够运行它们。当用户想要编译Go任务包时使用,询问…
testing-dags
astronomer
针对Airflow DAG的迭代式测试-调试-修复循环,提供全面的故障诊断。首先使用af runs trigger-wait <dag_id>运行DAG并等待完成,无需预检。失败时,使用af runs diagnose获取全面的故障摘要,并通过af tasks logs查看特定任务的错误详情。支持自定义配置、超时和重试次数;处理成功、失败和超时场景,并给出清晰的响应解读。提供快速验证功能...
tracing-downstream-lineage
astronomer
追踪下游数据血缘,在修改表或DAG前评估变更影响。通过源代码搜索、视图依赖和BI工具连接识别目标表或DAG的直接消费者,构建完整的依赖树,映射从表到仪表盘再到机器学习模型的所有下游影响。按关键性(关键、高、中、低)对依赖进行分类,以优先安排利益相关者沟通和测试。生成包含风险评估、受影响...的影响报告。