apache / apache/airflow

[AIP-94] airflowctl command migration

Open
#68,402 14 comments 0 reactions 0 assignees View on GitHub
area:airflow-ctl area:CLI kind:feature kind:meta
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Body

**Goal:** move every *remote* `airflow` CLI command (anything backed by a public REST API endpoint)
into `airflowctl`. Admin commands have no API and are local-only. They run components (scheduler,
api-server, triggerer, dag-processor, kerberos, standalone), manage the DB, or reach internals
directly (DAG parsing/rendering, local provider manager). Those remain in the `airflow` CLI and are
out of scope here.

**Why:**
[AIP-81: Enhanced Security in CLI via Integration of API](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-81+Enhanced+Security+in+CLI+via+Integration+of+API)
and its sub-proposal
[AIP-94: Decouple Remote Commands from the airflow CLI](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=382175838).

**Status: converted**. The only work per command now is to add the marker decorator to the
listed commands so they are frozen for development. There is no airflowctl migration to
do.

What to do:

- Add the marker decorator to each command on the list. See PR #68726 for how the
decorator is converted to a maintainer and developer facing marker (it replaces the
user facing deprecation warning). Apply that same decorator to each command.
- The decorator is maintainer and developer facing only. It records the airflowctl
equivalent and adds a note in the code, and it does not print anything to users or
change behavior.
- The point of the marker is to freeze these commands for development. Once a command is
marked, no new features or behavior changes should land on it.

**Admin (no API, local-only, not migrated, stay in the `airflow` CLI):**

- `dags`: `show`, `show-dependencies`, `test`, `reserialize` (parse/render DAGs directly)
- `tasks`: `test`, `render` (execute/render DAG files directly)
- `providers`: `links`, `widgets`, `hooks`, `triggers`, `behaviours`, `logging`, `secrets`,
`executors`, `queues`, `notifications`, `configs`, `lazy-loaded`, `auth-managers` (read the local
provider manager)

## dags

- [x] `dags trigger` #68175
- [x] `dags delete` #68175
- [ ] `dags list`
- [x] `dags details` #68529
- [ ] `dags state`
- [ ] `dags list-runs`
- [ ] `dags list-jobs`
- [x] `dags list-import-errors` #68602
- [ ] `dags report`
- [ ] `dags clear`
- [ ] `dags pause`
- [ ] `dags unpause`
- [ ] `dags next-execution`

## tasks

- [ ] operations for `tasks`
- [ ] `tasks state`
- [ ] `tasks states-for-dag-run`
- [ ] `tasks clear`
- [ ] `tasks list`
- [ ] `tasks failed-deps`

## assets

- [x] `assets materialize` (#68175)
- [x] `assets list` #68531
- [x] `assets details` #68531

## pools

- [x] `pools`: list / get / set / delete / import / export (#68175)

## variables

- [x] `variables list` #68932
- [x] `variables get` #68932
- [x] `variables set` #68932
- [x] `variables delete` #68932
- [x] `variables import` #68932

## connections

- [x] `connections list` #68972
- [x] `connections add` #68972
- [x] `connections delete` #68972
- [x] `connections import` #68972
- [x] `connections test` #68972
- [x] `connections create-default-connections` #68972

## backfill

- [ ] `backfill create`

## jobs

- [ ] `jobs check`

## config

- [x] `config get-value` #68958
- [x] `config list` #68958

## providers

- [x] `providers list` #68525
- [x] `providers get` #68525

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.