datopian / datopian/aircan

ckan_api_load_multiple_steps is not found in dags folder

Open
#135 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
34
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
1

Description

I have implemented the following:
(base) majjedfmci@k8sworker2:~$ airflow dags trigger ckan_api_load_multiple_steps --conf='{ "resource": {
"path": "/home/majjedfmci/Downloads/MAJJED.csv",
"format": "txt",
"ckan_resource_id": "e7078210-79ff-4514-8372-aaa88d1f8ddb",
"schema": {
"fields": [
{
'id': 'Industry', 'type': 'text',
"format": "default"
}
]
}
},
"ckan_config": {
"api_key": "J0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJXd1k3UGZRODRxcXo5NktMVU1ZWUNNNVExdmREUFlmc3VEZTZzbm44dUtrIiwiaWF0IjoxNzA5NjcwOTE1fQ.I9oE5ScEGhWacQwcxtrXADrkhhsVa95mGtXzAoBIBUI",
"site_url": "http://127.0.0.1:5000",
}
}'
/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/utils/dot_renderer.py:28 UserWarning: Could not import graphviz. Rendering graph to the graphical format will not be possible.
[2024-03-07T19:01:53.183+0000] {init.py:42} INFO - Loaded API auth backend: airflow.api.auth.backend.session
Traceback (most recent call last):
File "/home/majjedfmci/anaconda3/bin/airflow", line 8, in
sys.exit(main())
^^^^^^
File "/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/main.py", line 57, in main
args.func(args)
File "/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/cli/cli_config.py", line 49, in command
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/utils/cli.py", line 114, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/cli/commands/dag_command.py", line 164, in dag_trigger
message = api_client.trigger_dag(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/api/client/local_client.py", line 34, in trigger_dag
dag_run = trigger_dag.trigger_dag(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/majjedfmci/anaconda3/lib/python3.11/site-packages/airflow/api/common/trigger_dag.py", line 122, in trigger_dag
raise DagNotFound(f"Dag id {dag_id} not found in DagModel")
airflow.exceptions.DagNotFound: Dag id ckan_api_load_multiple_steps not found in DagModel

As you see above that ckan_api_load_multiple_steps is not found. Can someone help me how to handle this issue. Thanks in advance.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Airflow DAGs folder and the ckan_api_load_multiple_steps identifier, then compare the available DAG IDs with the ID used in the airflow dags trigger command. Review the reported DagNotFound traceback and determine what project or environment state prevents the DAG from appearing in DagModel. Done means the DAG is discoverable and the trigger command succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.