apache / apache/airflow

Error when emitting Asset Metadata if value in extra is of type np.float64

Open
#53,474 11 comments 0 reactions 1 assignee Assigned to @crobertson-astro View on GitHub
affected_version:3.0 area:core kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

3.0.3

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

In version 3.0.2 I was able to emit an asset with some values pulled from a dataframe in the extra field. These values were of type np.float64. In 3.0.3, the task failed on attempting to emit the asset extra.

### What you think should happen instead?

The asset should be created with the relevant extra.

### How to reproduce

```python
import random
from airflow.sdk import dag, task, Asset, AssetAlias

ALIAS = AssetAlias("s3://alias-example")

TEST_ASSET = Asset("s3://my-bucket/data/file.csv")

@dag(tags=["asset_demo"])
def asset_demo():

@task
def get_file_ids():
return [i for i in range(10)]

@task(outlets=[TEST_ASSET])
def add_assets_to_alias(i, outlet_events):
"""
adds assets to alias
"""
import numpy as np

random_value = random.randint(1, 100)
print(random_value, "is the random value for file_id:", i)
outlet_events[TEST_ASSET].extra = {
"file_id": i,
"random_value": random_value,
"float_value": np.float64(1.23412415),
}

add_assets_to_alias.expand(i=get_file_ids())

asset_demo()
```

### Operating System

Linux

### Versions of Apache Airflow Providers

N/A

### Deployment

Astronomer

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.