awslabs / awslabs/python-deequ

Unable to resolve column names with a '.' in the name

Open
#99 1 comment 0 reactions 0 assignees View on GitHub
bug enhancement
Dominant language
Jupyter Notebook
Stars
826
Forks
158
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
When attempting to use any check on a field that has a '.' in the name such as:
```
.isComplete("column.name")
```
You will get an exception like:
```
Exception: Verification checks failed: [{'check_status': 'Error', 'check_level': 'Error', 'constraint_status': 'Failure', 'check': 'Verification checks', 'constraint_message': "org.apache.spark.sql.AnalysisException: cannot resolve '`column.name`' given input columns: [column.name];
```

**To Reproduce**
Steps to reproduce the behavior:
1. Create a data source with a column name that has a '.' in such as 'column.name'
2. Create a verification suite that includes the check `.isComplete("column.name")`
3. Run the code
4. See the error

**Expected behavior**
The field name should be resolved, allowing checks to be performed

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.