ISISNeutronMuon / ISISNeutronMuon/analytics-data-platform

Intermittent errors on ingestion jobs

Open
#249 2 comments 0 reactions 0 assignees View on GitHub
bug bug: elt-pipeline
Dominant language
Python
Stars
0
Forks
0
Avg merge
2d 7h
Merged PRs (30d)
21

Description

Occasionally an ingestion job will fail with the error: "pyiceberg.exceptions.CommitFailedException: CatalogCommitConflicts: CatalogCommitConflicts => Requirement failed: Branch or tag `main` already exists". Rerunning the script fixes it most of the time but the pipelines need to be more robust.

Maybe write_to_table needs wrapping in a retry block?

Details:

```python
File "/home/ubuntu/.cache/uv/environments-v2/accelerator-sharepoint-085545dc75ef208d/lib/python3.13/site-packages/pyiceberg/catalog/rest/response.py", line 111, in _handle_non_200_response
raise exception(response) from exc
pyiceberg.exceptions.CommitFailedException: CatalogCommitConflicts: CatalogCommitConflicts => Requirement failed: Branch or tag `main` already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/ubuntu/.cache/uv/environments-v2/accelerator-sharepoint-085545dc75ef208d/lib/python3.13/site-packages/dlt/common/destination/client.py", line 424, in run_managed
self.run()
~~~~~~~~^^
File "/home/ubuntu/.cache/uv/environments-v2/accelerator-sharepoint-085545dc75ef208d/lib/python3.13/site-packages/elt_common/dlt_destinations/pyiceberg/pyiceberg.py", line 64, in run
self.iceberg_client.write_to_table(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self.load_table_name,
^^^^^^^^^^^^^^^^^^^^^
pq.read_table(self._file_path),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
self._load_table.get("write_disposition"),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/ubuntu/.cache/uv/environments-v2/accelerator-sharepoint-085545dc75ef208d/lib/python3.13/site-packages/elt_common/dlt_destinations/pyiceberg/exceptions.py", line 33, in _wrap
raise DestinationTerminalException(status_ex) from status_ex
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.