Azure / Azure/azure-kusto-python
Suggestion: Could ingest_* methods return something
- Lingua principale
- Python
- Stelle
- 204
- Fork
- 118
- Merge medio
- 13g 7h
- PR unite (30g)
- 1
Descrizione
**Is your feature request related to a problem? Please describe.**
ingestion_ methods make developer life difficult to write robust code. They currently silently fail even if their ingestion ends up in the ".show ingestion failures". Users have no way of monitoring this as all underlying ids are hidden from the user to even check against that table.
**Describe the solution you'd like**
have ingestion_ methods return something useful to handle failures programmatically. Ideally mimicking the robust C# api.
**Describe alternatives you've considered**
I guess KIT?
edit1: So it appears our friend the KIT library has a pattern with ingestion tag monitoring: https://github.com/Azure/azure-kusto-ingestion-tools/blob/a2a256a09a66aacfe9c4756b2b0b457014013c4a/kit/kit/backends/kusto.py#L183
However the problem with ingestion tag monitoring is that it only can identify successful rows, at least with how it's implemented in KIT currently. However, the API that KIT offers seems much more full featured. Why aren't some of KIT's API features available for ingestion? seems like most of KIT's methods lend themselves well to the ingestion library. KIT makes sense standalone for CLI and data schema inferencing, but it has some nice repeated patterns anyone using the ingestion library would eventually need to write
edit2: ehhh kit is pretty tangled up with this manifest business. Possibly have ingest return the url? https://github.com/Azure/azure-kusto-python/blob/d81a549a86a5463db1c131357c179a8f3830d075/azure-kusto-ingest/azure/kusto/ingest/_ingest_client.py#L64 This would at least allow users to inspect the failures table for their specific error filtering by the random GUID used in the dataframe/file helper methods? That would effectively be the unique key
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia leggendo azure/kusto/ingest/_ingest_client.py e la gestione degli ingestion-tag in kit/kit/backends/kusto.py. Confronta i metodi di ingestion esistenti con i pattern KIT indicati e verifica come vengono identificati i fallimenti. Il lavoro è completato quando il progetto dispone di un contratto di ritorno definito e documentato che consenta ai chiamanti di monitorare o correlare programmaticamente i fallimenti di ingestion.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- azure, python
- Ambito
- api, data-engineering
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100