Azure / Azure/azure-kusto-python
Suggestion: Could ingest_* methods return something
- Vorherrschende Sprache
- Python
- Sterne
- 204
- Forks
- 118
- Ø Merge
- 13 T. 7 Std.
- Gemergte PRs (30 T.)
- 1
Beschreibung
**Is your feature request related to a problem? Please describe.**
ingestion_ methods make developer life difficult to write robust code. They currently silently fail even if their ingestion ends up in the ".show ingestion failures". Users have no way of monitoring this as all underlying ids are hidden from the user to even check against that table.
**Describe the solution you'd like**
have ingestion_ methods return something useful to handle failures programmatically. Ideally mimicking the robust C# api.
**Describe alternatives you've considered**
I guess KIT?
edit1: So it appears our friend the KIT library has a pattern with ingestion tag monitoring: https://github.com/Azure/azure-kusto-ingestion-tools/blob/a2a256a09a66aacfe9c4756b2b0b457014013c4a/kit/kit/backends/kusto.py#L183
However the problem with ingestion tag monitoring is that it only can identify successful rows, at least with how it's implemented in KIT currently. However, the API that KIT offers seems much more full featured. Why aren't some of KIT's API features available for ingestion? seems like most of KIT's methods lend themselves well to the ingestion library. KIT makes sense standalone for CLI and data schema inferencing, but it has some nice repeated patterns anyone using the ingestion library would eventually need to write
edit2: ehhh kit is pretty tangled up with this manifest business. Possibly have ingest return the url? https://github.com/Azure/azure-kusto-python/blob/d81a549a86a5463db1c131357c179a8f3830d075/azure-kusto-ingest/azure/kusto/ingest/_ingest_client.py#L64 This would at least allow users to inspect the failures table for their specific error filtering by the random GUID used in the dataframe/file helper methods? That would effectively be the unique key
Beitragsleitfaden
Rechercherichtung
Beginnen Sie mit dem Lesen von azure/kusto/ingest/_ingest_client.py und der Verarbeitung von ingestion-Tags in kit/kit/backends/kusto.py. Vergleichen Sie die vorhandenen Ingestion-Methoden mit den referenzierten KIT-Mustern und untersuchen Sie, wie Fehler identifiziert werden. Als abgeschlossen gilt die Aufgabe, wenn das Projekt einen festgelegten, dokumentierten Rückgabevertrag hat, mit dem Aufrufer Ingestion-Fehler programmgesteuert überwachen oder korrelieren können.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- azure, python
- Bereich
- api, data-engineering
- Issue-Typ
- Feature
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Veraltet
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 25/100