Azure / Azure/azure-kusto-python

Suggestion: Could ingest_* methods return something

Abierto
#191 3 comentarios 2 reacciones 0 asignados Ver en GitHub
Discussion enhancement
Lenguaje dominante
Python
Estrellas
204
Forks
118
Merge medio
13 d 7 h
PR fusionados (30 d)
1

Descripción

**Is your feature request related to a problem? Please describe.**
ingestion_ methods make developer life difficult to write robust code. They currently silently fail even if their ingestion ends up in the ".show ingestion failures". Users have no way of monitoring this as all underlying ids are hidden from the user to even check against that table.

**Describe the solution you'd like**
have ingestion_ methods return something useful to handle failures programmatically. Ideally mimicking the robust C# api.

**Describe alternatives you've considered**
I guess KIT?

edit1: So it appears our friend the KIT library has a pattern with ingestion tag monitoring: https://github.com/Azure/azure-kusto-ingestion-tools/blob/a2a256a09a66aacfe9c4756b2b0b457014013c4a/kit/kit/backends/kusto.py#L183

However the problem with ingestion tag monitoring is that it only can identify successful rows, at least with how it's implemented in KIT currently. However, the API that KIT offers seems much more full featured. Why aren't some of KIT's API features available for ingestion? seems like most of KIT's methods lend themselves well to the ingestion library. KIT makes sense standalone for CLI and data schema inferencing, but it has some nice repeated patterns anyone using the ingestion library would eventually need to write

edit2: ehhh kit is pretty tangled up with this manifest business. Possibly have ingest return the url? https://github.com/Azure/azure-kusto-python/blob/d81a549a86a5463db1c131357c179a8f3830d075/azure-kusto-ingest/azure/kusto/ingest/_ingest_client.py#L64 This would at least allow users to inspect the failures table for their specific error filtering by the random GUID used in the dataframe/file helper methods? That would effectively be the unique key

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Empieza leyendo azure/kusto/ingest/_ingest_client.py y el manejo de ingestion-tags en kit/kit/backends/kusto.py. Compara los métodos de ingestion existentes con los patrones de KIT referenciados e inspecciona cómo se identifican los fallos. Se considera terminado cuando el proyecto tenga un contrato de retorno definido y documentado que permita a los llamadores supervisar o correlacionar programáticamente los fallos de ingestion.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
azure, python
Área
api, data-engineering
Tipo de issue
Nueva funcionalidad
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Estancado
Claridad
Necesita aclaración
Aptitud para principiantes
25/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.