AbsaOSS / AbsaOSS/enceladus

Improve INFO file metrics

Aperta
#392 0 commenti 0 reazioni 1 assegnatario Rivendicata da @yruslan Vedi su GitHub
Conformance feature priority: undecided Standardization under discussion
Lingua principale
Scala
Stelle
33
Fork
16
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Background
Currently, INFO file metrics have the following meaning:

| Metric | Description |
| ---------- |:--------------- |
| `std_records_succeeded` | The number of records that **don’t have any errors** in the error column after running **Standardization**. |
| `std_records_failed` | The number of records that have **at least 1 error** in the error column after running **Standardization**. |
| `std_errors_count` | The number of errors in the error column after running **Standardization**. A record can have several errors so this number can be larger than `std_records_failed`. |
| `conform _records_succeeded` | The number of records that **don’t have any errors** in the error column after running **Standardization+Conformance**. |
| `conform _records_failed` | The number of records that have **at least 1 error** in the error column after running **Standardization+Conformance**. |
| `conform_errors_count` | The number of errors in the error column after running **Standardization+Conformance**. A record can have several errors so this number can be larger than `conform_records_failed`. |

The names are counter intuitive and need to be changed.

## Feature
Add metrics for error totals and change the semantics of the current metrics according to the following:

| Metric | Description |
| ---------- |:--------------- |
| `std_records_succeeded` | The number of records that **don’t have any Standardization errors** in the error column. |
| `std_records_failed` | The number of records that have **at least 1 Standardization error** in the error column. |
| `std_errors_count` | The total number of **Standardization** errors in the error column. |
| `conform _records_succeeded` | The number of records that **don’t have any Conformance errors** in the error column. |
| `conform _records_failed` | The number of records that have **at least 1 Conformance error** in the error column. |
| `conform_errors_count` | The number of **Conformance errors** in the error column. |
| `publish _records_succeeded` | The number of records that **don’t have ANY errors** in the error column after running **Standardization+Conformance**. |
| `publish _records_failed` | The number of records that have **at least 1 error** in the error column after running **Standardization+Conformance**. |
| `publish_errors_count` | The number of errors in the error column after running **Standardization+Conformance**. A record can have several errors so this number can be larger than `publish_records_failed`. |

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.