influxdata / influxdata/influxdb-client-python
When writing a batch to an implicit bucket upon schema conflict, I receive a 400. The logs seems to be f-printed.
Dieses Issue hat noch niemand übernommen.
- Vorherrschende Sprache
- Python
- Sterne
- 792
- Forks
- 186
- Ø Merge
- 3 Std. 2 Min.
- Gemergte PRs (30 T.)
- 1
Beschreibung
Specifications
- Client Version: 1.40.0
- InfluxDB Version: Cloud 2.0 TSM
- Platform: GCP
I am extracting some data from BQ and pushing it to Influx. That part is working well.
Next, I write a point in an Influx bucket to mark the task completed. This was written by someone else and it uses a different call.
In this particular case, I have a conflict for the second parameter. In the GCP logs, I can see the error as "default" on top of the log generated by loguru (the log appears twice).
Code sample to reproduce problem
in the calling function
points = [
Point.from_dict(
{
"measurement": __DATA_MEASUREMENT,
"fields": {state: value},
"tags": {"type": "status", **tags},
}
),
Point.from_dict(
{
"measurement": __DATA_MEASUREMENT,
"fields": {state: datetime.now().isoformat()},
"tags": {"type": "timestamp", **tags},
}
),
]
write_points(
__INFLUX_URL,
__INFLUX_TOKEN,
__INFLUX_BUCKET,
__INFLUX_ORGANISATION,
points,
)
```
in the lib to write_points:
``````python
def write_points(
influx_url: str,
influx_token: str,
influx_bucket: str,
influx_organization: str,
points: List[Union[Point, Dict[str, Any]]],
) -> None:
logger.info("Writing to InfluxDB, Points length={}", len(points))
client = InfluxDBClient(url=influx_url, token=influx_token, timeout=300_000)
try:
write_api = client.write_api(
success_callback=callback.success,
error_callback=callback.error,
retry_callback=callback.retry,
write_options=WriteOptions(max_retries=10, max_retry_time=280_000)
)
try:
write_api.write(
batch_size=200,
bucket=influx_bucket,
org=influx_organization,
record=points
)
logger.debug("Write operation completed")
except InfluxDBError as e:
logger.error("InfluxDB Asynchronous write error: {}", str(e))
if hasattr(e, 'response'):
logger.error("Response status: {}", e.response.status_code)
logger.error("Response headers: {}", dict(e.response.headers))
logger.error("Response body: {}", e.response.text)
raise
except Exception as ex:
logger.error("Unexpected error during write operation: {}", str(ex))
raise
finally:
client.close()
```
I know this is not correctly configured to do SYNCHRONOUS write, but that is how I see the issue.
The callback object is just implementing calls to logger.{info|error|warning} resp.
### Expected behavior
The logs are formatted as per my loguru logger settings
### Actual behavior
some logs (error 400 schema conflict) appear twice.
### Additional info
(DEBUG) 2025-04-25 15:15:06.130 CEST
Write operation completed
(DEFAULT) 2025-04-25 15:15:10.914 CEST
The batch item wasn't processed successfully because: (400)
(DEFAULT) 2025-04-25 15:15:10.914 CEST
Reason: Bad Request
(DEFAULT) 2025-04-25 15:15:10.914 CEST
HTTP response headers: HTTPHeaderDict({'Date': 'Fri, 25 Apr 2025 13:15:09 GMT', 'Content-Type': 'application/json', 'Content-Length': '247', 'Connection': 'keep-alive', 'influxdata-partial-write': 'true', 'trace-id': 'xxxxxxxxxxxxxxxx', 'trace-sampled': 'false', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Influxdb-Request-ID': 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz', 'X-Influxdb-Build': 'Cloud'})
(DEFAULT) 2025-04-25 15:15:10.914 CEST
HTTP response body: {"code":"invalid","message":"partial write has occurred, errors encountered on line(s): line 2: batch schema conflict - line has column 'inserted_data' of type iox::column_type::field::string but type is iox::column_type::field::integer","line":2}
(ERROR) 2025-04-25 15:15:10.915 CEST
Cannot write batch: ('{bucket_name}', '{org_name}', 'ns'), data: b'GCP_Data_monitoring,app=myapp,stage=preprod,type=status,zone=eu-west-01 inserted_data=1i\nGCP_Data_monitoring,app=myapp,stage=preprod,type=timestamp,zone=eu-west-01 inserted_data="2025-04-25T13:15:06.127660"' due: (400) Reason: Bad Request HTTP response headers: HTTPHeaderDict({'Date': 'Fri, 25 Apr 2025 13:15:09 GMT', 'Content-Type': 'application/json', 'Content-Length': '247', 'Connection': 'keep-alive', 'influxdata-partial-write': 'true', 'trace-id': 'xxxxxxxxxxxxxxxx', 'trace-sampled': 'false', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Influxdb-Request-ID': 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz', 'X-Influxdb-Build': 'Cloud'}) HTTP response body: {"code":"invalid","message":"partial write has occurred, errors encountered on line(s): line 2: batch schema conflict - line has column 'inserted_data' of type iox::column_type::field::string but type is iox::column_type::field::integer","line":2}
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Erste Schritte
- Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
- Forke das Repository und arbeite in einem Branch.
- Öffne einen Pull Request, der die Issue-Nummer nennt.
Rechercherichtung
Beginnen Sie mit der gezeigten Funktion write_points, insbesondere mit InfluxDBClient.write_api, WriteOptions sowie den success-, error- und retry-Callbacks. Reproduzieren Sie den Schema-Konflikt im Batch und vergleichen Sie die Callback-Ausgabe mit der loguru-Ausgabe. Als erledigt gilt dies, wenn der Fehler einmal mit der erwarteten Formatierung ausgegeben wird oder das Issue eindeutig eine Einschränkung im Verhalten von Client gegenüber Callback identifiziert.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python
- Bereich
- api, observability
- Issue-Typ
- Bug
- Schwierigkeit
- 3/5
- Geschätzter Aufwand
- 1-2 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 30/100