Azure / Azure/azure-sdk-for-python

Monitor Ingestion Upload Chunking - Large Single Log Lines

Open
#42,648 2 comments 0 reactions 1 assignee Claimed by @pvaneck View on GitHub
Client customer-reported feature-request Monitor needs-team-attention question
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

- **Package Name**: azure-monitor-ingestion
- **Package Version**: 1.0.4
- **Operating System**: Debian
- **Python Version**: 3.13

**Describe the bug**

I noticed that in the chunking logic towards Log Ingestion API if a single log is passed that is too large over 1MB after gzip I assume, there is not any specific warnings or dropping that occurs.

Discovered the issue after a user sent one of my log ingestion pipelines some bad data.

There are some useful errors that occur when this happens that make it obvious whats up:

```
'x-ms-error-code': 'ContentLengthLimitExceeded'
[ERROR][2025-08-22 12:40:21][dbconsumers-oracle][azure.monitor.ingestion.aio._operations._patch]: Failed to upload chunk containing 1 log entries
```

and

```
File "/app/.venv/lib/python3.13/site-packages/commonmodules/databases/_process_db_logs.py", line 78, in process_db_logs
await uploader.upload_logs(logs=data.to_dict(orient="records")) # type: ignore # noqa: PGH003
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.13/site-packages/commonmodules/databases/_upload.py", line 50, in upload_logs
await client.upload(dcr_rule_id=self.dcr_rule_id, stream_name=self.stream_name, logs=mutable_logs)
File "/app/.venv/lib/python3.13/site-packages/commonmodules/ingestion/aio/_ingestion.py", line 60, in upload
return await self.client.upload(
^^^^^^^^^^^^^^^^^^^^^^^^^
...<3 lines>...
)
^
File "/app/.venv/lib/python3.13/site-packages/azure/monitor/ingestion/aio/_operations/_patch.py", line 93, in upload
raise err
File "/app/.venv/lib/python3.13/site-packages/azure/monitor/ingestion/aio/_operations/_patch.py", line 84, in upload
await super()._upload( # type: ignore
rule_id, stream_name=stream_name, body=gzip_data, content_encoding="gzip", **kwargs # type: ignore
)
Message: Maximum allowed content length: 1048576 bytes (1 MB). Provided content length: 1157880 bytes.

```

**To Reproduce**
Steps to reproduce the behavior:
1. send a few logs some of which being very large hitting the 1MB even after compression

**Describe the solution you'd like**

There is an error that gets through to the DCRLogErrors table so that is great, although I would hope some more resilient way to handle this could be done in the chunking code so that we maybe dont lose the entire batch of logs. Best case would be that offending oversize individual logs are dropped but rest of batch can make it through and then we continue without needing to catch HTTP error outside of the lib.

I think that if individual rows exceed the allowed size after gzip they should be dropped. However not lead to failure of remainder of the batch to upload.

Image

**Additional context**

https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-ingestion/azure/monitor/ingestion/_helpers.py

@pvaneck

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.