influxdata / influxdata/influxdb-client-csharp
Memory rising to 29Gb in one hour run
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 377
- Forks
- 93
- Avg merge
- 9m
- Merged PRs (30d)
- 2
Description
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
1. Singleton InfluxDBClient instance used for all writes. Single writeApi used for all calls throughout
```
client = new InfluxDBClient(url, Token);
var writeOptions = new WriteOptions
{
BatchSize = 5000,
FlushInterval = 500,
JitterInterval = 0,
};
converter = new DomainEntityConverter();
writeApiConv = client.GetWriteApi(writeOptions, converter);
```
// Below set data is called with other test client with load like:
// 200 parellel calls with each call having 500 samples, this is equal to 1lakh tags (or point data).
// Above load is called at every 1 second interval.
// Observation: Good news (happy with) writes are pretty fast in ~100ms, 200 parallel calls finishes easily.
// My main concern is memory rise.

Below details are for 10 min run memory going to ~15GB



```
public async Task SetData(SetDataRequest dataRequest, string bucket = bucketTagAsMeasurementName_Conv)
{
var data = new List();
foreach (var sampleSet in dataRequest.SampleSets)
{
for (int sampleIndex = 0; sampleIndex < sampleSet.Samples.Count; sampleIndex++)
{
var sample = sampleSet.Samples[sampleIndex];
data.Add(new StaticTagAsMeasurement2
{
Id = Guid.Parse(sampleSet.TagId),
HashId = sampleSet.TagId.GetHashCode(),
Value = double.Parse(sample.Value.ToString()),
DataStatus = sample.DataStatus,
NodeStatus = sample.NodeStatus,
IsValid = sample.DataStatus == 0,
Time = sample.Timestamp
});
}
}
writeApiConv.WriteMeasurements(data, WritePrecision.Ns, bucket, orgName);
}
// I tried first with commented PointData approach & then with PointData.Builder approach - same memory rise with both.
DomainEntityConverter{
public PointData ConvertToPointData(T entity, WritePrecision precision)
{
if (entity is StaticTagAsMeasurement2 tag2)
{
var pointBuilder = PointData.Builder.Measurement(tag2.Id.ToString());
pointBuilder = pointBuilder.Field("hashId", tag2.HashId);
pointBuilder = pointBuilder.Field("value", tag2.Value);
pointBuilder = pointBuilder.Field("isValid", tag2.IsValid);
pointBuilder = pointBuilder.Field("dataStatus", tag2.DataStatus);
pointBuilder = pointBuilder.Field("nodeStatus", tag2.NodeStatus);
pointBuilder = pointBuilder.Timestamp(new DateTime(tag2.Time, DateTimeKind.Utc), precision);
return pointBuilder.ToPointData();
// var pointData = PointData
//.Measurement(tag2.Id.ToString())
//.Field("hashId", tag2.HashId);
//.Field("value", tag2.Value)
//.Field("isValid", tag2.IsValid)
//.Field("dataStatus", tag2.DataStatus)
//.Field("nodeStatus", tag2.NodeStatus)
//.Timestamp(new DateTime(tag2.Time, DateTimeKind.Utc), precision);
// return pointData;
}
}
}
```
__Expected behavior:__
The memory should be stable say around 1GB so that long runs can be achieved
__Actual behavior:__
Memory is rising very fast
__Specifications:__
- Client Version:
- InfluxDB Version: v2.6
- Platform: Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz 2.59 GHz
64.0 GB (63.7 GB usable)
64-bit operating system, x64-based processor
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the supplied SetData method, WriteMeasurements call, and DomainEntityConverter.ConvertToPointData implementation. Reproduce the reported 200 parallel calls with 500 samples at one-second intervals, then determine whether client buffering retains data or whether the usage pattern is responsible. Done means the memory behavior is explained and a corrective change or supported usage guidance is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100