GoogleCloudPlatform / GoogleCloudPlatform/fluentbit-bigquery-writeapi-sink
[BUG] parseMap() doesn't handle arrays recursively, causing JSON marshal errors
- Dominant language
- Go
- Stars
- 14
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
The `parseMap()` function doesn't recursively process arrays, leaving
`map[interface{}]interface{}`
inside arrays unconverted. This causes JSON marshaling to fail with nested data structures.
## Environment
- Plugin version: latest (main branch)
- Go version: 1.23.0
- Error: `json: unsupported type: map[interface {}]interface {}`
## Problem Description
When processing data with nested arrays containing objects, the plugin fails with a JSON marshal
error.
Example data that causes the error:
```json
{
"request": {
"imp": [{
"banner": {"w": 300, "h": 250}
}]
}
}
```
### Root Cause
The current parseMap() implementation:
```go
func parseMap(m0 map[interface{}]interface{}) map[string]interface{} {
m1 := make(map[string]interface{})
for k, v := range m0 {
switch t := v.(type) {
case []byte:
m1[k.(string)] = string(t)
case map[interface{}]interface{}:
m1[k.(string)] = parseMap(t)
default: // Arrays pass through without processing
m1[k.(string)] = v
}
}
return m1
}
```
The issue: Arrays ([]interface{}) containing map[interface{}]interface{} are not processed,
causing json.Marshal() to fail when it encounters these unprocessed maps.
### Proposed Fix
Add recursive array processing:
```go
func parseMap(m0 map[interface{}]interface{}) map[string]interface{} {
m1 := make(map[string]interface{})
for k, v := range m0 {
switch t := v.(type) {
case []byte:
m1[k.(string)] = string(t)
case map[interface{}]interface{}:
m1[k.(string)] = parseMap(t)
case []interface{}: // Add array handling
m1[k.(string)] = parseArray(t)
default:
m1[k.(string)] = v
}
}
return m1
}
func parseArray(a0 []interface{}) []interface{} {
a1 := make([]interface{}, len(a0))
for i, v := range a0 {
switch t := v.(type) {
case map[interface{}]interface{}:
a1[i] = parseMap(t)
case []interface{}:
a1[i] = parseArray(t)
default:
a1[i] = v
}
}
return a1
}
```
## Impact
- Any data with arrays containing objects fails to write to BigQuery
- Common with OpenRTB bid request data and similar nested structures
I'm happy to submit a PR with this fix if you agree with the approach.
Contributor guide
Assessment
This issue has not been assessed yet.