google / google/flatbuffers

[C++] FlatBufferBuilder writes unreadable buffers on targets where alignof(scalar) < sizeof(scalar)

Open
#9,182 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
26.5k
Forks
3.7k
PR merge metrics
No merged PRs in 30d

Description

On a target whose ABI gives some scalar types a smaller alignment than their
size, `FlatBufferBuilder` can insert padding between a string's length prefix
and its payload. The result does not match the wire format and cannot be read
back — not even through FlatBuffers' own `String` accessor on the machine that
wrote it.

The default m68k Linux ABI is such a target: GCC gives `uint32_t` 2-byte
alignment even though it occupies 4 bytes. Thus `__alignof__(uint32_t)` is 2
while `sizeof(uint32_t)` is 4.

## Reproducer

```cpp
#include

#include
#include
#include

int main()
{
const char *kText = "m68k";
const size_t kLen = strlen(kText);

printf("sizeof(uoffset_t)=%zu AlignOf()=%zu __alignof__=%zu\n",
sizeof(flatbuffers::uoffset_t),
flatbuffers::AlignOf(),
(size_t)__alignof__(flatbuffers::uoffset_t));

flatbuffers::FlatBufferBuilder fbb;
auto off = fbb.CreateString(kText, kLen);

const flatbuffers::String *string =
flatbuffers::GetTemporaryPointer(fbb, off);
const uint8_t *str = reinterpret_cast(string);
const uint32_t len = string->size();
const char *data = string->c_str();

printf("raw bytes at the string offset:");
for (size_t i = 0; i < sizeof(uint32_t) + kLen + 2; i++)
printf(" %02x", str[i]);
printf("\n");

printf("length prefix = %lu (expected %zu)\n",
static_cast(len), kLen);
printf("payload = \"%.*s\" (expected \"%s\")\n", (int)len, data, kText);

const bool ok = len == kLen && memcmp(data, kText, kLen) == 0;

printf("%s\n", ok ? "PASS" : "FAIL: padding between length prefix and payload");
return ok ? 0 : 1;
}
```

Build and run:

```sh
g++ -std=c++17 -I include repro.cc -o repro && ./repro
m68k-linux-gnu-g++ -std=c++17 -I include -static repro.cc -o repro-m68k
qemu-m68k ./repro-m68k # or run it on any m68k machine
```

## Actual vs expected

x86-64 (correct):

```
sizeof(uoffset_t)=4 AlignOf()=4 __alignof__=4
raw bytes at the string offset: 04 00 00 00 6d 36 38 6b 00 00
length prefix = 4 (expected 4)
payload = "m68k" (expected "m68k")
PASS
```

m68k (broken):

```
sizeof(uoffset_t)=4 AlignOf()=2 __alignof__=2
raw bytes at the string offset: 04 00 00 00 00 00 6d 36 38 6b
length prefix = 4 (expected 4)
payload = "" (expected "m68k")
FAIL: padding between length prefix and payload
```

Note the two extra `00` bytes between the length prefix and `6d 36 38 6b`.

Contributor guide

Open the contributing guide

Research direction

Start with include/flatbuffers/flatbuffers.h, especially FlatBufferBuilder::CreateString and the AlignOf usage shown in repro.cc. Build and run the supplied reproducer on x86-64 and m68k, then verify that the string length prefix is immediately followed by its payload and that the m68k run reports PASS without unreadable padding.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.