argotorg / argotorg/solidity

Is padding of all memory allocations to a full word desirable?

Open
#16,935 0 comments 0 reactions 0 assignees View on GitHub
language design :rage4: low impact medium effort needs design
Dominant language
C++
Stars
25.7k
Forks
6.2k
Avg merge
1d 11h
Merged PRs (30d)
21

Description

Currently the compiler pads its memory allocations to a multiple of 32-bytes. This was meant as an extra defensive safeguard against potential compiler bugs. The problem is that this was never documented as intentional and we found cases where it is not being done. We need to decide if it is still desirable, document it and consistently enforce the new rule.

## How the compiler allocates memory
The compiler has some freedom in how much memory it allocates for variables in memory. As long as the allocated area is not too short, the actual size is not a problem, because the extra space will not be touched. Any code that follows the free memory pointer rather than make assumptions about the size of allocations is safe.

Only reference types can currently be allocated in memory and most of them are naturally padded to word boundaries due to their elements being padded themselves (see [Layout in Memory](https://docs.soliditylang.org/en/v0.8.36/internals/layout_in_memory.html#layout-in-memory)). The only exception is `bytes`/`string`, which consists of a 32-byte length field and a data area containing unpadded characters. Despite it being possible to allocate the exact number of bytes needed, the compiler normally rounds it up to the nearest multiple of 32. For example, a 2-character string takes up 64 bytes of memory, not 34.

This behavior is not guaranteed by the documentation. There is no padding requirement. As [stated under Memory Management](https://docs.soliditylang.org/en/v0.8.36/assembly.html#memory-management), there is also no word-alignment requirement for allocations (which would have a similar effect as padding in practice).

## Rationale for the extra padding
The idea behind padding in the first place was mainly a defensive mechanism. If we forget to clear higher order bits, the value we get is still likely zero. Bugs causing an operation on one variable to unintentionally spill into another are also less likely, since we usually read/write a whole word.

However, this was also done in a time where the compiler had more internal checks. This mechanism is only a second line of defense and it is unclear how much it actually helps us today.

The mechanism costs us a bit of extra memory, especially for contracts that use a lot of short strings. On the other hand, memory layout prioritizes the cost of operations over efficient use of memory so it's not unique in its wastefulness. The fact that the length field takes 32 bytes even though the length is capped at 64 bits (#13071) is just as wasteful.

## Inconsistency
As it has recently turned out, the padding is not always present. The `bytes.concat()`/`string.concat()`/`abi.encodePacked()` apply it only in the IR pipeline. The evmasm versions do not.

For example, this code allocates 68 bytes via evmasm and 128 bytes via IR:
```solidity
bytes1 a;
bytes.concat(a, a);
bytes.concat(a, a);
```

Since the behavior is not documented and not covered by tests, this may be happening in more cases. While neither violates any of our guarantees, inconsistencies between pipelines increase complexity and should be kept to a minimum.

Contributor guide

Open the contributing guide

Research direction

Compare the IR and evmasm handling of bytes.concat(), string.concat(), and abi.encodePacked(), starting with the example that allocates 68 versus 128 bytes. Decide whether full-word padding should be guaranteed, then document the rule and add consistent enforcement and test coverage for both pipelines.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, solidity
Domain
compilers, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.