google / google/orbax

Dedup device-mesh serialization in NamedShardingMetadata.to_serialized_string

Open
#3,494 3 comments 0 reactions 0 assignees View on GitHub
type:feature
Dominant language
Python
Stars
535
Forks
101
Avg merge
1d 13h
Merged PRs (30d)
23

Description

### Summary

`NamedShardingMetadata.to_serialized_string` serializes the device mesh via `dataclasses.asdict(self.device_mesh)` ([sharding.py#L261](https://github.com/google/orbax/blob/main/checkpoint/orbax/checkpoint/_src/metadata/sharding.py#L261)), and it runs once per array being saved. A checkpoint save typically has many arrays that all share the same mesh, so this re-walks and deep-copies the identical mesh once per array - O(num_arrays) redundant work that grows with mesh size and array count, and can become a significant fraction of save time at scale.

### Proposal

Since all arrays in a save share one mesh, the conversion can be memoized so the deep copy happens once per distinct mesh instead of once per array:

```python
# Build one shared DeviceMetadataMesh per distinct mesh
@classmethod
@functools.lru_cache(maxsize=128)
def from_jax_mesh(cls, mesh: jax.sharding.Mesh) -> Optional["DeviceMetadataMesh"]:
...

# Memoize the asdict deep-copy on that shared instance
@functools.cached_property
def as_serialized_dict(self) -> dict[str, Any]:
return dataclasses.asdict(self)

# to_serialized_string uses the cached dict instead of asdict per call
sharding_data[_DEVICE_MESH] = self.device_mesh.as_serialized_dict
```

The two work together: `from_jax_mesh` returns a single shared instance per distinct mesh, which lets `as_serialized_dict` memoize the deep copy exactly once — collapsing O(num_arrays) copies to O(num_unique_meshes).

Output is byte-identical to `dataclasses.asdict`, so existing checkpoints and the restore path are unaffected. The cache is stored on the instance (`cached_property` in `__dict__`), leaving `dataclasses.asdict` / `__eq__` semantics unchanged.

### Questions for maintainers

- Is memoizing the mesh serialization this way acceptable in general?
- `lru_cache` on `from_jax_mesh` keyed by `jax.sharding.Mesh` relies on Mesh hashability/equality — is that a safe assumption across backends?
- Any concern with the shared-instance semantics or cache lifetime (e.g. long-lived meshes retained by the `lru_cache`) that we should account for?

Contributor guide

Open the contributing guide

Research direction

Start in orbax/checkpoint/_src/metadata/sharding.py at NamedShardingMetadata.to_serialized_string and inspect DeviceMetadataMesh.from_jax_mesh. Confirm how mesh identity, hashing, equality, and instance lifetime behave before evaluating the proposed caches. Done means serialization remains byte-identical and the restore path is unaffected while repeated saves avoid redundant mesh copying.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.