Bound the size of an app config fragment
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Objective
Put an upper bound on how large a single app config fragment may be. Today no layer bounds it, so any principal allowed to write a fragment can store a document of practically unlimited size. Decide the limit, enforce it on every write path, and reject oversize documents with a typed error.
## Background
- `AppConfigFragmentRow.config` is a plain `JSONB` column with no check constraint (`manager/models/app_config_fragment/row.py`). Postgres stops only at the ~1 GB field ceiling.
- `AppConfigFragmentUpsertItem.config` is typed `dict[str, Any]` with no `Field()` constraint (`common/dto/manager/v2/app_config_fragment/request.py`), and `AppConfigFragmentService.bulk_upsert` adds no size check of its own.
- The bulk upsert inputs (`ScopedUpsertAppConfigFragmentsInput.items`, `MyUpsertAppConfigFragmentsInput.items`) declare `min_length=1` but no maximum, so one request may also carry an unbounded number of items.
- The `my` upsert lets an ordinary user write fragments at their own user scope, so the growth path is reachable without any admin role.
- Every app config read merges the fragments for the requested names (`AppConfigService.get_app_configs`), so an oversize fragment is paid for on each read, not just once at write time.
## Scope
- Decide the limit and how it is measured (serialized UTF-8 JSON bytes is the obvious unit).
- Enforce it in the service layer so GraphQL and REST v2 are both covered, and raise a `BackendAIError` subclass added to `manager/errors/app_config.py` rather than a bare validation error.
- Keep the per-item partial-success contract from BA-7250: in a bulk upsert an oversize item fails on its own and does not sink the rest of the batch.
- Decide the maximum item count per bulk upsert request, or split it off explicitly if it is not taken here.
- State the limit in the DTO field description so it shows up in the generated API reference.
## Open questions
- The value itself is undecided — the candidates are on the order of 64 KiB / 256 KiB / 1 MiB per fragment.
- Fixed constant, or configurable through the manager config? A per-config-name limit carried on the definition or the allow-list entry is a third option.
- Whether the merged result of all fragments for one config name needs its own cap on top of the per-fragment one.
## Acceptance Criteria
- A per-fragment size limit is chosen, documented, and enforced on both the scoped and the `my` upsert paths.
- An oversize fragment is rejected with a typed app config error carrying a distinguishable error code, not a generic 400.
- In a bulk upsert, an oversize item rejects the whole batch. Per-item partial reporting waits until the repository supports partial success — the upsert's failed list is never populated today.
- A decision on the per-request item count limit is recorded — implemented, or deferred to a linked issue.
- Tests cover a document just under the limit (accepted), one just over (rejected), and the mixed bulk batch.
## Related
- BA-7230 — App Config follow-up issues (parent epic).
- BA-7250 — Support partial success in bulk app config fragment upsert.
JIRA Issue: BA-7345
Contributor guide
Research direction
Start by reading AppConfigFragmentService.bulk_upsert, manager/models/app_config_fragment/row.py, common/dto/manager/v2/app_config_fragment/request.py, and manager/errors/app_config.py, then trace the scoped and my upsert inputs. Resolve the limit, item-count decision, and conflicting partial-success criteria before implementing. Done means both paths enforce the documented limit, return the typed error, and have boundary and mixed-batch tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- api, backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100