openwisp / openwisp/openwisp-utils
[feature] Add persistent internal configuration storage
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 93
- Forks
- 104
- Avg merge
- 14h 10m
- Merged PRs (30d)
- 22
Description
⚠️ NOT SUITED TO BEGINNER CONTRIBUTORS.
Is your feature request related to a problem? Please describe.
OpenWISP installers and provisioning tools need to remember small pieces of internal state for the lifetime of an installation. Examples include the UUID of the management VPN selected during setup and the version of the setup steps that completed successfully.
There is no shared database-backed facility for this data today. Docker OpenWISP stores the UUID and access key of its default VPN, plus the UUID of its certificate authority, as raw values in the Redis database used by the Django cache. The bootstrap script runs again when the dashboard starts and falls back to a VPN selected by its mutable name or to the first VPN in the database. It also stores topology identifiers in Redis using the same pattern.
The Redis volume is persistent in the standard Docker deployment, but this is still cache storage. Clearing or replacing the cache can remove these selectors while the related database objects remain. The bootstrap script may then select a different object. docker-openwisp PR #609 improves object selection but continues to use Redis as the durable source of these identifiers.
Ansible OpenWISP also reruns initialization and decides what to create by inspecting existing objects. There is no shared record of which version of the application setup completed.
Describe the solution you'd like
Add a small optional Django app to openwisp-utils for persistent internal installation metadata.
The first version should provide:
- a database model with a unique, namespaced key, a JSON-compatible value, and creation and modification timestamps;
- a small Python API for reading, creating or replacing, and deleting values;
- atomic, idempotent writes so installers can run the same operation safely more than once;
- explicit behavior for missing keys and optional defaults;
- documentation requiring each installer or module to use its own namespace;
- documentation recommending a versioned setup marker instead of a permanent Boolean, so new setup steps can be added later;
- documentation explaining that a completion marker must be written only after all related setup work succeeds;
- tests for create, read, update, delete, idempotent writes, unique keys, missing keys, invalid values, and concurrent writes.
The facility should be hidden from regular users. A read-only view for superusers could be considered later for troubleshooting, but a web interface is not required for the first version.
This store must not contain passwords, access keys, private keys, tokens, certificates, or other secrets. It should store the UUID of a selected domain object, not duplicate that object's own data. It should not add foreign keys to optional OpenWISP modules.
Docker and Ansible integration should be handled in follow-up issues. For Docker, the database value can remain the durable source of truth while startup code copies the current UUID and derived access data to Redis for containers that cannot use the Django ORM.
Describe alternatives you've considered
- Continue using Redis: this works as an operational handoff, but cache storage should not be the only durable record of installer-selected objects.
- Add installer-specific models: this would duplicate the same basic mechanism in Docker, Ansible, and future provisioning tools.
- Infer state from names or object existence: names can change, more than one matching object may exist, and object existence does not show which setup version completed.
- Reuse anonymous metric collection models: metric collection is optional and tracks installation/version events for a different purpose. It should not control installer behavior.
Additional context
- Related Docker issue: openwisp/docker-openwisp#572
- Related Docker work: openwisp/docker-openwisp#609
- Related Ansible proposal: openwisp/ansible-openwisp2#201
- Existing reusable model base:
TimeStampedEditableModel
The generic storage API should not decide how to recover from a missing or deleted VPN, template, or topology. Each installer owns that policy and must still verify that referenced objects exist and are valid.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading openwisp_utils/base.py and the existing Django utility structure, then map the requested optional app, model, API, and test coverage to the repository. Done means persistent namespaced JSON-compatible values support safe create, read, replace, delete, defaults, validation, uniqueness, and concurrent writes, with documentation covering namespaces, setup markers, completion ordering, and prohibited secrets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- backend, database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100