[Feature] Snapshot supports the function of recording additional information of users
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
To add an attribute for recording additional user-stored information in the existing snapshot structure, considering flexibility and expandability, a new attribute named extraInfo can be added to the top-level structure. This attribute stores a key-value pair structure, similar to a dictionary or Map, to facilitate the storage of extra user information.
```json
{
"extraInfo": {}
}
```
Design of extraInfo Structure
Basic Key-Value Pairs: For storing simple user information such as preference settings.
Nested Structure: Supports more complex data structures, such as lists or dictionaries, to store more detailed user data.
```json
{
"extraInfo": {
"preferences": {
"theme": "dark",
"language": "zh-CN"
},
"history": [
{
"action": "login",
"time": "2024-02-23T10:00:00Z"
}
]
}
}
```
Integration into the Existing Structure
The extraInfo attribute is integrated into the structure you provided, with the new structure as follows:
```json
{
"version": 3,
"id": 1,
"schemaId": 0,
"baseManifestList": "manifest-list-cd8aff2b-3fbb-430a-881f-1a5064fd1edd-0",
"deltaManifestList": "manifest-list-cd8aff2b-3fbb-430a-881f-1a5064fd1edd-1",
"changelogManifestList": null,
"indexManifest": "index-manifest-5ef89060-6da5-4561-8e06-0394432e61d4-0",
"commitUser": "bd9de2a7-64d0-4395-958e-f88c7c99383e",
"commitIdentifier": 9223372036854775807,
"commitKind": "APPEND",
"timeMillis": 1708617516783,
"logOffsets": {},
"totalRecordCount": 1,
"deltaRecordCount": 1,
"changelogRecordCount": 0,
"watermark": -9223372036854775808,
"extraInfo": {
"preferences": {
"theme": "dark",
"language": "zh-CN"
},
"history": [
{
"action": "login",
"time": "2024-02-23T10:00:00Z"
}
]
}
}
```
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue does not name implementation files, tests, or an entry point. Begin by locating the existing snapshot structure and its serialization and deserialization paths, then determine how an optional nested extraInfo map should be represented while preserving existing snapshots. Done means snapshots can record and recover the requested additional information without breaking existing formats.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100