0surface / 0surface/arxiveprior
Redesign PublicationItemEntity Table Storage Class
- Dominant language
- C#
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
[Not Required] Redesign to handle large Author collections don't exceed the Azure Table Storage's property size limit of 64 Kb
Implemented Solution
From PublicationItem -> PublicationItemEntity
Added custom Auto Mapper resolver (AuthorListSplitResolver) to split excess Author data into 3 more Author list properties, in chunks of 400.
That is 4 List of AuthorItem properties each holding 400 items.
Any left will not be persisted,and the '_AuthorListTruncated_' flag is set to true.
From PublicationItemEntity -> PublicationItem
Added Auto Mapper resolver (AuthorListAggregateResolver) to add any authors in the 'spill' over Author list properties back to te single Authors property of the PublicationItem type.
See arxiv id 2007.06722v3 which has 948 Authors i.e. (name, affiliation) pair in JSON format.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue involves redesigning the PublicationItemEntity table storage class to handle large author collections within Azure Table Storage's 64 KB property limit. Look at the existing PublicationItem to PublicationItemEntity mapping and the custom AutoMapper resolvers (AuthorListSplitResolver and AuthorListAggregateResolver). Examine how authors are split into chunks of 400 across multiple properties and how the '_AuthorListTruncated_' flag is used. Test with a paper like arXiv ID 2007.06722v3 that has 948 authors to verify the implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- backend, databases
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100