internetarchive / internetarchive/openlibrary
Refactor Goodreads Import to Server-Side Handler & Improve Shelf Support
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Feature Request
## Problem / Opportunity
The current Goodreads import flow:
- Works entirely on the frontend, making complexlogic (Did not finish shelf, custom lists) hard to implement
- Fails silently when no ISBNs are present
- Cannot be reused for onboarding
- Does not have any analytics linked
- Most importantly, currently - does not even work with the abovementioned features.
This results in a confusing patron experience and restricts maintainance of the feature.
By upgrading this code, and moving to a server-side handler we can make the importer more powerful and support features such as custom lists. This step will also be a stepping stone for integraging goodreads imports into other places (such as patron onboarding flow) if needed.
- Working server side handler that correctly imports the books
- Importing of custom shelves
- Improved UI/UX with better feedback
- Addition of analytics tools will allow us to measure the adoption of this feature and in what ways import is failing the most - in order to direct our efforts in that direction
## Proposal
1. Switch importing to a server side handler which is closer to the database and infogami for easier imports
2. Add analytics tools to track import statistics
3. Improve UI for more feedback and clarity
## Current Flow (Client-side)
Patron uploads file -> POSTed to backend, reads and returns, books (with isbns) and books_wo_isbns -> Patron selects books and clicks import -> Frontend makes all the get/post requests to different api routes to insert books -> Shows incremental results [SLOW]
## Proposed Flow (Server-side)
Patron uploads file -> POSTed to backend, reads and returns, books (with isbns) and books_wo_isbns -> Patron selects books and clicks import -> Frontend makes 1 post request to backend with all the relevant data -> Backend handles insertions and everthing -> Returns final state of import [MUCH Faster]
Benefits:
- Faster
- Easier to maintain
- Supports complex logic (custom shelves, batching)
### Breakdown
Implementation Details (for maintainers)
#### Related files
Refer to [this map of common Endpoints](https://docs.openlibrary.org/developers/backend/endpoints.html):
- `openlibrary/plugins/upstream/account.py` - class process_imports POST request handles the importing
- `openlibrary/templates/account/import.html` - frontend
#### Requirements Checklist
Checklist of requirements that need to be satisfied in order for this issue to be closed:
* [x] Upgrade the frontend UI to render two different tables for importable/unimportable books (based on ISBN)
* [x] New backend handler that manages the import and performs all operations
* [x] Creates custom lists
* [x] Adds books to lists / default shelves
* [x] Imports ratings
* [x] Imports reviews
* [ ] Add tracking for analytics
#### Stakeholders
*
#### Instructions for Contributors
* **Before** [creating a new branch](https://docs.openlibrary.org/developers/tools/git.html#making-changes-and-creating-a-pull-request) or pushing up changes to a PR, please first [run these commands](https://docs.openlibrary.org/developers/tools/git.html#working-on-your-branch) to ensure your repository is up to date, as the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Assessment
This issue has not been assessed yet.