internetarchive / internetarchive/openlibrary

Refactor Goodreads Import to Server-Side Handler & Improve Shelf Support

Open
#12,277 6 comments 0 reactions 1 assignee Claimed by @Sadashii View on GitHub
Lead: @jimchamp Needs: Assignee Check-in Needs: Breakdown Needs: Response Needs: Review Assignee Priority: 3 Theme: Onboarding Type: Feature Request
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Feature Request

## Problem / Opportunity

The current Goodreads import flow:
- Works entirely on the frontend, making complexlogic (Did not finish shelf, custom lists) hard to implement
- Fails silently when no ISBNs are present
- Cannot be reused for onboarding
- Does not have any analytics linked
- Most importantly, currently - does not even work with the abovementioned features.

This results in a confusing patron experience and restricts maintainance of the feature.

By upgrading this code, and moving to a server-side handler we can make the importer more powerful and support features such as custom lists. This step will also be a stepping stone for integraging goodreads imports into other places (such as patron onboarding flow) if needed.

- Working server side handler that correctly imports the books
- Importing of custom shelves
- Improved UI/UX with better feedback
- Addition of analytics tools will allow us to measure the adoption of this feature and in what ways import is failing the most - in order to direct our efforts in that direction

## Proposal

1. Switch importing to a server side handler which is closer to the database and infogami for easier imports
2. Add analytics tools to track import statistics
3. Improve UI for more feedback and clarity

## Current Flow (Client-side)
Patron uploads file -> POSTed to backend, reads and returns, books (with isbns) and books_wo_isbns -> Patron selects books and clicks import -> Frontend makes all the get/post requests to different api routes to insert books -> Shows incremental results [SLOW]

## Proposed Flow (Server-side)
Patron uploads file -> POSTed to backend, reads and returns, books (with isbns) and books_wo_isbns -> Patron selects books and clicks import -> Frontend makes 1 post request to backend with all the relevant data -> Backend handles insertions and everthing -> Returns final state of import [MUCH Faster]

Benefits:
- Faster
- Easier to maintain
- Supports complex logic (custom shelves, batching)

### Breakdown

Implementation Details (for maintainers)

#### Related files

Refer to [this map of common Endpoints](https://docs.openlibrary.org/developers/backend/endpoints.html):
- `openlibrary/plugins/upstream/account.py` - class process_imports POST request handles the importing
- `openlibrary/templates/account/import.html` - frontend

#### Requirements Checklist
Checklist of requirements that need to be satisfied in order for this issue to be closed:
* [x] Upgrade the frontend UI to render two different tables for importable/unimportable books (based on ISBN)
* [x] New backend handler that manages the import and performs all operations
* [x] Creates custom lists
* [x] Adds books to lists / default shelves
* [x] Imports ratings
* [x] Imports reviews
* [ ] Add tracking for analytics

#### Stakeholders

*


#### Instructions for Contributors

* **Before** [creating a new branch](https://docs.openlibrary.org/developers/tools/git.html#making-changes-and-creating-a-pull-request) or pushing up changes to a PR, please first [run these commands](https://docs.openlibrary.org/developers/tools/git.html#working-on-your-branch) to ensure your repository is up to date, as the pre-commit bot may add commits to your PRs upstream.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.