Automattic / Automattic/data-liberation-agent

Feedback from testing runs with Wix and Shopify

Open
#25 5 comments 0 reactions 1 assignee Claimed by @borkweb View on GitHub
Dominant language
TypeScript
Stars
31
Forks
3
Avg merge
10h 14m
Merged PRs (30d)
81

Description

My feedback from two separate runs, one from Wix and one from Shopify:

- The scraping/extraction from Wix took a long time.
- This might be improved by spawning multiple Node.js worker threads/processes to extract pages in parallel (e.g., a pool of 4 browser instances for Wix, or concurrent HTTP fetchers for Squarespace).
- When I got to the step about whether or not I had a site, I selected N, then it gave me a URL on the second step that does not work.
- `Create an Application Password at WordPress Admin > Users > Profile > Application Passwords (WordPress.com: http://wordpress.com/me/security/application-passwords)`
- I could not get it to connect to [WordPress.com](http://wordpress.com/) Simple at all.
- This whole process needs clarity, and would benefit from a clear, simple flow on [WordPress.com](http://wordpress.com/), perhaps with an OAuth flow. If it needs a paid plan that includes plugins, be explicit about that get send someone down a flow to set that up.
- I eventually switched to Pressable and got an application password from a fresh install there, which did connect.
- Media import worked well from both Wix and Shopify.
- Both runs, the Page import failed part of the way through with the Node error below, and no way to restart the process.
- The page imports are buggy.
- Zero pages from the Wix site imported content, only titles. The WXR seems to contain all of the content in the excerpt:encoded tag, not the content:encoded tag.
- Some pages from the Shopify site imported content, but not all of the content. Lots of the text was missing.

Right now I cannot get past the page import failures for further testing.

```
ERROR Cannot read properties of undefined (reading 'total')

src/ui/import.tsx:128:21

125:
126: {STAGES.map((stage) => {
127: const s = result[stage];
128: if (s.total === 0) return null;
129: return (
130:
131:

- (src/ui/import.tsx:128:21)
- at Array.map ()\t
- Import (src/ui/import.tsx:126:21)
- Object.react_stack_bottom_frame (node_modules/react-reconciler/cjs/react-reconciler.development.js:17596:20)
- renderWithHooks (node_modules/react-reconciler/cjs/react-reconciler.development.js:5335:22)
- updateFunctionComponent (node_modules/react-reconciler/cjs/react-reconciler.development.js:7720:19)
- beginWork (node_modules/react-reconciler/cjs/react-reconciler.development.js:9277:18)
- runWithFiberInDEV (node_modules/react-reconciler/cjs/react-reconciler.development.js:2505:30)
- performUnitOfWork (node_modules/react-reconciler/cjs/react-reconciler.development.js:15273:22)
- workLoopSync (node_modules/react-reconciler/cjs/react-reconciler.development.js:15099:41)

TypeError: Cannot read properties of undefined (reading 'total')
at (/Users/cagrimmett/Projects/data-liberation-agent/src/ui/import.tsx:128:21)
at Array.map ()
at Import (/Users/cagrimmett/Projects/data-liberation-agent/src/ui/import.tsx:126:21)
at Object.react_stack_bottom_frame (/Users/cagrimmett/Projects/data-liberation-agent/node_modules/react-reconciler/cjs/react-reconciler.development.js:17596:20)
at renderWithHooks (/Users/cagrimmett/Projects/data-liberation-agent/node_modules/react-reconciler/cjs/react-reconciler.development.js:5335:22)
at updateFunctionComponent (/Users/cagrimmett/Projects/data-liberation-agent/node_modules/react-reconciler/cjs/react-reconciler.development.js:7720:19)
at beginWork (/Users/cagrimmett/Projects/data-liberation-agent/node_modules/react-reconciler/cjs/react-reconciler.development.js:9277:18)
at runWithFiberInDEV (/Users/cagrimmett/Projects/data-liberation-agent/node_modules/react-reconciler/cjs/react-reconciler.development.js:2505:30)
at performUnitOfWork (/Users/cagrimmett/Projects/data-liberation-agent/node_modules/react-reconciler/cjs/react-reconciler.development.js:15273:22)
at workLoopSync (/Users/cagrimmett/Projects/data-liberation-agent/node_modules/react-reconciler/cjs/react-reconciler.development.js:15099:41)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.