NUKnightLab / NUKnightLab/StoryMapJS

Import fails on large ZIP exports

Open
#487 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
248
Forks
61
Avg merge
1m
Merged PRs (30d)
1

Description

In discussion in #304 we found that StoryMap creates export ZIP files containing all files associated with a project, sometimes creating a ZIP which is larger than the 20MB maximum upload supported by the system. Thus, the export cannot be re-imported.

In this specific case, there were many files in the export which were not in active use in the storymap. This is unsurprising, since there's no "delete" functionality. It's also the case that many of the images were much larger than they reasonably needed to be for display in a storymap which would almost certainly have been smaller (in pixel dimensions) than the image file.

The workaround was to create a new ZIP, omitting all image files which weren't referenced in the JSON files in the export. This imported without any issue.

We might consider filtering the images part of the export process to only include images which are actively used in the media, background, marker, or sharing image locations. It's possible in theory that we'd still create an export larger than 20MB. I don't really think it's a good idea to increase that threshold, but then, I don't know what we'd do besides tell people "sorry."

I suppose we could also consider downsampling uploaded images, although that should go in its own issue if we decide it's the right way to go.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the export and import entry points discussed in #304, focusing on how media, background, marker, and sharing image references are represented in the exported JSON. Reproduce the large-ZIP case and verify that an export containing only referenced images can be imported without exceeding the 20MB upload limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
web-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.