indexzero / indexzero/autocache

cli: `waybackify manifest --source <url>` populates `sources`

Open
#10 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Problem

After #8, a manifest can carry sources. There is no command that writes the key. waybackify manifest resolves the links inside a source file and writes entries. A user who knows where a document first ran must edit wayback.json by hand, and must find the Memento by hand.

Proposed solution

Add a --source <url> flag to waybackify manifest.

  • The flag names one Original Resource of the document. It repeats for a merged document.
  • The command resolves each URL to a Memento with the same resolver, --near hint, and --offline rule that entries use.
  • The command writes each result under sources in the output manifest. entries are not affected.
  • When the manifest already has sources, the command merges. A URL already present keeps its Memento. A new URL is added. Nothing is removed.
  • When a URL resolves to no Memento, the command reports it and exits non-zero, the same as an unresolved link. It does not write a partial sources map. (If #8 decides that an unarchived source goes in exclude, the command writes that instead.)

Example:

waybackify manifest post.md -u universe.json -o wayback.json \
  --source http://blog.nodejitsu.com/a-simple-webservice-in-nodejs
Acceptance criteria
  • With --source <url>, the output manifest has one sources entry with a parseable wayback and timestamp.
  • With two --source flags, the map has two entries in sorted order.
  • Rerun on the same file with the same flag: the manifest does not change.
  • Without the flag, the command output is identical to today.
  • --help documents the flag.
Verification
  • Tests pass: new cases in src/waybackify-cli/test/manifest-command.test.js, and the help fixture regenerated.
  • pnpm -r test passes offline.
Dependencies

Blocked by #8.

Files likely touched
  • src/waybackify-cli/src/commands/manifest.js
  • src/waybackify-cli/src/cli.js
  • src/waybackify-cli/test/manifest-command.test.js
  • src/waybackify-cli/README.md
Estimated scope

Small (3-4 files)

Alternatives considered
  • A new waybackify sources add command. Rejected. The resolution is the same work that manifest already does, and one command keeps one write path to wayback.json.
  • Read the source URL from the markdown frontmatter. Rejected. The library has no frontmatter convention, and the flag keeps the input explicit.
Non-goals
  • Fetching the capture. That is #9.
  • A bulk mode over a content tree. A shell loop over the manifests covers the first consumer.
Context
  • Command: src/waybackify-cli/src/commands/manifest.js.
  • First consumer: the charlie.dev backfill of every historical post.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

After dependency #8 is resolved, start with src/waybackify-cli/src/commands/manifest.js and src/waybackify-cli/src/cli.js, then read the existing manifest tests. Add coverage in src/waybackify-cli/test/manifest-command.test.js and regenerate the help fixture. Done means source URLs resolve and merge without changing entries, failures avoid partial writes, help documents the flag, and pnpm -r test passes offline.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.