jackwener / jackwener/OpenCLI

Migration tooling: yaml-only adapters silently un-loaded in v1.7.x — propose `adapter migrate` codemod

Open
#1,519 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
29.5k
Forks
2.9k
Avg merge
15h 36m
Merged PRs (30d)
70

Description

## Context

We maintained ~172 custom local-override adapters across 5 sites (`lingxing`, `sellerboard`, `sellersprite`, `egate`, `accio`) under `~/.opencli/clis//*.yaml`. After upgrading to v1.7.x (currently on 1.7.18), every yaml-only adapter silently stopped registering at the top-level CLI:

```bash
$ opencli adapter status
Local overrides in ~/.opencli/clis/ (6 sites):
amazon-sc [custom] # has .js + .yaml.migrated → works
sellerboard [custom] # has .yaml only → does NOT register
lingxing [custom] # same
sellersprite [custom] # same
egate [custom] # same
accio [custom] # same

$ opencli sellerboard dashboard
error: unknown command 'sellerboard'
```

There's no warning, no error. `opencli adapter status` lists the adapter as installed but `opencli list` and `opencli ` show nothing. The contract changed from "yaml or js" to "js only" without a migration affordance.

We worked around this by writing a yaml → js codemod and running it on all 172 files. Want to surface the gap and offer the codemod upstream.

## Real-world impact

Before/after stats from our workspace:

| Adapter | yaml files | Commands registered before | Commands registered after |
|---|---|---|---|
| lingxing | 102 | 0 | 102 |
| sellerboard | 6 | 0 | 6 |
| sellersprite | 26 | 0 | 25 |
| egate | 5 | 0 | 5 |
| accio | 33 | 0 | 33 |
| **Total** | **172** | **0** | **170** |

`opencli list 2>&1 | grep "Failed to load"` returns empty post-migration.

## What the codemod does

Repo: [`codemod-yaml-to-js.mjs` gist](https://gist.github.com/usernametooshort/cb5d138f7ab9ff380f9e666a43e6c3d3)

Pattern reverse-engineered from `amazon-sc/keepalive.yaml.migrated ↔ keepalive.js`:

- `pipeline: [navigate, wait, evaluate]` → `func: async (page, kwargs) => { await page.goto(...); return await page.evaluate(...) }`
- Jinja `${{ args.X | default(Y) }}` → `${X}` (with Node-side destructure)
- Jinja `${{ args.X | tojson }}` → `${JSON.stringify(X)}`
- Reserved-name collisions (`page`, `kwargs`, `url`, `result`) auto-renamed to `${name}_arg`
- Strategy mapping: `cookie` → `Strategy.COOKIE`, etc.
- `args: { foo: { type: int, default: 50 } }` → `args: [{ name: 'foo', type: 'number', default: 50, ... }]`

## Known limitations (documented in codemod source)

After running on 172 files, 4 cases needed hand-fixes:

1. **`Strategy.PUBLIC` / `Strategy.LOCAL` adapters**: the codemod generates `func: async (page, kwargs)` with `page.goto(...)`, but `BrowserStrategy = Exclude` — public/local don't get a `page` arg. Runtime fails with `page.goto is not a function`. Affected our 2 login adapters. Fix path: codemod should branch on strategy and emit a non-browser `func: async (kwargs) => { ... fetch(...) ... }` shape instead.

2. **Jinja object lookup**: `${{ {'US':'1','DE':'4'}[args.market] | default('4') }}` — codemod's regex doesn't recognize object literals inside Jinja braces. Emits an `UNSUPPORTED_JINJA` comment marker. Hand-fix: lift to Node-side `const MARKET_ID_MAP = {...}`.

3. **Nested template literals inside JS evaluate body**: yaml `evaluate:` block with template literals like \`\${a.textContent.trim()}→\${a.href.replace(/https?:\/\/[^/]+/,'')}\` clashes with the outer `page.evaluate(\`...\`)` template literal. Workaround: hand-write with `String.raw` and avoid nested `${}` substitutions.

4. **`domain: ''` empty value**: causes cookie-store lookup failure for Strategy.COOKIE. Codemod doesn't validate. Fix: warn on empty domain when strategy is COOKIE.

## Three possible shapes upstream might want

**A. Re-enable yaml loading as a fallback** in the adapter loader, with a deprecation warning that points users to `opencli adapter migrate `. Maintains backwards compatibility.

**B. Ship the codemod as a new subcommand** `opencli adapter migrate [site]` that handles the 4 known cases gracefully (strategy detection, Jinja object lookup support, validation warnings). Migration becomes self-service.

**C. Document the migration path in CHANGELOG + migration guide** + accept the codemod as a `scripts/migrate-yaml-to-js.mjs` contrib script. Lowest-friction; users discover it via docs.

Personally I think B is the cleanest user experience, but happy to follow whatever direction you prefer.

## Offering a PR

Happy to draft a PR for whichever shape you'd want (A, B, or C). My codemod is ESM with one runtime dep (`yaml` package), ~200 lines. The 4 edge case handlers are ~50 LoC each. Would need to:
- Convert ESM → TS to match your code style
- Add tests using your existing test infrastructure
- Wire the subcommand into `src/cli.ts` adapter subcommand tree
- Update CHANGELOG + docs

Estimated 2-3 hours of focused work. Let me know if you want this and which shape, and I'll spin a PR.

## Workspace info

- opencli v1.7.18, extension v1.0.12 (both current as of 2026-05-13)
- node v25.5.0
- macOS arm64
- 870 total opencli commands across 143 sites after migration

Contributor guide

Open the contributing guide

Research direction

Start with the adapter command tree in src/cli.ts and compare the YAML-only adapters with the amazon-sc .js/.yaml.migrated pair described in the issue. Review codemod-yaml-to-js.mjs and the existing test infrastructure before choosing among fallback loading, a migrate subcommand, or documentation. Done criteria depend on the selected shape, including coverage for the four listed edge cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, yaml
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.