Engineering health: consolidate architecture, strengthen quality gates, and reduce platform risk

## Summary

A repository-wide architecture and engineering-health review found that MemberJunction has a strong metadata-driven platform core, meaningful test depth, mature CodeGen, and unusually thoughtful migration/CI machinery. The main risk is that capability expansion is outpacing consolidation: critical orchestration is concentrated in very large files, stated type-safety rules are not consistently enforced, package/dependency drift is substantial, and product-level end-to-end coverage is thin.

This issue tracks the follow-up work that is intentionally out of scope for the small documentation-accuracy PR.

@AN-BC @rkihm-bc — please review the findings and help shape owners, sequencing, and which items should split into dedicated issues.

## Evidence snapshot

- 301 workspace packages; 291 public `@memberjunction/*` packages
- Approximately 321k lines of TypeScript across 6,469 TypeScript files
- 1,665 test files and 292 Vitest configurations
- No declared cycles in the internal package dependency graph
- Very high centrality: `@memberjunction/global`, `@memberjunction/core`, and `@memberjunction/core-entities` have 256, 231, and 179 internal dependents respectively
- Large hand-maintained hotspots include:
- `packages/AI/Agents/src/base-agent.ts` — 13.5k lines
- `packages/CodeGenLib/src/Database/manage-metadata.ts` — 6.3k lines
- `packages/AI/Prompts/src/AIPromptRunner.ts` — 5.8k lines
- `packages/MJServer/src/resolvers/IntegrationDiscoveryResolver.ts` — 5.7k lines
- multiple Angular components in the 4k–5.8k line range
- `npm run deps:check` currently reports 602 auto-fixable dependency/version mismatches
- A clean full `npm test` attempt stops during the build dependency phase because `@memberjunction/ai-bridge-vonage` cannot resolve the optional `@vonage/server-sdk` import
- Turborepo reports widespread test-output warnings because normal test tasks declare `coverage/**` outputs without producing coverage
- Only two repository E2E test files were found, with no routine Playwright E2E workflow in `.github/workflows`
- PostgreSQL migration parity is strong, but the migration idempotency check is still advisory (`continue-on-error`)
- No checked-in CodeQL, dependency-update bot, CODEOWNERS, or equivalent security ownership configuration was found (organization-level settings may exist separately)
- 460 planning documents were created or updated within roughly 90 days, alongside 32 releases since April 2, suggesting a need to balance expansion with consolidation

## Recommended workstreams

### P0 — restore a reliably green baseline

- [ ] Fix the Vonage optional-dependency build failure and verify `npm test` completes from a standard checkout
- [ ] Correct Turborepo test outputs/caching so expected behavior does not generate warning noise
- [ ] Triage the 602 dependency mismatches into formatting-only alignment, safe upgrades, and deliberate compatibility splits
- [ ] Add a small deterministic integration smoke tier to PR CI

### P1 — reduce architectural concentration

- [ ] Create an incremental decomposition plan for `BaseAgent`, retaining it as a façade/state machine while extracting realtime, skills/plan mode, sub-agent orchestration, recovery, prompt assembly, and persistence collaborators
- [ ] Identify the top 10 non-generated complexity hotspots and assign characterization-test-backed decomposition work
- [ ] Establish file-size/complexity review thresholds for new growth rather than attempting a risky rewrite
- [ ] Add package-boundary rules around the highly central `global`, `core`, and `core-entities` packages

### P1 — make type-safety policy enforceable

- [ ] Baseline existing explicit `any` usage by package and reject new occurrences in changed lines
- [ ] Enable strict TypeScript settings incrementally, prioritizing high-centrality packages
- [ ] Align contributor guidance with current reality: distinguish existing debt from rules enforced on new code
- [ ] Track `any`, TypeScript suppression, dynamic-import, and lint-disable burn-down metrics

### P1 — add product-level confidence

- [ ] Define five golden-path E2E journeys: bootstrap/login, entity CRUD, agent+Action execution, query/search+permissions, and migration→CodeGen→API/UI verification
- [ ] Run a fast browser subset on PRs and the full matrix nightly/release
- [ ] Move a deterministic integration subset earlier than release-only validation
- [ ] Make PostgreSQL idempotency failures blocking after the existing backlog is triaged

### P1 — security and ownership

- [ ] Confirm organization-level security controls, then add repository-visible CodeQL/dependency review/update automation where missing
- [ ] Add CODEOWNERS for authentication, permissions, Actions/code execution, MCP, Remote Operations, storage, migrations, and publishing
- [ ] Add SBOM/provenance verification and focused threat models for tool execution and external integration surfaces

### P2 — package and product consolidation

- [ ] Classify packages as public SDK, internal boundary, provider plugin, application-only, or generated
- [ ] Merge packages that do not need independent consumption, deployment, permissions, or release cadence
- [ ] Define and document a stable public API; reduce broad `export *` surfaces over time
- [ ] Consider a minimal MJ core distribution plus a batteries-included enterprise/AI distribution
- [ ] Select three flagship user journeys and prioritize installation, docs, samples, telemetry, and E2E quality around them

## Suggested next step

Hold a short architecture/engineering-health triage with maintainers to:

1. Validate or correct the evidence above.
2. Select owners for P0 items.
3. Choose the first two characterization-backed decomposition targets.
4. Split accepted workstreams into scoped implementation issues with measurable acceptance criteria.

The intent is not a rewrite or feature freeze. It is to reserve explicit capacity for consolidation so future capability work becomes safer and faster.