Meta: repository-level linting for data managers and tool data tables

## Summary

Planemo applies ordinary tool-wrapper lint to data managers and can execute data-manager tests, but it does not validate a data-manager/reference-data repository as a connected bundle.

This meta issue proposes a repository-level linting block covering:

- data-manager wrappers and `data_manager_conf.xml`;
- `tool_data_table_conf.xml`, `.sample`, and `.test` files;
- referenced `.loc` / `.loc.sample` fixtures;
- literal `data_manager_json` examples; and
- expanded tool consumers using `from_data_table` or `data_ref`.

The goal is to automate objective bundle-contract checks before adding more requirements to the IUC reviewer checklist.

## Motivation and evidence

A frozen audit of 6,791 `galaxyproject/tools-iuc` pull requests and their review comments combined text/history searches with a current-tree producer/configuration/consumer graph.

The current-tree scan found:

- 82 data managers;
- 297 table consumers;
- 689 table configurations;
- 655 loc paths; and
- 1,321 producer/configuration/consumer relationships.

Its highest-confidence static queue contained ten missing test loc fixtures and six loc/configuration row-shape inconsistencies. A separate review of nine rare sentinels confirmed four schema/row issues, three missing or ineffective fixture contracts, one path/configuration problem, and one macro-expansion false positive.

Representative cases:

- schema/separator/row problems: galaxyproject/tools-iuc#3381, galaxyproject/tools-iuc#4524, galaxyproject/tools-iuc#4533, galaxyproject/tools-iuc#4671;
- missing fixture/configuration problems: galaxyproject/tools-iuc#2682, galaxyproject/tools-iuc#3215, galaxyproject/tools-iuc#3867, galaxyproject/tools-iuc#4953, galaxyproject/tools-iuc#6089;
- cross-component table naming/design: galaxyproject/tools-iuc#2080, galaxyproject/tools-iuc#2736, galaxyproject/tools-iuc#3151, galaxyproject/tools-iuc#6933; and
- required macro-expansion false-positive guard: galaxyproject/tools-iuc#5003.

This overlaps several long-standing Planemo issues:

- #96 — check sample loc files exist;
- #531 — require `tool_data_table_conf.xml.sample` when needed;
- #706 — missing table configuration is not noticed; and
- #869 — detect empty/undocumented loc files.

Issue #96 records IUC cases that passed review and failed later during Tool Shed update. The proposed work should consolidate rather than duplicate those issues.

## Current behavior and gap

`planemo lint` delegates individual expanded wrapper sources to `galaxy-tool-util`. Generic linters now run on `tool_type="manage_data"`, but they only see an individual wrapper and check ordinary command/input/output/test/help/citation rules.

`planemo shed_lint --tools` adds repository and Tool Shed metadata checks, but it does not parse `data_manager_conf.xml`, `tool_data_table_conf.xml*`, and loc assets into one graph.

`planemo test` discovers `tool_data_table_conf.xml.test`, falling back to `.sample`, and can execute a data manager. That is runtime setup rather than lint. Planemo's own data-manager fixture still contains `TODO: need some way to test that new entry was added to data table`.

Galaxy already has reusable runtime behavior for loc-file resolution, `.sample` fallback, configured-separator parsing, short-row detection, and unknown returned table names. The missing layer is deterministic repository lint that surfaces these conditions before Galaxy startup or Tool Shed installation.

## Proposed repository model

Build a reusable *realized data-table bundle* containing, with source locations:

- expanded tool/data-manager wrappers;
- manager IDs, declared outputs, and `output_ref` values;
- manager-configured table names and column mappings;
- configured table names, types, columns/indexes, separators, duplicate policy, and file paths;
- resolved `.test` and `.sample` loc assets;
- literal/example `data_manager_json` table names and keys; and
- consumer references from `from_data_table`, `data_ref`, filters, validators, and output actions.

Every diagnostic should distinguish:

- direct, statically resolved evidence;
- unresolved dynamic/macro/generated evidence; and
- tables intentionally supplied by Galaxy core or another installed repository.

Unresolved cases should be reported as advisory or “not checked,” never as demonstrably missing.

## P0 static checks

- [ ] Parse every repository-owned `tool_data_table_conf.xml.test` and `.sample` selected for the realized bundle.
- [ ] Resolve recognized path variables such as `${__HERE__}` and fail when a literal `.test` fixture path does not exist.
- [ ] Apply Galaxy's configured separator and fail when a non-comment loc row cannot supply every declared column index.
- [ ] Resolve literal `from_data_table` consumers after macro/token expansion and validate them against local definitions.
- [ ] Validate literal manager table names against configured tables.
- [ ] Validate `data_manager_conf.xml` `output_ref` values against real outputs in the expanded manager wrapper.
- [ ] Detect duplicate column names/indexes and conflicting schemas for the same local table.
- [ ] Validate the expected local bundle is complete: manager/configuration, production sample configuration/loc, test configuration/loc, and consumer references as applicable.
- [ ] Detect repository-owned absolute paths and unresolved literal paths that cannot be portable through Tool Shed installation.

### Proposed severity

Hard errors should initially be limited to conditions Planemo can prove:

- a referenced literal test fixture is absent;
- a loc row is too short for a declared index or uses an unmistakably wrong separator;
- an `output_ref` names no expanded wrapper output;
- a literal local table name conflicts across components; or
- a local configuration is structurally ambiguous.

## P1 warnings to calibrate

- [ ] Warn when a loc row contains extra unused fields. Galaxy currently requires enough fields to address declared indexes but permits extras, so exact-width enforcement needs an explicit policy decision.
- [ ] Warn when a local `from_data_table` reference has no repository definition but might be supplied externally.
- [ ] Warn when a `.loc.sample` is empty and contains no comment documenting its format; do not require default data rows.
- [ ] Warn when no declared functional test appears to activate a literal cached-table-backed branch.
- [ ] Warn on duplicate rows in shipped loc fixtures and, if adopted as a convention, on an unspecified `allow_duplicate_entries` policy.
- [ ] Treat literal producer coverage as discovery-only when arbitrary Python, shell, or Cheetah constructs `data_manager_json`.

## False-positive and compatibility requirements

1. **Expand macros before resolving names.** In galaxyproject/tools-iuc#5003, raw `@IDX_DATA_TABLE@` text looks unconfigured but resolves to a complete versioned table bundle.
2. **Model externally supplied tables.** Galaxy core or another installed repository may validly provide a consumer table.
3. **Follow Galaxy row semantics.** Too few fields is demonstrably invalid; extra ignored fields are not necessarily invalid today.
4. **Do not execute arbitrary producer code during lint.** Literal/example JSON can be validated; dynamic producers require functional tests.
5. **Preserve source locations and linter names** so diagnostics can be skipped narrowly and rendered as PR annotations.

## Functional contract tests: follow-up, not static lint

The following are important but cannot be proven reliably by static analysis. They should become a separate Planemo data-manager contract-test mode or follow-up issue:

- run a manager twice and verify intentional reject/replace/coexist behavior (examples: galaxyproject/tools-iuc#2654 and galaxyproject/tools-iuc#7949);
- apply and inspect the produced table update, not only compare the `data_manager_json` file;
- run an ordinary consumer against the cached entry;
- mount cached/reference inputs read-only to detect writes (example: galaxyproject/tools-iuc#4105);
- run from a relocated profile/workspace to expose administrator-local assumptions (example: galaxyproject/tools-iuc#4707); and
- inject download/checksum/extraction/build failures and verify that no loc row or partial reference is published.

Reference-source suitability, compatibility, licensing, checksum policy, and the semantics of duplicate/update behavior remain maintainer decisions. Static lint can enforce those only after a schema or convention exists.

## Implementation sketch

The cross-file model and core validations should be reusable `galaxy-tool-util` logic. Planemo can invoke them from its repository-aware `shed_lint` path and as a preflight for `planemo test`.

Keeping the implementation solely in the current per-wrapper linter would not provide enough repository-boundary or cross-file context.

Suggested sequence:

1. Introduce the realized-bundle model and parser tests.
2. Implement path existence and minimum-row-shape checks.
3. Implement expanded consumer/configuration/manager name relationships.
4. Implement `output_ref`, conflicting-schema, and bundle-completeness checks.
5. Add warnings and calibrate them across `tools-iuc` before promoting any to errors.
6. Close or supersede #96, #531, #706, and #869 as their scopes are implemented.
7. Open the functional contract-test follow-up with a concrete harness design.

## Definition of done

- [ ] The P0 checks have unit tests derived from confirmed IUC cases and counterexamples.
- [ ] Macro-generated and externally supplied tables have explicit non-error behavior.
- [ ] The lints run from a repository-aware Planemo command and before functional tests.
- [ ] Diagnostics identify the exact component/path/table and explain the violated relationship.
- [ ] Existing issues #96, #531, #706, and #869 are linked, resolved, or explicitly left for remaining scope.
- [ ] The new checks are calibrated against `tools-iuc`, with false positives reviewed before error-level enforcement.
- [ ] A separate dynamic contract-test issue captures idempotence, actual table application/consumption, write isolation, relocation, and failure atomicity.

Completing this block should leave any future IUC data/reference checklist focused on behavioral and policy judgments rather than mechanically detectable bundle errors.

Start by reading Planemo's repository-aware shed_lint path and the existing galaxy-tool-util wrapper lint behavior, then review how planemo test discovers data-manager fixtures. Define the realized-bundle model and parser tests before implementing the listed P0 checks. Done means repository-aware lint runs before functional tests, has source-specific diagnostics, and includes confirmed cases and counterexamples without false positives for macros or external tables.

Written by the indexing model from the issue text.