Dallinger / Dallinger/Dallinger

Remove ec2 extra

Open
#8,902 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

technical debt
Dominant language
Python
Stars
123
Forks
44
Avg merge
1d 16h
Merged PRs (30d)
6

Description

I propose removing the few dependencies required by the ec2 extra. Here's an autogenerated scoping:

Current state

  • In this branch, ec2 currently includes numpy and pandas only (no tqdm).pyproject.toml:99-102
  • EC2 code is coupled to pandas in dallinger/command_line/lib/ec2.py via DataFrame, concat, merge, query, groupby, to_numeric, and to_markdown.ec2.py:13 ec2.py:138 ec2.py:164-169 ec2.py:220 ec2.py:250-287 ec2.py:301-304 ec2.py:855-856 ec2.py:985-1015
  • EC2 tests also assume DataFrame return types and APIs (columns, iloc, etc.).test_ec2.py:4 test_ec2.py:42-44 test_ec2.py:90-107
  • EC2 CLI registration is currently guarded by optional import + “install ec2 extra” messaging.__init__.py:126-134

Proposed refactor scope

  1. Replace DataFrame-based EC2 internals with plain Python (list[dict] + small helper functions):

    • filtering by state/pem
    • selecting one instance by name/dns
    • grouping and joining instance pricing details
    • cost and uptime formatting/sorting
      (all currently done by pandas ops).ec2.py:250-287 ec2.py:301-304 ec2.py:985-1015
  2. Replace DataFrame to_markdown() output with Rich table rendering (render_rich_table) so EC2 has no extra tabular dependency.ec2.py:138 ec2.py:304 ec2.py:352 ec2.py:369 utils.py:68-123

  3. Update EC2 tests to assert behavior/output semantics on plain objects rather than DataFrame API specifics.test_ec2.py:42-44 test_ec2.py:90-107

  4. Packaging cleanup:

    • remove numpy/pandas from [project.optional-dependencies].ec2
    • decide whether to keep an empty ec2 extra as a compatibility alias or remove it entirely
    • optionally remove ec2 from dev install aggregates if no longer needed.pyproject.toml:99-102 dev-requirements.in:1 constraints.in:1

Effort and risk

  • Effort: small-to-medium (~0.5–1 day including tests).
  • Main risk: preserving exact filtering/sorting/formatting behavior currently provided by pandas (especially cost computation and missing-value handling).ec2.py:279-291
  • API compatibility risk: low for CLI users, moderate for external code importing EC2 helper functions and expecting DataFrames.

Optional follow-up refactors

  • Once EC2 has no optional deps, remove the EC2 import guard and always register the command in CLI bootstrap.__init__.py:126-134
  • Replace string-based query logic with explicit Python predicates to avoid quoting/escaping edge cases in instance names/DNS values.ec2.py:985-991

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the EC2 implementation in dallinger/command_line/lib/ec2.py and its tests in test_ec2.py, then inspect pyproject.toml, init.py, utils.py, dev-requirements.in, and constraints.in. Run the EC2 tests before changing behavior. Done means EC2 filtering, selection, pricing, formatting, CLI registration, tests, and dependency configuration work without the pandas or numpy extra.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
build-system, cli, testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.