DataTalksClub / DataTalksClub/website

A readable non-export source should refuse by name, not by traceback

Open
#331 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug data-migration needs grooming operations P2 testing
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Pointed at a path that exists but is not a CMP export — an empty file, or a
SQLite database with a different schema — scripts/prod/import_cmp_learners.py
raises a raw sqlite3.OperationalError instead of the bounded
source-unreadable refusal its siblings give.

Found by the tester on #322 while exercising the new Make targets against a
zero-byte stand-in.

The three CMP importers share a deliberate refusal contract: _readonly(...)
_refuse("source-unreadable"), with a named condition code rather than a
traceback (courses/services/cmp_content_import.py:305-312,
accounts/services/cmp_learner_import.py:346-353,
courses/services/cmp_learner_history_import.py:303-310). That contract is why
#322's Make-level guard was scoped to a named operator message rather than to
closing a hole — the importers were already assumed to fail closed on a bad
source.

They do fail closed on an unreadable source. A readable file with the wrong
contents is a different condition, and it surfaces as a stack trace naming a
table, which tells an operator at 2am less than the condition code would.

Severity is low and deliberate to record as such: it fails loudly, writes
nothing, and leaks nothing beyond a table name. It is filed so the refusal
contract is uniform rather than mostly-uniform.

Scope: extend the source check to distinguish "not readable" from "readable but
not a CMP export", with a bounded condition code for the second. Whether the
other two importers need the same treatment should be checked rather than
assumed.

Explicit non-goal: any change to what the importers do with a valid export.

Refs #310, #322

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in scripts/prod/import_cmp_learners.py and compare its source handling with _readonly(...) and _refuse("source-unreadable") in the three service files named in the issue. Run the importer against an empty file or SQLite database with a different schema, then check the other two importers for the same condition. Done means a readable non-CMP source returns a bounded condition code instead of a raw traceback, without changing valid-export behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.