firebase / firebase/extensions
fix(firestore-bigquery-change-tracker): partitioning e2e suite fails on next and the package never runs in CI
- Dominant language
- TypeScript
- Stars
- 979
- Forks
- 433
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 127
Description
Running the change tracker's jest suite on `next` (c3522bf4) against live BigQuery fails 8 of the 25 tests in `src/__tests__/bigquery/e2e.test.ts`, all in the partitioning block. Nobody noticed because the root `jest.config.js` `projects` list includes `*/functions` and `gen-schema-view` but not `firestore-bigquery-export/firestore-bigquery-change-tracker`, so CI has never run this package's tests. The 12 failures in `backupSettings.test.ts` that #3134 had to fix went through CI green for the same reason.
The 8 failures have four causes, all introduced by the partitioning refactor 442722a3 (#2447) and one follow-on in 11b34063:
1. Package regression. `src/bigquery/partitioning/config.ts` `determineType` only rejects a falsy or `NONE` granularity, so an unrecognised value such as `UNKNOWN` is classed as ingestion time and `src/bigquery/partitioning/index.ts` sends `timePartitioning: { type: "UNKNOWN" }` to `createTable`, which BigQuery rejects with `Invalid value for timePartitioning.type: UNKNOWN is not a valid value`. The pre-refactor code whitelisted HOUR, DAY, MONTH and YEAR. Test: "does not partition with an unrecognized timePartitioning option".
2. Four tests had their invalid inputs replaced with valid ones during the refactor but kept `expect(metadata.timePartitioning).toBeUndefined()`: the "without a valid timePartitioningField", "without a valid timePartitioningFieldType", "unknown timePartitioningFieldType" and "unknown timePartitioningFirestoreField" tests.
3. Two stale assertions: the "`timestamp` as field name" test asserts `DAY` after its config moved to `HOUR`, and "valid schema with timePartitioning only" asserts a warn log the refactor deleted; the logger spy is also never cleared between tests.
4. 11b34063 made `tableRequiresUpdate` add a missing custom partition column to an existing table, so "does not update add a custom partitioning column" now sees one column added where it expects none. Intent needs deciding.
Suggested fix: validate the granularity in `determineType` and omit `timePartitioning` on an unrecognised value (the existing TODO there), restore genuinely invalid inputs in the four tests, correct the two stale assertions, clear the logger spy in `beforeEach`, decide test 4 against 11b34063, and add the package to the root jest `projects` so CI runs it. Two related gaps for the CI entry: the suite needs live BigQuery credentials, and on firebase-admin 14 the ts-jest config cannot load the ESM-only `jose` that firebase-functions v2 pulls in, so `transformIgnorePatterns` needs adjusting before the admin 14 path is covered.
Found while validating #3134; the same 8 fail with the pre-#3134 test file, so this is independent of that change.
Contributor guide
Research direction
Start with src/bigquery/partitioning/config.ts, src/bigquery/partitioning/index.ts, and src/__tests__/bigquery/e2e.test.ts; run the partitioning suite against live BigQuery. Review jest.config.js and the package's ts-jest setup for CI coverage, credentials, and the firebase-admin 14 jose issue. Done means the listed tests pass, the pending table-update behavior is decided, and the package runs in CI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, typescript
- Domain
- backend, ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100