Skip to content

chore(firestore-bigquery-export): bump change-tracker to 2.2.1, release 0.3.4 - #3223

Merged
CorieW merged 1 commit into
nextfrom
chore/fbe-bump-change-tracker
Sep 30, 2026
Merged

CorieW merged 1 commit into
nextfrom
chore/fbe-bump-change-tracker

Conversation

@CorieW

@CorieW CorieW commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Changes

  • Bump @firebaseextensions/firestore-bigquery-change-tracker in functions from ^2.0.4 to ^2.2.1 (the version the kit already uses). The lockfile still resolves a single firebase-admin 13 and firebase-functions 6.
  • Bump the extension to 0.3.4 and add a changelog entry for the user-facing behaviour changes from change-tracker 2.1.0:
    • When BigQuery rejects an insert, the write now fails: the row goes to BACKUP_COLLECTION, the error is logged, and Cloud Tasks retries it. Before, the extension reported success and silently dropped fields BigQuery did not recognise.
    • Re-running setup no longer updates the table metadata when nothing has changed.

Testing

  • npm run build passes. Unit tests: 54/55 pass. The failure is e2e.test.ts (a table is missing in dev-extensions-testing), and it fails the same way on next.
  • Deployed old (next, 2.0.4) and new (this branch, 2.2.1) side by side on corie-testing, fed both the same Firestore writes and compared their BigQuery output. Two configs: defaults, and wildcard IDs + custom created_at partitioning + clustering. The functions ran as plain Cloud Functions because a local-source extension install was not permitted, and setup (initialize()) was run by hand.
Test Old (2.0.4) New (2.2.1)
Create/update/delete, every Firestore data type, 200-write burst, 20 rapid updates (224 rows) ✅ ✅ identical rows
Wildcard path_params; partition field as Timestamp / ISO string / missing / invalid ✅ ✅ identical
Latest views (rows and SQL) ✅ ✅ identical
Load run after reconfiguring (370 writes) ✅ no errors ✅ identical, no errors
Setup re-run with nothing changed ⚠️ updates the table anyway, every time ✅ does nothing
Clustering added or changed on an existing table ✅ applied ✅ applied
Rewriting an existing view (code path changed in 2.2.1; tested with BigQuery in the same project only) ✅ ✅
Column BigQuery doesn't see yet (old_data dropped) Row saved, field silently dropped (debug log only) Row saved, warning names the stripped column
Permanent insert error (event_id dropped) ❌ Rows saved without event_id, no backup, no warning ✅ Nothing wrong written; backups record the reason, and retries delivered complete rows once the column was restored

@CorieW
CorieW requested a review from a team as a code owner September 30, 2026 13:01

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the firestore-bigquery-export extension to version 0.3.4 and bumps the @firebaseextensions/firestore-bigquery-change-tracker dependency to version 2.2.1. The changelog has been updated to document these changes, which address issues with failed BigQuery inserts and unnecessary metadata updates. As there are no review comments, I have no further feedback to provide.

@CorieW
CorieW merged commit 300a93f into next Sep 30, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants