Repository navigation
embedded read paths: take embedded mode from backup tags, fix #1671 - #1672
Merged
Merged
Conversation
download/restore/restore_remote/upload checked CAS disks and resolved the local embedded backup path from clickhouse.use_embedded_backup_restore, so an embedded backup created with --env CLICKHOUSE_USE_EMBEDDED_BACKUP_RESTORE=1 failed on Antalya CAS disks unless the override was repeated for every command. Relaxing only the disk check was not enough: EmbeddedBackupDataPath stayed empty, restore could not find metadata.json and download wrote to a relative path. - checkDisksConsistency gets isEmbeddedBackup, evaluated after the backup metadata is read; create keeps the config-only behaviour - EmbeddedBackupDataPath is resolved by the `embedded` tag on read paths, restore also looks for metadata.json on the embedded backup disk - restore opens the remote destination for an embedded backup without embedded_backup_disk by the tag - embedded ON CLUSTER fan-out of restore/restore_remote/delete local is enabled by the tag when use_embedded_backup_restore_cluster is set, remote tags come from the existing remoteCloudBackupPrefix listing, workers get --env CLICKHOUSE_USE_EMBEDDED_BACKUP_RESTORE=1 because they validate the flag before reading the tags Fixes #1671 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage Report for CI Build 42384833894Coverage increased (+0.08%) to 70.745%Details
Uncovered Changes
Coverage Regressions16 previously-covered lines in 6 files lost coverage.
Coverage Stats
💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1671
Problem
download/restore/restore_remote/uploadof an embedded backup requiredclickhouse.use_embedded_backup_restore: true(or--env CLICKHOUSE_USE_EMBEDDED_BACKUP_RESTORE=1on every call), even though the backup metadata already carries theembeddedtag. On Antalya CAS disks this failed in ~50ms withdisk "cas" ... has no local files, set clickhouse.use_embedded_backup_restore: true.Relaxing only the disk check (the fix suggested in the issue) is not enough: the local embedded backup path (
EmbeddedBackupDataPath) was also resolved from the config flag only, so with the check relaxedrestorefailed withread backup metadata file: ... no such file or directoryanddownloadwrote into a relative path.Fix
createkeeps deciding by the config; read paths decide by the backupembeddedtag:checkDisksConsistency(disks, isEmbeddedBackup)runs after the backup metadata is known indownload/restore/upload(streamingrestore_remoterejects embedded backups before the check, so it gets the specific error instead of the misleading CAS one)EmbeddedBackupDataPathis resolved by the tag (setEmbeddedBackupDataPathByTags),restorealso looks formetadata.jsonon the embedded backup diskrestoreopens the remote destination for an embedded backup withoutembedded_backup_diskby the taguse_embedded_backup_restore_clusterset,restore,restore_remoteanddelete localof an embedded backup drive the other nodes too (embeddedClusterInitiator(isEmbeddedBackup)), remote tags come from the listingremoteCloudBackupPrefixalready does, worker commands get--env="CLICKHOUSE_USE_EMBEDDED_BACKUP_RESTORE=1"because a worker validates the flag before it reads the tagsTests
TestCheckDisksConsistency(CAS + embedded tag without config flag),TestEmbeddedOnClusterWorkerValidation(fan-out by tag),TestEmbeddedClusterRestoreWorkerCommand(--envin worker command, shlex round trip);make testpassesTestAntalyaCASDisk(altinity/clickhouse-server:26.6.8.20002.altinityantalya):download,restore,uploadof the embedded backup withCLICKHOUSE_USE_EMBEDDED_BACKUP_RESTORE=false— passes locally (Antalya is not in the CI matrix)embedded on cluster/restore remote on cluster by backup tags: two shards,restore_remote --env=CLICKHOUSE_USE_EMBEDDED_BACKUP_RESTORE=falsevia API, worker gets the--env, row counts match —/clickhouse backup/embedded*7/7 ok locally🤖 Generated with Claude Code