diff --git a/docs/api.md b/docs/api.md index fd3c4a0d..643d767d 100644 --- a/docs/api.md +++ b/docs/api.md @@ -27,6 +27,7 @@ Starts the replication process. | `includeNamespaces` | string[] | No | List of namespaces to include in replication (for example, ["db.*", "db.collection"]) | | `excludeNamespaces` | string[] | No | List of namespaces to exclude from replication | | `cloneSkipPresplit` | boolean | No | Set to `true` to skip pre-splitting collections on the sharded target. {{pcsm.short}} still shards each collection with the source shard key. If you omit this field, {{pcsm.short}} uses the value from its startup configuration, which defaults to `false`. Available starting with version 1.1.0. See [Skip pre-splitting](sharding.md#skip-pre-splitting). | +| `targetWriteConcern` | string | No | Write concern for target data writes during the initial clone and replication. Accepts `"majority"` or a decimal integer string from `"1"` to `"2147483647"`. Defaults to `"majority"` when omitted. Available starting with 1.1.0. See [Configure target write concern](install/usage.md#configure-target-write-concern) for usage and rollback risks. | | `replNumWorkers` | int | No | Controls how many concurrent replication worker goroutines PCSM uses to apply DML (insert/update/replace/delete) events to the target cluster.| | `replChangeStreamBatchSize` | int | No | Sets the maximum number of change stream events PCSM will request and read from MongoDB per batch while streaming changes from the source cluster. | | `replEventQueueSize` | int | No | Controls the size of the internal event queue used by the replication subsystem. | diff --git a/docs/high-availability.md b/docs/high-availability.md index d074c713..ce94a87e 100644 --- a/docs/high-availability.md +++ b/docs/high-availability.md @@ -49,6 +49,8 @@ If a previous active instance resumes after losing its lease, its checkpoint wri When a standby becomes `ACTIVE`, PCSM resumes replication from the last persisted checkpoint. Failover uses the existing recovery mechanism and happens automatically. +Starting with 1.1.0, HA takeover also retains the run's selected target write concern. HA state and checkpoints always use `majority`, but a lower write concern for data writes can allow acknowledged data to roll back after a target primary failure. See [Configure target write concern](install/usage.md#configure-target-write-concern) before choosing a lower value. + The timings are fixed: | **Setting** | **Value** | diff --git a/docs/install/parameters.md b/docs/install/parameters.md index 0d235f4c..1fdf27e2 100644 --- a/docs/install/parameters.md +++ b/docs/install/parameters.md @@ -36,6 +36,7 @@ Alternatively, you can define the following environment variables: |----------|-------------|---------| | `PCSM_SOURCE_URI` | MongoDB connection string for the source cluster | - | | `PCSM_TARGET_URI` | MongoDB connection string for the target cluster | - | +| `PCSM_TARGET_WRITE_CONCERN` | Write concern requested by `pcsm start` for target data writes during the initial clone and replication. Accepts `majority` or a decimal integer from `1` to `2147483647`. The server process ignores this variable. Available starting with 1.1.0. See [Configure target write concern](usage.md#configure-target-write-concern). | `majority` | | `PCSM_GROUP_NAME` | A name that identifies the HA deployment in member information, API responses, metrics, and logs. | `default` | | `PCSM_PORT` | Server port number | `2242` | | `PCSM_CLONE_NUM_PARALLEL_COLLECTIONS` | Number of collections cloned in parallel | `2` | diff --git a/docs/install/usage.md b/docs/install/usage.md index e89a2326..adcd9821 100644 --- a/docs/install/usage.md +++ b/docs/install/usage.md @@ -42,6 +42,55 @@ Then it uses [change streams :octicons-link-external-16:](https://www.mongodb.co ``` +### Configure target write concern + +!!! admonition "Version added: 1.1.0" + +You can choose the [write concern :octicons-link-external-16:](https://www.mongodb.com/docs/v8.0/reference/write-concern/){:target="_blank"} for data written to the target during the initial clone and ongoing replication. Write concern determines how many MongoDB members must acknowledge a write before PCSM continues. The default is `majority`. + +Using `1` requires acknowledgment from the target primary only. This can reduce write stalls when target secondaries lag. The setting applies to data writes only. Checkpoints, high availability (HA) state, and schema changes, such as creating collections and indexes, always use `majority`. + +!!! warning "Rollback risk with a lower write concern" + + With a write concern below majority, a target primary failure or stepdown can roll back data that PCSM has already recorded as copied. Majority checkpoints do not protect those data writes from rollback. A fresh synchronization may be required. + + Keep `majority` when you rely on automatic recovery to preserve acknowledged data. Use a lower value only for a controlled migration where you can restart and validate the copy after a target failure. Avoid target primary stepdowns during the run. Retain the source and verify target consistency before cutover. + +Set the write concern when starting a new run. Use `majority` or a decimal integer from `1` to `2147483647`. Numeric values specify the number of data-bearing members that must acknowledge each write. PCSM rejects `0`, negative values, values above this range, and custom named write concerns. + +The following examples use `1`: + +=== "Command line" + + ```{.bash data-prompt="$"} + $ pcsm start --target-write-concern=1 + ``` + +=== "HTTP API" + + Send the value as a string in the `/start` request: + + ```{.bash data-prompt="$"} + $ curl -X POST http://localhost:2242/start \ + -H "Content-Type: application/json" \ + --data '{"targetWriteConcern": "1"}' + ``` + +=== "Environment variable" + + Set the variable for the `pcsm start` command: + + ```{.bash data-prompt="$"} + $ export PCSM_TARGET_WRITE_CONCERN=1 + $ pcsm start + ``` + +The `--target-write-concern` flag overrides `PCSM_TARGET_WRITE_CONCERN`. This environment variable applies only to `pcsm start`. The PCSM server process ignores it. A run started without an explicit write concern uses `majority`, including a run started automatically with the server's `--start` option. Write concern options in the target connection string do not override this setting. + +The selected value is saved with the run and retained during checkpoint recovery and [HA takeover](../high-availability.md#checkpoint-recovery). You cannot change it with `pcsm resume` or `/resume`. To use a different value, start a new run. Starting a new run drops and recreates the selected target collections, as described in [Start the replication](#start-the-replication). + +When a run starts or recovers, the [logs](../logging.md) show `Config: TargetWriteConcern: ` for the clone and replication components. If the target slows down during migration, monitor the [source oplog window](../oplog-sizing.md#extend-the-oplog-window-if-the-lag-approaches-its-limit) and extend it before required changes expire. + ## Start the filtered replication You can replicate the whole dataset or specific namespaces - databases and collections. You can specify what namespaces to include and/or exclude from the replication. diff --git a/docs/pcsm-commands.md b/docs/pcsm-commands.md index b90814ba..5e351a61 100644 --- a/docs/pcsm-commands.md +++ b/docs/pcsm-commands.md @@ -68,6 +68,7 @@ Available flags: | `--clone-num-insert-workers` | Number of insert workers that write batches to the target. Shared for all collections.| | `--clone-segment-size` | Defines the size of each clone segment processed during the initial clone phase.| | `--clone-skip-presplit` | Skips pre-splitting collections on the sharded target. {{pcsm.short}} still shards each collection with the source shard key but keeps the initial chunk layout that MongoDB creates. Default: `false`. Available starting with version 1.1.0. See [Skip pre-splitting](sharding.md#skip-pre-splitting). | +| `--target-write-concern` | Write concern for target data writes during the initial clone and replication. Accepts `majority` or a decimal integer from `1` to `2147483647`. Default: `majority`. Available starting with 1.1.0. See [Configure target write concern](install/usage.md#configure-target-write-concern) for usage and rollback risks. | | `--repl-num-workers` | Controls how many concurrent replication worker goroutines PCSM uses to apply DML (insert/update/replace/delete) events to the target cluster.| | `--repl-change-stream-batch-size`| Sets the maximum number of change stream events PCSM will request and read from MongoDB per batch while streaming changes from the source cluster.| | `--repl-event-queue-size`| Controls the size of the internal event queue used by the replication subsystem.|