Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Starts the replication process.
| `includeNamespaces` | string[] | No | List of namespaces to include in replication (for example, ["db.*", "db.collection"]) |
| `excludeNamespaces` | string[] | No | List of namespaces to exclude from replication |
| `cloneSkipPresplit` | boolean | No | Set to `true` to skip pre-splitting collections on the sharded target. {{pcsm.short}} still shards each collection with the source shard key. If you omit this field, {{pcsm.short}} uses the value from its startup configuration, which defaults to `false`. Available starting with version 1.1.0. See [Skip pre-splitting](sharding.md#skip-pre-splitting). |
| `targetWriteConcern` | string | No | Write concern for target data writes during the initial clone and replication. Accepts `"majority"` or a decimal integer string from `"1"` to `"2147483647"`. Defaults to `"majority"` when omitted. Available starting with 1.1.0. See [Configure target write concern](install/usage.md#configure-target-write-concern) for usage and rollback risks. |
| `replNumWorkers` | int | No | Controls how many concurrent replication worker goroutines PCSM uses to apply DML (insert/update/replace/delete) events to the target cluster.|
| `replChangeStreamBatchSize` | int | No | Sets the maximum number of change stream events PCSM will request and read from MongoDB per batch while streaming changes from the source cluster. |
| `replEventQueueSize` | int | No | Controls the size of the internal event queue used by the replication subsystem. |
Expand Down
2 changes: 2 additions & 0 deletions docs/high-availability.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ If a previous active instance resumes after losing its lease, its checkpoint wri

When a standby becomes `ACTIVE`, PCSM resumes replication from the last persisted checkpoint. Failover uses the existing recovery mechanism and happens automatically.

Starting with 1.1.0, HA takeover also retains the run's selected target write concern. HA state and checkpoints always use `majority`, but a lower write concern for data writes can allow acknowledged data to roll back after a target primary failure. See [Configure target write concern](install/usage.md#configure-target-write-concern) before choosing a lower value.

The timings are fixed:

| **Setting** | **Value** |
Expand Down
1 change: 1 addition & 0 deletions docs/install/parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ Alternatively, you can define the following environment variables:
|----------|-------------|---------|
| `PCSM_SOURCE_URI` | MongoDB connection string for the source cluster | - |
| `PCSM_TARGET_URI` | MongoDB connection string for the target cluster | - |
| `PCSM_TARGET_WRITE_CONCERN` | Write concern requested by `pcsm start` for target data writes during the initial clone and replication. Accepts `majority` or a decimal integer from `1` to `2147483647`. The server process ignores this variable. Available starting with 1.1.0. See [Configure target write concern](usage.md#configure-target-write-concern). | `majority` |
| `PCSM_GROUP_NAME` | A name that identifies the HA deployment in member information, API responses, metrics, and logs. | `default` |
| `PCSM_PORT` | Server port number | `2242` |
| `PCSM_CLONE_NUM_PARALLEL_COLLECTIONS` | Number of collections cloned in parallel | `2` |
Expand Down
49 changes: 49 additions & 0 deletions docs/install/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,55 @@

```

### Configure target write concern

!!! admonition "Version added: 1.1.0"

You can choose the [write concern :octicons-link-external-16:](https://www.mongodb.com/docs/v8.0/reference/write-concern/){:target="_blank"} for data written to the target during the initial clone and ongoing replication. Write concern determines how many MongoDB members must acknowledge a write before PCSM continues. The default is `majority`.

Check notice on line 49 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L49 <Google.Acronyms>(https://developers.google.com/style/abbreviations)

Spell out 'PCSM', if it's unfamiliar to the audience.
Raw output
{"message":"Spell out 'PCSM', if it's unfamiliar to the audience.","location":{"path":"docs/install/usage.md","range":{"start":{"line":49,"column":305},"end":{"line":49,"column":309}}},"severity":"INFO","code":{"value":"Google.Acronyms","url":"https://developers.google.com/style/abbreviations"}}

Using `1` requires acknowledgment from the target primary only. This can reduce write stalls when target secondaries lag. The setting applies to data writes only. Checkpoints, high availability (HA) state, and schema changes, such as creating collections and indexes, always use `majority`.

Check notice on line 51 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L51 <Google.Parens>(https://developers.google.com/style/parentheses)

Use parentheses judiciously.
Raw output
{"message":"Use parentheses judiciously.","location":{"path":"docs/install/usage.md","range":{"start":{"line":51,"column":195},"end":{"line":51,"column":199}}},"severity":"INFO","code":{"value":"Google.Parens","url":"https://developers.google.com/style/parentheses"}}

!!! warning "Rollback risk with a lower write concern"

With a write concern below majority, a target primary failure or stepdown can roll back data that PCSM has already recorded as copied. Majority checkpoints do not protect those data writes from rollback. A fresh synchronization may be required.

Keep `majority` when you rely on automatic recovery to preserve acknowledged data. Use a lower value only for a controlled migration where you can restart and validate the copy after a target failure. Avoid target primary stepdowns during the run. Retain the source and verify target consistency before cutover.

Set the write concern when starting a new run. Use `majority` or a decimal integer from `1` to `2147483647`. Numeric values specify the number of data-bearing members that must acknowledge each write. PCSM rejects `0`, negative values, values above this range, and custom named write concerns.

Check notice on line 59 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L59 <Google.Acronyms>(https://developers.google.com/style/abbreviations)

Spell out 'PCSM', if it's unfamiliar to the audience.
Raw output
{"message":"Spell out 'PCSM', if it's unfamiliar to the audience.","location":{"path":"docs/install/usage.md","range":{"start":{"line":59,"column":202},"end":{"line":59,"column":206}}},"severity":"INFO","code":{"value":"Google.Acronyms","url":"https://developers.google.com/style/abbreviations"}}

Check warning on line 59 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L59 <Google.WordListCase>(https://developers.google.com/style/word-list)

Use 'preceding' instead of 'above'.
Raw output
{"message":"Use 'preceding' instead of 'above'.","location":{"path":"docs/install/usage.md","range":{"start":{"line":59,"column":244},"end":{"line":59,"column":249}}},"severity":"WARNING","code":{"value":"Google.WordListCase","url":"https://developers.google.com/style/word-list"}}

The following examples use `1`:

=== "Command line"

```{.bash data-prompt="$"}
$ pcsm start --target-write-concern=1
```

=== "HTTP API"

Send the value as a string in the `/start` request:

```{.bash data-prompt="$"}
$ curl -X POST http://localhost:2242/start \
-H "Content-Type: application/json" \
--data '{"targetWriteConcern": "1"}'
```

=== "Environment variable"

Set the variable for the `pcsm start` command:

```{.bash data-prompt="$"}
$ export PCSM_TARGET_WRITE_CONCERN=1
$ pcsm start
```

The `--target-write-concern` flag overrides `PCSM_TARGET_WRITE_CONCERN`. This environment variable applies only to `pcsm start`. The PCSM server process ignores it. A run started without an explicit write concern uses `majority`, including a run started automatically with the server's `--start` option. Write concern options in the target connection string do not override this setting.

Check notice on line 88 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L88 <Google.Acronyms>(https://developers.google.com/style/abbreviations)

Spell out 'PCSM', if it's unfamiliar to the audience.
Raw output
{"message":"Spell out 'PCSM', if it's unfamiliar to the audience.","location":{"path":"docs/install/usage.md","range":{"start":{"line":88,"column":134},"end":{"line":88,"column":138}}},"severity":"INFO","code":{"value":"Google.Acronyms","url":"https://developers.google.com/style/abbreviations"}}

Check notice on line 88 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L88 <Google.Contractions>(https://developers.google.com/style/contractions)

Use 'don't' instead of 'do not'.
Raw output
{"message":"Use 'don't' instead of 'do not'.","location":{"path":"docs/install/usage.md","range":{"start":{"line":88,"column":359},"end":{"line":88,"column":365}}},"severity":"INFO","code":{"value":"Google.Contractions","url":"https://developers.google.com/style/contractions"}}

The selected value is saved with the run and retained during checkpoint recovery and [HA takeover](../high-availability.md#checkpoint-recovery). You cannot change it with `pcsm resume` or `/resume`. To use a different value, start a new run. Starting a new run drops and recreates the selected target collections, as described in [Start the replication](#start-the-replication).

Check notice on line 90 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L90 <Google.Passive>(https://developers.google.com/style/voice)

In general, use active voice instead of passive voice ('is saved').
Raw output
{"message":"In general, use active voice instead of passive voice ('is saved').","location":{"path":"docs/install/usage.md","range":{"start":{"line":90,"column":20},"end":{"line":90,"column":28}}},"severity":"INFO","code":{"value":"Google.Passive","url":"https://developers.google.com/style/voice"}}

Check notice on line 90 in docs/install/usage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/install/usage.md#L90 <Google.Contractions>(https://developers.google.com/style/contractions)

Use 'can't' instead of 'cannot'.
Raw output
{"message":"Use 'can't' instead of 'cannot'.","location":{"path":"docs/install/usage.md","range":{"start":{"line":90,"column":150},"end":{"line":90,"column":156}}},"severity":"INFO","code":{"value":"Google.Contractions","url":"https://developers.google.com/style/contractions"}}

When a run starts or recovers, the [logs](../logging.md) show `Config: TargetWriteConcern: <value>` for the clone and replication components. If the target slows down during migration, monitor the [source oplog window](../oplog-sizing.md#extend-the-oplog-window-if-the-lag-approaches-its-limit) and extend it before required changes expire.

## Start the filtered replication

You can replicate the whole dataset or specific namespaces - databases and collections. You can specify what namespaces to include and/or exclude from the replication.
Expand Down
1 change: 1 addition & 0 deletions docs/pcsm-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ Available flags:
| `--clone-num-insert-workers` | Number of insert workers that write batches to the target. Shared for all collections.|
| `--clone-segment-size` | Defines the size of each clone segment processed during the initial clone phase.|
| `--clone-skip-presplit` | Skips pre-splitting collections on the sharded target. {{pcsm.short}} still shards each collection with the source shard key but keeps the initial chunk layout that MongoDB creates. Default: `false`. Available starting with version 1.1.0. See [Skip pre-splitting](sharding.md#skip-pre-splitting). |
| `--target-write-concern` | Write concern for target data writes during the initial clone and replication. Accepts `majority` or a decimal integer from `1` to `2147483647`. Default: `majority`. Available starting with 1.1.0. See [Configure target write concern](install/usage.md#configure-target-write-concern) for usage and rollback risks. |
| `--repl-num-workers` | Controls how many concurrent replication worker goroutines PCSM uses to apply DML (insert/update/replace/delete) events to the target cluster.|
| `--repl-change-stream-batch-size`| Sets the maximum number of change stream events PCSM will request and read from MongoDB per batch while streaming changes from the source cluster.|
| `--repl-event-queue-size`| Controls the size of the internal event queue used by the replication subsystem.|
Expand Down
Loading