ns8-imapsync

Install

Instantiate the module with:

add-module imapsync

The output of the command will return the instance name. Output example:

{"module_id": "imapsync1", "image_name": "imapsync", "image_url": "ghcr.io/nethserver/imapsync:1.0.5"}

Configure

Let's assume that the imapsync instance is named imapsync1.

We need to bind imapsync to a mail server inside the cluster, we use the MODULE_UUID of mail server to configure imapsync, reveal the vmail master secret of local users and start the container waiting for cron tasks or individual user tasks

Launch configure-module, by setting the following parameters:

mail_server: module uuid of the mail server
mail_host: local ip of the mail server

Example:

api-cli run module/imapsync1/configure-module --data '{
    "mail_server":"8dd3b3fe-609c-42f7-a2d1-cecba9461bea",
    "mail_host":"10.5.4.1"
}'

The above command will start and configure the imapsync instance.

start to sync a remote imap account to local user account

Example:

api-cli run module/imapsync1/create-task --data '{
    "localuser": "administrator",
    "remotehostname": "imap.foo.com",
    "remoteport": 143,
    "security": "tls",
    "delete_local": false,
    "delete_remote": false,
    "delete_remote_older": 0,
    "exclude": "folder1,folder2",
    "remoteusername": "username",
    "remotepassword": "password",
    "cron": "5m",
    "task_id": "a0241w",
    "foldersynchronization": "all"
}'

Field descriptions:

security: tls, ssl, or "" (empty for no security)
delete_local: true or false – if true, deletes mails and folders on the local account that are not in the remote account.
delete_remote: true or false – if true, deletes mails on the remote account after synchronization to the local account.
delete_remote_older: Number of days after which remote emails are deleted. If set to 0, messages are removed after the transfer.
exclude: Comma-separated list of folders to exclude (e.g., folder1,folder2,^folder3$). Use ^ to mark folder names that must start with a pattern and $ for those that must end with one.
cron: Defines task scheduling (e.g., 5m runs every 5 minutes, "2h" runs every 2 hours). Internally converted to crontab format (0/5 * * * * for minutes, 1 */2 * * * for hours).
remote credentials: Provide the username and password of the remote IMAP account. Alternatively, use the IMAP master administrator credentials (e.g., user*vmail with the corresponding password).
foldersynchronization: Controls which folders are synchronized:
- all – synchronizes all folders.
- inbox – synchronizes only the inbox.
- exclusion – synchronizes all folders except excluded ones.
task_id: Unique identifier for the task, automatically generated by the UI.
sieve_enabled: Boolean (true or false). If true, applies the user's Sieve filters to synchronized messages. This option is only considered when foldersynchronization is set to inbox.

delete env and stop a running synchronisation

Example:

api-cli run module/imapsync1/delete-task --data '{
    "localuser":"administrator"
}'

stop a running synchronisation

Example:

api-cli run module/imapsync1/stop-task --data '{
    "localuser":"administrator"
}'

start a running synchronisation

Example:

api-cli run module/imapsync1/start-task --data '{
    "localuser":"administrator"
}'

start all configured tasks

Read all imapsync/*.env files and start the synchronization

Example:

api-cli run module/imapsync1/start-all-tasks

stop all configured tasks

Read all imapsync/*.env files and stop the synchronization

Example:

api-cli run module/imapsync1/stop-all-tasks

read configuration

Example:

api-cli run module/imapsync1/get-configuration

Answer:

{
  "mail_server": "e8a6177c-9ae5-4356-826b-0a5f93b2dbaf",
  "mail_host": "10.5.4.1",
  "mail_server_URL": [
    {
      "name": "mail2",
      "label": "mail2 (R3.rocky9-3.org)",
      "value": "e8a6177c-9ae5-4356-826b-0a5f93b2dbaf,10.5.4.1"
    }
  ],
}

list-tasks

Read configuration from environment and from imapsync/*.{env,pwd}

api-cli run module/imapsync1/list-tasks

{
  "enabled_mailboxes": [
    {
      "name": "administrator",
      "label": "administrator",
      "value": "administrator"
    },
    {
      "name": "foo",
      "label": "foo",
      "value": "foo"
    },
    {
      "name": "john",
      "label": "john",
      "value": "john"
    }
  ],
  "user_properties": [
    {
      "task_id": "qu1dcb",
      "localuser": "administrator",
      "remoteusername": "user2@domain.com",
      "remotehostname": "imap.domain.com",
      "remoteport": "143",
      "security": "tls",
      "delete_local": false,
      "deletefolder": "",
      "exclude": "",
      "delete_remote": false,
      "delete_remote_older": 0,
      "sieve_enabled": false,
      "expunge_remote": "",
      "cron": "",
      "folder_inbox": "",
      "foldersynchronization": "all",
      "remotepassword": "password",
      "service_running": false
    },
    {
      "task_id": "vd2am3",
      "localuser": "john",
      "remoteusername": "user@domain.com",
      "remotehostname": "imap.domain.com",
      "remoteport": "143",
      "security": "tls",
      "delete_local": false,
      "deletefolder": "",
      "exclude": "",
      "delete_remote": false,
      "delete_remote_older": 0,
      "sieve_enabled": false,
      "expunge_remote": "",
      "cron": "",
      "folder_inbox": "",
      "foldersynchronization": "all",
      "remotepassword": "password",
      "service_running": false
    }
  ]
}

list-informations

You can retrieve the email and folder numbers and the email size for the local and remote account Example:

api-cli run module/imapsync9/list-informations --data '{
  "localuser": "john",
  "task_id": "vd2am3"
  }'

output:

{"status": true, "host1Folders": 78, "host2Folders": 78, "host1Messages": 164625, "host2Messages": 155, "host1Sizes": 3060488650, "host2Sizes": 72036894}

in case of error you will have

{"status": false}

custom scripts

If necessary, you can add your custom scripts inside the container. To facilitate this, two volumes, included in the backup, are mounted from their respective directories under the ./state module.

cron (e.g. ~imapsync1/.config/state/cron) This folder is designated for storing cron files, each of which should end its name with .custom, while adhering to the cron syntax.
```
1 */1 * * *  root /usr/bin/imapsync --host1 SOURCE_SERVER --user1 SOURCE_USERNAME --password1 SOURCE_PASSWORD --host2 DESTINATION_SERVER --user2 DESTINATION_USERNAME --password2 DESTINATION_PASSWORD
```
Instead of directly add an imapsync command, you have the option to run a script, which can be stored in the imapsync volume.
```
1 */1 * * *  root /etc/imapsync/script.custom
```
imapsync (e.g. ~imapsync1/.config/state/imapsync) In this location, you can store custom scripts (ending with .custom) intended for execution by cron. These scripts must be set to be executable by all users.

troubleshot issues

The state/imapsync folder contains the following files:

Environment files (*.env) – Stores task-related environment variables.
Password files (*.pwd) – Contains credentials. The special vmail.pwd file holds the Dovecot master user's credentials.
Lock files (*.lock) – Created when a task is running to prevent multiple simultaneous executions.
Log files (*.log) – Stores the last Imapsync run log. This file is overwritten with each task execution. A summary, including the number of transferred messages and the Imapsync exit code, is always sent to the system log.

You can filter these files by task name, which is a combination of the localuser and the task_id.

Example of files for two tasks, both for local user foo. One of them is running, the other is stopped:

.config/state/
├── cron
│   ├── foo_a0241w.cron
│   └── foo_a1j44r.cron
└── imapsync
    ├── foo_a0241w.env
    ├── foo_a0241w.lock
    ├── foo_a0241w.pwd
    ├── foo_a0241w.log
    ├── foo_a1j44r.env
    ├── foo_a1j44r.pwd
    ├── foo_a1j44r.log
    └── vmail.pwd

to launch manually a task

runagent -m imapsync1
podman exec -ti imapsync /usr/local/bin/syncctl start foo_a1j44r
podman exec -ti imapsync /usr/local/bin/syncctl stop foo_a1j44r
podman exec -ti imapsync /usr/local/bin/syncctl status foo_a1j44r

Bulk import tasks from CSV

Create multiple synchronization tasks at once using a CSV file with the import-csv-tasks utility script.

Overview

This Python script automates the bulk creation of IMAP synchronization tasks by reading user data from a CSV file and calling the create-task API endpoint for each entry. It's ideal for migrating multiple email accounts from a remote IMAP server to your local mail system.

CSV File Format

The CSV file must contain a header row with these 6 required columns:

This users.csv file can be found at .config/examples/users.csv

runagent -m imapsync1
cat ../examples/users.csv

localusername,remoteusername,remotepassword,remotehostname,remoteport,security
pansy.dumbledore5,user1@example.org,"enquotedPasswordIfSeparatorInside",imap.example.org,993,ssl
lavender.umbridge7,user2@example.org,"enquotedPasswordIfSeparatorInside",imap.example.org,993,ssl
dolores.slughorn3,user3@example.org,"enquotedPasswordIfSeparatorInside",imap.example.org,143,tls

Required columns:

localusername – Local user account name (must exist on your mail server)
remoteusername – Remote IMAP account username/email
remotepassword – Remote IMAP account password (quote if contains special characters)
remotehostname – Remote IMAP server hostname or IP
remoteport – Remote IMAP server port (typically 993 for SSL/TLS, 143 for STARTTLS)
security – Security protocol: ssl, tls, or empty string "" for no encryption

Important notes:

Column order does not matter – the script maps columns by header name
Delimiter is comma (,)
Passwords with special characters should be quoted: "myP@ss,word"
All fields are mandatory except security – rows with empty required values will be rejected
The security field can be empty (for no encryption), but the other 5 fields must contain values
Empty lines in the CSV are automatically skipped

Usage

The script is designed to be used with NS8 runagent to bulk import synchronization tasks.

Basic usage (file redirection):

runagent -m imapsync1 import-csv-tasks < users.csv

Alternative (using pipe):

cat users.csv | runagent -m imapsync1 import-csv-tasks

Check CSV format before creating tasks (check-only mode):

runagent -m imapsync1 import-csv-tasks -c < users.csv

This performs comprehensive validation without making API calls:

Checks CSV structure (required columns present)
Validates delimiter (comma-separated)
Verifies all mandatory fields have values (except security which can be empty)
Validates port numbers are numeric
Reports specific errors for each invalid row
Reports the rows that would be skipped as already configured

Example validation output:

📋 CSV Column Validation:
   Delimiter: ',' (comma-separated)
   Found 6 column(s): localusername, remotehostname, remotepassword, remoteport, remoteusername, security
   Column order: does not matter (mapped by header name)
✓ All 6 required columns present
✓ Found 2 data row(s) (empty lines skipped)

🔍 Validating row data...
✓ All 2 rows validated successfully
✓ Generated 2 unique task IDs

✓ CSV file is valid. No tasks were created (check-only mode).

Display help:

runagent -m imapsync1 import-csv-tasks -h

Options

-c, --check – Check-only mode: validates CSV format without creating tasks
-u, --update – Apply the file to tasks that already exist instead of skipping them
-h, --help – Display comprehensive help message

Rows already configured

A row is skipped when that local user already synchronizes the same remote account on the same server. The comparison ignores case, and catches rows repeated inside a single file. This makes an import safe to re-run: fix a couple of lines, run it again, and only the missing tasks are created. A skipped row is not an error, so the exit code stays 0.

The port and the security protocol are not part of the comparison: the same account reached on 143 or 993 is the same mailbox. An IP address and a hostname of the same server do count as two different accounts.

Use -u, --update to apply the file to the existing tasks instead, which is how credentials are changed in bulk. The task keeps its ID, its log and its status, so no duplicate is created. If the new credentials are refused, the task keeps its previous settings.

Features

Flexible input methods:
- Reads from standard input (stdin)
- Works with file redirection (<), pipes (|), or heredoc
- Compatible with NS8 runagent for direct module integration
- No file path arguments needed
CSV parsing:
- Comma-separated delimiter (,) - required
- Validates all 6 required columns are present before processing
- Handles quoted fields with embedded delimiters
- Reports missing columns with clear error messages
- Skips empty lines automatically
Data validation:
- Mandatory field checking: ensures 5 required fields are not empty (localusername, remoteusername, remotepassword, remotehostname, remoteport)
- Optional field: security can be empty (for no encryption)
- Port validation: ensures remoteport contains only numeric values
- Check mode (-c flag): validates all rows without creating tasks
- Clear error messages with row numbers for easy debugging
Automatic field population:
- Generates unique random 6-character task ID (lowercase alphanumeric) for each user
- Auto-fills optional fields with sensible defaults:
  - cron: "" (empty - no automatic scheduling)
  - delete_local: false (preserve local emails)
  - delete_remote: false (preserve remote emails)
  - delete_remote_older: 0 (no age-based deletion)
  - exclude: "" (no folder exclusions)
  - foldersynchronization: "all" (sync all folders)
  - sieve_enabled: false (don't apply Sieve filters)
Robust error handling:
- Validates port numbers are numeric
- Continues processing remaining users if one fails
- Displays detailed error messages per user
- Returns non-zero exit code if any task fails
Progress tracking:
- Shows real-time progress for each user creation
- Displays success/failure status per task
- Final summary counting tasks created, failed, updated and skipped
API integration:
- Calls action/create-task for each user
- Passes data as properly formatted JSON
- Captures and displays API response output

Example Output

Create 3 synchronization tasks from a CSV file:

runagent -m imapsync1 import-csv-tasks < users.csv

Reading CSV from standard input...

📋 CSV Column Validation:
   Delimiter: ',' (comma-separated)
   Found 6 column(s): localusername, remotehostname, remotepassword, remoteport, remoteusername, security
   Column order: does not matter (mapped by header name)
✓ All 6 required columns present
✓ Found 3 data row(s) (empty lines skipped)

📦 Starting task creation with module: imapsync1
   Processing 3 user(s)...

Creating task for pansy.dumbledore5 (ID: a1b2c3)...
✓ Success: pansy.dumbledore5

Creating task for lavender.umbridge7 (ID: d4e5f6)...
✓ Success: lavender.umbridge7

Creating task for dolores.slughorn3 (ID: g7h8i9)...
✓ Success: dolores.slughorn3

======================================================================
📊 Summary: 3 created, 0 failed
======================================================================

Troubleshooting

"Missing required columns"

Check CSV header row contains all 6 required column names exactly as specified
Verify no typos in column names (case-sensitive)

"Missing required value(s)"

The script rejects rows with empty mandatory fields
Ensure all 5 required fields have values: localusername, remoteusername, remotepassword, remotehostname, remoteport
Only security can be empty (for no encryption)
Use -c flag to validate CSV before creating tasks

"Invalid port number"

Ensure remoteport column contains only numeric values (e.g., 993, 143)

Rows were skipped instead of created

That local user already synchronizes the same remote account on that server. Run with -c to list what would be skipped, or with -u to apply the file to the existing tasks.

Remove all tasks after a bad import

If you need to clean up after a failed or incorrect bulk import, you can remove all task configuration files:

# Stop all running tasks first
api-cli run module/imapsync1/stop-all-tasks

# Remove all task configuration files (replace imapsync1 with your module ID)
rm -f /home/imapsync1/.config/state/imapsync/*.env
rm -f /home/imapsync1/.config/state/imapsync/*.pwd
rm -f /home/imapsync1/.config/state/imapsync/*.log
rm -f /home/imapsync1/.config/state/imapsync/*.lock
rm -f /home/imapsync1/.config/state/cron/*.cron

# Verify cleanup
ls -la /home/imapsync1/.config/state/imapsync/

Note: This preserves the vmail.pwd file (master password). To start fresh completely, you can also remove it, but you'll need to reconfigure the module afterward.

Uninstall

To uninstall the instance:

remove-module --no-preserve imapsync1

Running tests locally

This module uses the NS8 standard testing infrastructure. For instructions on how to run the test suite locally, refer to the Running tests locally section of the ns8-github-actions repository.

UI translation

Translated with Weblate.

To setup the translation process:

add GitHub Weblate app to your repository
add your repository to [hosted.weblate.org]((https://hosted.weblate.org) or ask a NethServer developer to add it to ns8 Weblate project

Name		Name	Last commit message	Last commit date
.devcontainer		.devcontainer
.github/workflows		.github/workflows
.vscode		.vscode
imageroot		imageroot
imapsync/usr/local/bin		imapsync/usr/local/bin
tests		tests
ui		ui
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
build-images.sh		build-images.sh
imapsync-delete1older.patch		imapsync-delete1older.patch
imapsync-setflag1.patch		imapsync-setflag1.patch
imapsync-sievedelivery2.patch		imapsync-sievedelivery2.patch
renovate.json		renovate.json

Navigation Menu

Repository navigation

About

Repository files navigation

ns8-imapsync

Install

Configure

start to sync a remote imap account to local user account

delete env and stop a running synchronisation

stop a running synchronisation

start a running synchronisation

start all configured tasks

stop all configured tasks

read configuration

list-tasks

list-informations

custom scripts

troubleshot issues

Bulk import tasks from CSV

Overview

CSV File Format

Usage

Options

Rows already configured

Features

Example Output

Troubleshooting

Uninstall

Running tests locally

UI translation

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

ns8-imapsync

Install

Configure

start to sync a remote imap account to local user account

delete env and stop a running synchronisation

stop a running synchronisation

start a running synchronisation

start all configured tasks

stop all configured tasks

read configuration

list-tasks

list-informations

custom scripts

troubleshot issues

Bulk import tasks from CSV

Overview

CSV File Format

Usage

Options

Rows already configured

Features

Example Output

Troubleshooting

Uninstall

Running tests locally

UI translation

About

Resources

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages