Skip to content

fix(traefik): harden the Traefik update and rollback - #5881

Merged
imrja8 merged 2 commits into
canaryfrom
fix/traefik-update-rename-stopped
Nov 9, 2026
Merged

imrja8 merged 2 commits into
canaryfrom
fix/traefik-update-rename-stopped

Conversation

@imrja8

@imrja8 imrja8 commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

What is this PR about?

Follow-up to #5756, which added the in-place Traefik update with automatic rollback. This builds on that flow to cover a few more real-world cases, so an update either finishes with everything carried over or leaves the old Traefik exactly as it was.

The first case came up on a production host, where the update stopped at its first step with:

could not add service state for endpoint dokploy-traefik-previous to cluster on UpdateDNSNames: cannot create entry in table endpoint_table ... already exists

When a running container on a swarm overlay network is renamed, Docker re-registers its DNS names in swarm's shared network table, and on some hosts that table is out of sync with the daemon. A stopped container is renamed without touching the network at all (see the !ctr.State.Running early return in moby's daemon/rename.go), so the update now stops Traefik first. Traefik was being stopped right after the rename anyway, so downtime is unchanged. Nothing was lost on that host: Docker reverts the name and Traefik kept running.

Changes

Update

  • Stop Traefik before renaming it to dokploy-traefik-previous. If the rename fails, start it again before reporting the error.
  • Join the new container to every network the old one was on (besides dokploy-network), with aliases, before starting it. If joining fails, the update rolls back.

Restore after an interrupted update

  • Runs through the same per-server queue as updates. Toggling the dashboard or saving Traefik env/ports runs writeTraefikSetup, whose restore step ran outside the queue. In the few seconds where only the backup exists, it could rename the backup back, and the update's rollback would then remove it.
  • A new dokploy-traefik left in created state (the update was interrupted before starting it) is removed and the backup brought back. A Traefik that ran and was stopped is left alone.
  • A running backup is stopped before it's renamed, and started again even if the rename fails.

Reconnecting isolated compose networks

  • Each docker network connect now ends with || true. Traefik can already be on those networks, and a compose network that no longer exists made the command exit non-zero, so a successful update showed "Last update failed".

What an update keeps

Env vars, custom ports, traefik.yml, dynamic config and acme.json (host files, mounted in), dokploy-network, isolated compose networks, and now any other network Traefik was on. As with Reload or a port/env change, extra volume mounts, labels, static IPs and host-IP-specific port bindings come from Dokploy's settings, not the old container.

Tests

  • The fake Docker in standalone-traefik-recreate.test.ts rejects renaming a running container with the same error, so every recreate test checks the order.
  • New tests cover: networks carried over; rollback when a network can't be joined; the old Traefik restarted when it can't be moved aside; a restore waiting for an in-progress update (fails with Conflict: dokploy-traefik is already in use without the queue); a never-started container cleared; a stopped Traefik left alone; a running backup stopped before it's renamed back.
  • pnpm typecheck and Biome pass. pnpm test passes apart from application.real.test.ts, which needs nixpacks installed locally.

Checklist

Before submitting this PR, please make sure that:

  • You created a dedicated branch based on the canary branch.
  • You have read the suggestions in the CONTRIBUTING.md file https://github.com/Dokploy/dokploy/blob/canary/CONTRIBUTING.md#pull-request
  • You have tested this PR in your local instance. If you have not tested it yet, please do so before submitting. This helps avoid wasting maintainers' time reviewing code that has not been verified by you.

Issues related (if applicable)

Follow-up to #5756

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new actionable issue was established.

Summary

Hardens Traefik replacement and recovery after an interrupted update.

  • Traefik updates stage a replacement and keep the old container ready to restore.
  • Interrupted Traefik updates wait for active swaps before restoring containers.
  • Isolated network reconnect errors no longer turn an update into a failure.

The earlier restart finding is addressed: recovery now uses dokploy-traefik-next to detect an update interrupted before the old container was renamed.

Reviews (2) · Last reviewed commit: "fix(traefik): recover an update interrup..." · Reviewed by Greptile

@imrja8
imrja8 requested a review from Siumauricio as a code owner November 5, 2026 13:45

@narcisonunez narcisonunez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@imrja8
imrja8 merged commit 1048bcb into canary Nov 9, 2026
5 checks passed
@imrja8
imrja8 deleted the fix/traefik-update-rename-stopped branch November 9, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants