feat(swarm)!: support multi-node deployment - #301
Merged
Conversation
BREAKING CHANGE: services holding a production volume or a `configurations/` bind mount now require a node labelled `vibetype.storage=true`; label the node before deploying, and deploy with `--force` so the overlay network is recreated at the new MTU.
|
🎉 This PR is included in version 21.0.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The stack currently runs on a single node, where nothing distinguishes services that can move from services that cannot. Adding a second node makes that distinction load-bearing, since Swarm is free to reschedule anything onto the new node on any update, drain, or reboot.
Two things tie a service to one machine. Named volumes use the
localdriver, so a rescheduled service finds an empty volume rather than its data, which forpostgresmeans an empty database served to a live application. Bind mounts underconfigurations/resolve to an absolute path on the node the stack was deployed from, and on any other node Docker silently creates an empty directory there, so the service starts misconfigured instead of failing.Pins the 22 affected services to the node labelled
vibetype.storage=true: the 19 holding a production volume, plusadminer,debezium-postgres-connectorandredpanda-console, which hold only aconfigurations/bind mount. The shared label also keeps the three sets that share a volume together:postgreswithpostgres-backupwithjobber,traefikwithtraefik-certs-dumper, and the sixzammadservices.Constraints live in the production deltas only. Development runs on a single unlabelled node, where the constraint would leave the service unschedulable.
Also declares the stack's overlay network explicitly in order to set its MTU to 1400. Hetzner Cloud's private networks run at 1450 rather than the 1500 the overlay driver assumes, and the VXLAN header takes another 50 bytes. Left at the default, health checks and small requests keep succeeding while large payloads stall.
docs/multi-node.mdrecords which services are pinned and why, the rule for adding a service, and the node setup.Deploying this needs the node label to exist first, and the MTU change needs
--force, since Docker never updates an existing network's options.