fmsg-docker

Dockerised stack composing a full fmsg setup including: fmsgd, fmsgid and fmsg-webapi

Other Docs

Name	Description
QUICKSTART.md	Get a production stack up and running on your server in minutes.
README_LOCAL_DEV.md	Run the stack locally for development purposes.

Contents

• Structure
• Services
• Persistent Data Volumes
• Integration Tests
• Getting Started
• Environment Variables
  • General
  • Database
    • Database Init Scripts

Structure

fmsg-docker/
├── docker/
│   ├── fmsgd/
│   │   └── Dockerfile        # builds fmsgd from source
│   ├── fmsgid/
│   │   └── Dockerfile        # builds fmsgid from source
│   └── fmsg-webapi/
│       └── Dockerfile        # builds fmsg-webapi from source
│
├── compose/
│   ├── docker-compose.yml    # full fmsg stack
│   └── .env                  # environment configuration
│
└── README.md

Services

Service	Description
postgres	PostgreSQL database shared by fmsgd, fmsgid and fmsg-webapi
fmsgid	fmsg Id HTTP API — manages users and quotas
fmsgd	fmsg host — sends and receives fmsg messages
fmsg-webapi	fmsg Web API — HTTP interface to the fmsg db
fmsg-mcp	Optional (--profile mcp): fmsg-mcp MCP server so AI agents can use fmsg through the Web API

Optional MCP server

fmsg-mcp is not started by default. It serves the Model Context Protocol over Streamable HTTP on port 8765 (bound to 127.0.0.1 unless FMSG_MCP_HOST_PORT says otherwise); every MCP client sends its own fmsg API key as Authorization: Bearer fmsgk_..., so one instance serves all users of the host.

# alongside the stack
docker compose --profile mcp up -d

# behind a TLS-terminating reverse proxy (e.g. Caddy: `mcp.example.com { reverse_proxy 127.0.0.1:8765 }`)
FMSG_MCP_ALLOWED_HOSTS=mcp.example.com docker compose --profile mcp up -d

FMSG_MCP_API_URL overrides the Web API URL the server talks to (default https://fmsgapi.<FMSG_DOMAIN>); FMSG_MCP_REF pins the npm version (default latest). Give the proxy an idle timeout of at least 240 s: the wait_for_message tool holds a request open for up to FMSG_MCP_WAIT_MAX_SECONDS.

Persistent Data Volumes

The compose stack uses Docker named volumes:

Volume	Mounted at	Used by	Contents
postgres_data	/var/lib/postgresql/data	postgres	All PostgreSQL databases and WAL
fmsg_data	/opt/fmsg/data	fmsgd, fmsg-webapi	fmsg host data (keys, messages)
fmsgid_data	/opt/fmsgid/data	fmsgid	fmsgid data (addresses CSV)
letsencrypt	/etc/letsencrypt	certbot, fmsgd, fmsg-webapi	Let's Encrypt TLS certificates

WARNING: These volumes contain sensitive application data including user identities and messages. Restrict access to the Docker host and the volumes directory accordingly.

Ensure you have a backup plan for both volumes. Data loss from a volume being deleted or corrupted is not recoverable without backups. Access to backups should equally restricted - consider encryption needs.

Getting Started

1. Copy the example environment file and edit it:

   cp .env.example compose/.env
   

   Set all required variables in compose/.env:

   FMSG_DOMAIN=example.com
   CERTBOT_EMAIL=admin@example.com
   FMSG_API_TOKEN_ED25519_PRIVATE_KEY=<base64-ed25519-seed>
   FMSGD_WRITER_PGPASSWORD=<strong random password>
   FMSGID_WRITER_PGPASSWORD=<strong random password>
   

2. On the first run, supply the one-time initialisation passwords as command-line arguments rather than storing them in .env. From the compose/ directory:

   PGPASSWORD=<superuser password> \
   FMSGD_READER_PGPASSWORD=<reader password> \
   FMSGID_READER_PGPASSWORD=<reader password> \
     docker compose up -d
   

   These variables are only needed during the first startup when the database volume is empty. Passing them on the command line keeps them out of files on disk. PGUSER defaults to postgres if not set.

3. On subsequent starts, only the .env file is needed:

   docker compose up -d
   

4. fmsgd will be available on port 4930 (or the port set by FMSG_PORT in .env).

Integration Tests

End-to-end tests that spin up two full stacks (hairpin.local and example.com) on a shared Docker network and exchange messages between them using fmsg-cli. Test 008 drives fmsg-mcp-claude over stdio and test 014 drives fmsg-mcp over Streamable HTTP (FMSG_MCP_NPM_SPEC picks the version, default @markmnl/fmsg-mcp@latest). The test runner enables fmsg-webapi API-key auth, creates delegated API keys for the test actors during setup, and passes them to fmsg-cli with FMSG_API_KEY.

Prerequisites: Docker, docker compose, Go 1.24+, curl, jq, Node.js 22+ (test 014 runs the published @markmnl/fmsg-mcp with npx).

# Run tests (starts stacks fresh)
./test/run-tests.sh

# Run tests against already-running stacks (skips stack teardown, startup, and seeding)
./test/run-tests.sh --no-start

# Tear down stacks & network
./test/run-tests.sh cleanup

# Refresh local database DD scripts from component branches
FMSGD_REF=main FMSGID_REF=main FMSG_WEBAPI_REF=main ./scripts/update-dd.sh

# CI drift check for database DD scripts
./scripts/update-dd.sh --check

Tests also run on demand via the Integration Test GitHub Actions workflow.

Environment Variables

Configure these in compose/.env. Variables marked required have no default and must be set.

General

Variable	Required	Default	Description
FMSG_DOMAIN	yes		The domain name for your fmsg host
CERTBOT_EMAIL	yes		Email address for Let's Encrypt certificate registration
FMSG_API_TOKEN_ED25519_PRIVATE_KEY	auth		Base64 Ed25519 seed/private key used to mint first-party JWTs from API keys
FMSG_JWT_JWKS_URL	auth		JWKS endpoint for external RS256 user JWT login
FMSG_JWT_ISSUER	JWKS		Expected issuer for external user JWTs
FMSG_JWT_AUDIENCE	JWKS		Expected audience for external user JWTs
FMSG_JWT_ADDRESS_CLAIM	JWKS		Claim containing the fmsg address
FMSG_PORT	no	4930	Host port fmsgd listens on
FMSGID_PORT	no	8080	Internal port for the fmsgid API
GIN_MODE	no	release	Gin framework mode for fmsgid (release or debug)
FMSG_SKIP_DOMAIN_IP_CHECK	no	false	Skip domain-to-IP validation in fmsgd (useful for dev)

At least one auth mode is required for fmsg-webapi: API-key auth with FMSG_API_TOKEN_ED25519_PRIVATE_KEY, external user JWT auth with the JWKS variables, or both. API keys can be created or rotated with the fmsg-webapi operator command and used by fmsg-cli through FMSG_API_KEY.

Database

The PostgreSQL instance hosts two separate databases (fmsgd and fmsgid) with dedicated roles per service.

Variable	Required	Default	Description
PGUSER	no	postgres	PostgreSQL superuser name (used for first-run init only)
PGPASSWORD	init		PostgreSQL superuser password (only needed on first run)
FMSGD_WRITER_PGPASSWORD	yes		Password for fmsgd_writer role (used by fmsgd & webapi)
FMSGD_READER_PGPASSWORD	init		Password for fmsgd_reader role (only needed on first run)
FMSGID_WRITER_PGPASSWORD	yes		Password for fmsgid_writer role (used by fmsgid)
FMSGID_READER_PGPASSWORD	init		Password for fmsgid_reader role (only needed on first run)

Variables marked init are only required on the first startup when the database is being initialised. They can be passed as command-line environment variables (see Getting Started) to avoid storing them on disk.

Database Init Scripts

On first startup (empty data volume), PostgreSQL runs the scripts in docker/postgres/init/ in order:

Script	Purpose
001-init.sh	Creates roles (with passwords from env) and databases
002-fmsgd-dd.sql	Creates tables and other database objects for fmsgd
002-fmsgid-dd.sql	Creates tables and other database objects for fmsgid
003-fmsg-webapi-dd.sql	Creates fmsg-webapi API-key grant tables
999-permissions.sql	Grants permissions after all objects exist

WARNING: To re-run initialisation you must remove the postgres_data volume. This permanently destroys all data in both the fmsgd and fmsgid databases — including user accounts, messages, and any other application state stored in PostgreSQL. Only do this if you intend to start from scratch.

docker compose down
docker volume rm <project>_postgres_data
docker compose up -d   # supply init passwords again

Testing message finalization

The message schema requires a SHA-256 and durable wire representation for sent messages. The initialization SQL is for new databases only. To upgrade an existing stack, stop the daemon and API, back up the database and shared data volume, then run the standalone fmsg-backfill binary with access to the database and stored payload paths. It embeds the schema upgrade; run without -apply for a full dry run and with -apply to commit. Start the matching daemon/API only after migration succeeds. The migration binary is not bundled in the daemon image. Retain the shared message volume, including .fmsg-wire-* directories. The daemon owns the schema; the initialization file here is its bootstrap copy.

Test 015-message-sha256.sh verifies local-only hashing, later federation with compression, batch-hash replies, and notification-only add-to. Test 009 now creates batch replies through the API. To exercise challenge responses on every exchange:

FMSG_CHALLENGE_MODE=ALWAYS ./test/run-tests-podman.sh

For coordinated feature branches, set FMSGD_REF and FMSG_WEBAPI_REF to those branches. FMSG_TEST_NETWORK optionally changes the shared test network name (default fmsg-test).

Pull-request CI selects the matching component branch when it exists, otherwise main. Manual workflow inputs take precedence. CI forces challenge responses.