Skip to content
daemonlessPublic

About

Lightweight, geo-distributed S3-compatible object storage service designed for self-hosting on commodity hardware.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Garage

Build Status Last Commit OCI Pulls

Lightweight, geo-distributed S3-compatible object storage service designed for self-hosting on commodity hardware.

Port 3900
Registry ghcr.io/daemonless/garage
Source https://git.deuxfleurs.fr/Deuxfleurs/garage
Website https://garagehq.deuxfleurs.fr/

Version Tags

Tag Description Best For
latest / pkg FreeBSD Quarterly. Uses stable, tested packages. Most users — recommended.
pkg-latest FreeBSD Latest. Rolling package updates. Staying current.

Prerequisites

Before deploying, ensure your host environment is ready. See the Quick Start Guide for host setup instructions.

Deployment

Podman Compose

services:
  garage:
    image: "ghcr.io/daemonless/garage:latest"
    container_name: garage
    environment:
      - PUID=1000  # User ID for the application process
      - PGID=1000  # Group ID for the application process
      - TZ=UTC  # Timezone for the container
      - RPC_SECRET=${RPC_SECRET}  # 32-byte hex RPC secret for cluster authentication (auto-generated on first run if unset)
      - ADMIN_TOKEN=${ADMIN_TOKEN}  # Admin API and metrics authorization bearer token (auto-generated on first run if unset)
      - RPC_PUBLIC_ADDR=${RPC_PUBLIC_ADDR:-127.0.0.1:3901}  # Address other cluster nodes reach this one on. The default suits a single node; clustering needs a routable address.
      - GARAGE_AUTO_LAYOUT=${GARAGE_AUTO_LAYOUT:-true}  # Assign a single-node layout on first boot so the node stores data without manual setup. Set false when joining an existing cluster.
      - GARAGE_ZONE=${GARAGE_ZONE:-dc1}  # Zone name used by the automatic single-node layout
      - GARAGE_CAPACITY=${GARAGE_CAPACITY:-10G}  # Storage this node advertises to the cluster, e.g. 10G or 2T
      # - GARAGE_ARGS=  # Additional command-line arguments passed to garage server
    volumes:
      - "/containers/garage:/config"
      - "/containers/garage/data:/data"
    ports:
      - "3900:3900"
      - "3901:3901"
      - "3902:3902"
      - "3903:3903"
    # always (not unless-stopped) so FreeBSD's podman rc.d auto-starts it at boot
    restart: always

Save as compose.yaml, then run podman-compose up -d.

AppJail Director

.env:

# .env

DIRECTOR_PROJECT=garage
PUID=1000
PGID=1000
TZ=UTC
RPC_SECRET=${RPC_SECRET}
ADMIN_TOKEN=${ADMIN_TOKEN}
RPC_PUBLIC_ADDR=${RPC_PUBLIC_ADDR:-127.0.0.1:3901}
GARAGE_AUTO_LAYOUT=${GARAGE_AUTO_LAYOUT:-true}
GARAGE_ZONE=${GARAGE_ZONE:-dc1}
GARAGE_CAPACITY=${GARAGE_CAPACITY:-10G}

appjail-director.yml:

# appjail-director.yml

options:
  - virtualnet: ':<random> default'
  - nat:
services:
  garage:
    name: garage
    options:
      - container: 'args:--pull'
      - expose: '3900:3900 proto:tcp'
      - expose: '3901:3901 proto:tcp'
      - expose: '3902:3902 proto:tcp'
      - expose: '3903:3903 proto:tcp'
    oci:
      user: root
      environment:
        - PUID: !ENV '${PUID}'
        - PGID: !ENV '${PGID}'
        - TZ: !ENV '${TZ}'
        - RPC_SECRET: !ENV '${RPC_SECRET}'
        - ADMIN_TOKEN: !ENV '${ADMIN_TOKEN}'
        - RPC_PUBLIC_ADDR: !ENV '${RPC_PUBLIC_ADDR}'
        - GARAGE_AUTO_LAYOUT: !ENV '${GARAGE_AUTO_LAYOUT}'
        - GARAGE_ZONE: !ENV '${GARAGE_ZONE}'
        - GARAGE_CAPACITY: !ENV '${GARAGE_CAPACITY}'
    volumes:
      - garage: /config
      - garage_data: /data
volumes:
  garage:
    device: '/containers/garage'
  garage_data:
    device: '/containers/garage/data'

Makejail:

# Makejail

ARG tag=latest

OPTION container=boot
OPTION overwrite=force
OPTION from=ghcr.io/daemonless/garage:${tag}

Save the files above, then run appjail-director up.

Warning

Exposing ports in AppJail means that your service can be reached from remote hosts. If that is not your intention, do not expose the ports and communicate with the service using the jail's IPv4 address or hostname assigned by the virtual network.

To avoid exposing ports, just remove the expose option in your appjail-director.yml or from your command-line arguments.

Bastille

Warning

Bastille's OCI support is experimental. It requires buildah and shares the host network stack (inherit). Mount volumes with --volume HOST JAIL; without it, image-declared volumes are stored under ${bastille_volumesdir}/${jail}.

services:
  garage:
    name: garage
    image: "ghcr.io/daemonless/garage:latest"
    network:
      - mode: host
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=UTC
      - RPC_SECRET=${RPC_SECRET}
      - ADMIN_TOKEN=${ADMIN_TOKEN}
      - RPC_PUBLIC_ADDR=${RPC_PUBLIC_ADDR:-127.0.0.1:3901}
      - GARAGE_AUTO_LAYOUT=${GARAGE_AUTO_LAYOUT:-true}
      - GARAGE_ZONE=${GARAGE_ZONE:-dc1}
      - GARAGE_CAPACITY=${GARAGE_CAPACITY:-10G}
    volumes:
      - "/containers/garage:/config"
      - "/containers/garage/data:/data"

Save as bastille-compose.yml, then run bastille up.

Parameters

Environment Variables

Variable Default Description
PUID 1000 User ID for the application process
PGID 1000 Group ID for the application process
TZ UTC Timezone for the container
RPC_SECRET ${RPC_SECRET} 32-byte hex RPC secret for cluster authentication (auto-generated on first run if unset)
ADMIN_TOKEN ${ADMIN_TOKEN} Admin API and metrics authorization bearer token (auto-generated on first run if unset)
RPC_PUBLIC_ADDR ${RPC_PUBLIC_ADDR:-127.0.0.1:3901} Address other cluster nodes reach this one on. The default suits a single node; clustering needs a routable address.
GARAGE_AUTO_LAYOUT ${GARAGE_AUTO_LAYOUT:-true} Assign a single-node layout on first boot so the node stores data without manual setup. Set false when joining an existing cluster.
GARAGE_ZONE ${GARAGE_ZONE:-dc1} Zone name used by the automatic single-node layout
GARAGE_CAPACITY ${GARAGE_CAPACITY:-10G} Storage this node advertises to the cluster, e.g. 10G or 2T
GARAGE_ARGS `` Additional command-line arguments passed to garage server

Volumes

Path Description
/config Configuration directory containing garage.toml
/data Object storage: metadata in /data/meta, data blocks in /data/data

Ports

Port Protocol Description
3900 TCP S3 API endpoint
3901 TCP RPC endpoint for inter-cluster node communication
3902 TCP S3 web endpoint (static website hosting)
3903 TCP Admin API and Prometheus metrics endpoint

There is no web UI

Port 3900 is the S3 API and only answers SigV4-signed requests. Opening it in a browser always returns:

<Error><Code>AccessDenied</Code>
<Message>Forbidden: Garage does not support anonymous access yet</Message></Error>

That is Garage working, not a fault. Use an S3 client, the admin API on 3903, or the garage CLI.

Modern AWS clients: turn off flexible checksums

botocore >= 1.36 and aws-cli >= 2.23 send CRC32 "flexible checksums" by default, and Garage's multipart checksum does not match what they compute. A large download then aborts with FlexibleChecksumError even though the object is intact (verified: md5 of a 12 MB object matches end to end).

export AWS_RESPONSE_CHECKSUM_VALIDATION=when_required
export AWS_REQUEST_CHECKSUM_CALCULATION=when_required

Small objects are unaffected; this only bites on multipart transfers.

First run: assign a layout

A fresh node starts with NO ROLE ASSIGNED and stores nothing until a layout is applied. This container does that for you on first boot (GARAGE_AUTO_LAYOUT=true, zone GARAGE_ZONE, size GARAGE_CAPACITY).

Set GARAGE_AUTO_LAYOUT=false when joining an existing cluster, and assign the layout across nodes yourself:

podman exec garage garage status          # note the node ID
podman exec garage garage layout assign -z dc1 -c 10G <node-id>
podman exec garage garage layout apply --version 1

-c is the capacity this node advertises; -z its zone.

Create a bucket and key

podman exec garage garage bucket create mybucket
podman exec garage garage key create mykey          # prints the Key ID + secret
podman exec garage garage bucket allow --read --write --owner mybucket --key mykey

Then point any S3 client at it, with region garage:

aws --endpoint-url http://your-host:3900 s3 ls s3://mybucket

Admin API

The admin token is generated into /config/garage.toml on first run (or set ADMIN_TOKEN):

curl -H "Authorization: Bearer $ADMIN_TOKEN" \
     http://your-host:3903/v2/GetClusterStatus

Static website hosting

Served on 3902, not 3900, and matched by Host header:

podman exec garage garage bucket website --allow mybucket
curl -H "Host: mybucket.web.garage" http://your-host:3902/

FreeBSD notes

The generated config binds 0.0.0.0 rather than [::]. FreeBSD defaults net.inet6.ip6.v6only=1, so an IPv6 wildcard socket refuses IPv4 and the service would be unreachable on 127.0.0.1 -- taking the healthcheck and the s6 readiness probe down with it.

rpc_public_addr is pinned to 127.0.0.1:3901, which suits a single node. Clustering needs it set to an address the other nodes can reach.

Architectures: amd64, aarch64 User: bsd (UID/GID via PUID/PGID, defaults to 1000:1000) Base: FreeBSD 15.1


Need help? Join our Discord community.

About

Lightweight, geo-distributed S3-compatible object storage service designed for self-hosting on commodity hardware.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages