Skip to content
Astrum-StudioPublic
forked from long2ice/meilisync

About

Realtime sync data from MySQL/PostgreSQL/MongoDB to Meilisearch

Resources

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

 
 

Repository files navigation

meilisync

Realtime sync from PostgreSQL, MySQL, and MongoDB to Meilisearch.

CI Docker Docs PyPI Python License

A maintained fork of long2ice/meilisync by Astrum Agency. Built for production PostgreSQL: wal2json format 2 (no 1 GB string-buffer crash), table-filtered decoding, and automatic reconnect with LSN resume.

Documentation · astrum.agency · Changelog · Docker image


Features

  • CDC, not polling — PostgreSQL logical replication, MySQL binlog, MongoDB change streams
  • Full + incremental — optional backfill when an index is missing, then live updates
  • Batching — flush by document count, interval, or both
  • Index refresh — rebuild via a tmp index and atomic swap
  • Plugins — mutate events before/after they hit Meilisearch
  • Progress — file or Redis, so restarts continue from the last cursor

Install

pip install "meilisync[postgres]"   # PostgreSQL (also the default extra)
pip install "meilisync[mysql]"      # MySQL
pip install "meilisync[mongodb]"    # MongoDB
pip install "meilisync[all]"        # everything

Or Docker:

docker pull ghcr.io/astrum-studio/meilisync:latest

Quick start

# config.yml
progress:
  type: file
  path: progress.json
source:
  type: postgres
  host: 127.0.0.1
  port: 5432
  user: postgres
  password: postgres
  database: app
meilisearch:
  api_url: http://127.0.0.1:7700
  api_key: MASTER_KEY
  insert_size: 1000
  insert_interval: 5
sync:
  - table: products
    index: products
    pk: id
    full: true
meilisync start
# docker-compose.yml
services:
  meilisync:
    image: ghcr.io/astrum-studio/meilisync:latest
    restart: unless-stopped
    volumes:
      - ./config.yml:/app/config.yml:ro
      - ./data:/app/data

Full configuration, source setup, and CLI: docs.

Commands

Command Purpose
meilisync start Full sync (if needed) + incremental CDC
meilisync refresh -t TABLE Rebuild an index via swap
meilisync check -t TABLE Compare source count vs Meilisearch
meilisync version Print version

Development

This repo uses uv for environments/lockfile and hatchling as the build backend.

uv sync --all-extras --group dev --group docs
make ci          # lint, typecheck, unit tests
make docs-serve  # MkDocs Material
uv build         # sdist + wheel

Releases are Git tags (vX.Y.Z): PyPI via trusted publishing, GHCR image, GitHub Release. See Publishing.

License

Apache-2.0. See NOTICE for attribution to the original project.

About

Realtime sync data from MySQL/PostgreSQL/MongoDB to Meilisearch

Resources

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages