Sentry-Node

Unified system observability agent — a single Rust binary that replaces and enhances proc-sentry + log-sentry.

                ┌─────────────┐
   /proc ──────►│             │──────► Prometheus /metrics
   /sys  ──────►│ sentry-node │
   journald ───►│             │──────► Loki Push API
   /var/log ───►│             │
                └─────────────┘

Features

Feature	Source	Status
Process CPU/Memory/DiskIO scraping	/proc/[pid]/*	✅
Container detection (Docker/K8s)	/proc/[pid]/cgroup	✅
Namespace-aware port resolution	/proc/[pid]/net/tcp	✅
Prometheus metrics export	/metrics endpoint	✅
Log file tailing (inotify)	configurable paths	✅
Log rotation detection	inode tracking + re-open	✅
Journald reader	journalctl -f -o json	✅
Keyword filter (Aho-Corasick)	CRITICAL/ERROR/FATAL/OOM/etc	✅
Attack detection	SQLi/XSS/Path traversal	✅
Loki push client	batched HTTP	✅
Threshold-based crash events	>90% triggers deep snapshots	✅
Multi-signal correlation	CPU+Log, Disk+Error rules	✅
Process lineage tracking	PPID chain + environ	✅
SIGTERM survival mode	keeps exporting until SIGKILL	✅
eBPF short-lived process capture	aya tracepoints	🔜

Quick Start

# Build
cargo build --release

# Run locally
./target/release/sentry-node

# Or with Docker
docker compose up -d

# Verify
curl http://localhost:9100/metrics
curl http://localhost:9100/health

Configuration

Edit config.toml or use environment variables:

Env Var	Default	Description
METRICS_PORT	9100	HTTP server port
PROC_HOSTNAME	auto	Hostname label
TOP_N	40	Number of top processes to export
PROCFS_PATH	/proc	Path to procfs (use /host/proc in Docker)
ENABLE_DISK_IO	true	Enable disk I/O metrics
ENABLE_PORTS	true	Enable port resolution
LOKI_URL	empty	Loki push URL (empty = disabled)
THRESHOLD	90	Critical threshold percentage

Docker Capabilities

The container requires:

• pid: host — access to host process namespace
• SYS_PTRACE — read /proc/[pid]/io and /proc/[pid]/fd
• DAC_READ_SEARCH — read other users' process info
• NET_ADMIN — read network namespace info
• apparmor=unconfined — bypass AppArmor restrictions

Architecture

src/
├── main.rs                   # Entry: tokio + axum + signal handling
├── config.rs                 # TOML + env var config
├── proc/
│   ├── cpu.rs                # /proc/stat delta CPU
│   ├── memory.rs             # /proc/meminfo
│   ├── process.rs            # /proc/[pid]/* scanner
│   ├── container.rs          # Cgroup → runtime detection
│   ├── disk.rs               # statfs disk usage
│   └── network.rs            # TCP port resolution
├── log/
│   ├── filter.rs             # Aho-Corasick keyword matcher
│   ├── tailer.rs             # inotify log tailing
│   ├── journald.rs           # journalctl JSON reader
│   └── attack.rs             # Security pattern detector
├── export/
│   ├── prometheus.rs         # Prometheus registry
│   └── loki.rs               # Loki Push API client
├── engine/
│   ├── crash_event.rs        # Event lifecycle
│   ├── threshold.rs          # Threshold state machine
│   ├── correlation.rs        # Multi-signal rules
│   └── lineage.rs            # Process parent chain
└── ebpf/                     # Future eBPF support

Metrics

Key exported metrics:

sentry_process_cpu_percent{pid,user,command,runtime,...}
sentry_process_memory_bytes{pid,user,command,runtime,...}
sentry_process_disk_read_bytes{pid,user,command,...}
sentry_process_disk_write_bytes{pid,user,command,...}
sentry_system_cpu_percent
sentry_system_memory_percent
sentry_system_disk_percent
sentry_processes_total
sentry_crash_events_total
sentry_critical_active
sentry_log_error_count{severity,source}
sentry_attack_detected_total{attack_type,severity}
sentry_scrape_duration_seconds

See docs/METRICS.md for the complete reference with PromQL examples.

Documentation

Document	Description
Architecture	System design, module breakdown, data flow diagrams
Metrics Reference	All Prometheus metrics, labels, cardinality, PromQL queries
Configuration	TOML config, env vars, threshold behavior, tuning
Deployment	Docker, Kubernetes DaemonSet, systemd, troubleshooting
Development	Build, test, extend — adding metrics, rules, sources

License

Internal project — Sentry ecosystem.