DevOps Notes

Comprehensive reference for DevOps, DevSecOps, SRE, and Senior Backend Engineering.

Structure

docs/
├── devops/
│   ├── cicd.md                   # CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins, Tekton)
│   ├── gitops.md                 # GitOps with ArgoCD, FluxCD, Kustomize
│   ├── release-strategies.md      # Blue-green, canary, rolling, feature flags
│   └── chaos-engineering.md       # GameDays, LitmusChaos, fault injection, AWS FIS
├── platform/
│   ├── docker.md                  # Multi-stage builds, security, layer caching
│   ├── kubernetes.md              # Pod patterns, HPA/VPA/KEDA, service meshes, Helm
│   ├── internal-developer-platforms.md  # Backstage, golden paths, self-service
│   ├── runtime-security.md       # Falco, Tetragon, container escape detection
│   └── kubernetes-cluster-lifecycle.md # EKS/GKE provisioning, upgrades, Karpenter
├── iac/
│   ├── terraform.md              # State management, modules, workspaces, providers
│   ├── ansible.md                # Playbooks, roles, inventory, Vault
│   └── finops.md                 # Cloud cost optimization, rightsizing, Kubecost
├── observability/
│   ├── logging.md                # Structured logging, ELK Stack, Loki, Promtail
│   ├── monitoring.md             # Prometheus, SLO/SLI/SLA, alerting, Grafana
│   ├── tracing.md                # Distributed tracing, OpenTelemetry, Jaeger, Tempo
│   └── continuous-profiling.md    # Pyroscope, Parca, eBPF profiling, flame graphs
├── devsecops/
│   ├── security-pipeline.md       # SAST, DAST, SCA, container scanning, secrets detection
│   └── supply-chain.md           # SLSA, Sigstore/Cosign, SBOM, IAM, compliance
├── sre/
│   ├── on-call.md                # SLOs, error budgets, on-call, runbooks, postmortems, TOIL
│   ├── incident-response.md       # IR lifecycle, triage, mitigation, stakeholder comms
│   ├── dora-metrics.md          # Deploy frequency, lead time, MTTR, change failure rate
│   ├── postmortems.md          # Real-world incidents: GitHub, Facebook, AWS, Knight Capital, Google
│   └── edge-cases.md           # DB locks, OOMKilled, DNS, throttling, split-brain, retry storms
├── backend/
│   ├── api-design.md            # REST, GraphQL, gRPC, versioning, idempotency, rate limiting
│   ├── patterns.md              # Circuit breakers, CQRS, event sourcing, saga, retry
│   ├── distributed-systems.md     # Raft, Paxos, CRDTs, split-brain, HLC, multi-region
│   ├── optimizations.md          # Indexing, query plans, N+1, caching, sharding, profiling
│   └── database-migrations.md   # Zero-downtime migrations, expand-contract, Flyway
└── system-design/
    ├── fundamentals.md           # Caching, CDNs, load balancing, DNS, consistent hashing
    ├── api-gateway.md          # Kong, rate limiting, JWT auth, circuit breaking
    ├── databases.md             # SQL vs NoSQL vs NewSQL, replication, sharding
    ├── messaging.md             # Kafka, RabbitMQ, SQS, pub/sub, exactly-once
    ├── authentication.md        # JWT, OAuth2, mTLS, RBAC, secrets management
    ├── networking.md            # TCP/UDP, HTTP/2, gRPC, DNS, TLS, WebSockets
    ├── search.md                # Elasticsearch, PostgreSQL FTS, indexing strategies
    ├── storage.md               # S3, block vs object storage, CDN patterns
    ├── async-processing.md      # Job queues, cron, batch processing, idempotency
    ├── real-time.md             # WebSockets, SSE, GraphQL subscriptions, push notifications
    ├── file-uploads.md         # Presigned URLs, multipart, virus scanning, CDN
    ├── vector-databases.md     # pgvector, Pinecone, RAG, embeddings, semantic search
    ├── multi-tenancy.md         # Namespace-per-tenant, pooled DB, RBAC, tenant isolation
    └── microservices.md         # Saga, CQRS, DDD, API composition, service mesh, backpressure

Quick Reference

CI/CD Tool Selection

Tool	Hosted?	Best For
GitHub Actions	Yes (GitHub)	GitHub-native projects
GitLab CI	Yes (GitLab)	GitLab-native, integrated security
Jenkins	Self-hosted	Maximum control, legacy systems
Tekton	Self-hosted (K8s)	Kubernetes-native pipelines

Kubernetes Service Mesh

Mesh	Overhead	Best For
Istio	~10-15%	Fine-grained traffic control, mTLS
Linkerd	~1-2%	Teams wanting mesh without Istio complexity

IaC Tool Selection

Tool	Language	Best For
Terraform	HCL	Multi-cloud, declarative, massive provider ecosystem
Ansible	YAML	Configuration management, orchestration
Pulumi	TypeScript/Python/Go	Real programming language features

Observability Stack

Component	Tool
Metrics	Prometheus + Grafana
Logs	Loki + Promtail (or ELK Stack)
Traces	Jaeger / Tempo / Zipkin
APM	Datadog / New Relic / Elastic APM
Alerting	Alertmanager, PagerDuty, OpsGenie

Security Tooling

Category	Tool
SAST	Semgrep (default), SonarQube (enterprise), Bandit (Python only)
DAST	OWASP ZAP (free), Burp Suite (manual), Escape (GraphQL)
SCA	Snyk, Dependabot (GitHub-native)
Container Scan	Trivy (default), Clair (Red Hat ecosystem)
Secrets	Gitleaks (pre-commit), Semgrep secrets
Policy-as-Code	OPA/Conftest, Kyverno (K8s-native)
Supply Chain	SLSA, Sigstore/Cosign, Syft (SBOM)

Message Queue Selection

Queue	Best For	Throughput	Exactly-Once
Kafka	Event streaming, CDC	Very High	Yes (transactions)
RabbitMQ	Task queues, RPC	Medium	With manual ack
SQS	AWS integration, serverless	High	With DLQ
Redis Streams	Low latency, simple queue	Medium	With XACK

Database Selection

Need	Database
ACID transactions	PostgreSQL, CockroachDB
Scale horizontally	Cassandra, DynamoDB, CockroachDB
Flexible schema	MongoDB, DynamoDB
Time-series data	InfluxDB, TimescaleDB
Full-text search	Elasticsearch, PostgreSQL FTS

Storage Selection

Type	Best For
S3/Object	Files, media, backups
Block (EBS)	Databases, boot volumes
EFS/NFS	Shared file access
GlusterFS/CephFS	Scale-out NAS

Database Index Selection

Query Type	Index Type
Equality + range (=, <, >)	B-tree (default)
Arrays, JSONB containment	GIN
Geospatial, nearest-neighbor	GiST
Append-only, sequential data	BRIN
Subset of frequently queried rows	Partial

Contributing

Add .md files to the appropriate directory. Keep examples concrete and runnable.