Release v2.1.x: Penguin-DAL migration, centralized libs, and test suite overhaul - #46
Merged
Conversation
Version files added (all at v1.0.0):
- dns-client-go/.version - Go DNS client
- dns-server/.version - DNS server
- dns-client/.version - Python DNS client (docker-client)
- dns-server/flask_app/.version - Web console (webui)
Workflow updates:
- Go client workflow now uses dns-client-go/.version
- Build triggers only when component's .version changes
- Added Windows ARM64 to build targets (6 platforms total)
- Release tags now use format: v{VERSION}-go-client
Release notes reorganized with sections for each package:
- go-client, dns-server, docker-client, webui
- Each with independent versioning and changelogs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add .version file monitoring to all build workflows - Implement epoch64 timestamp-based naming (alpha/beta-<epoch64>) - Add version-based release naming (vX.X.X-alpha/beta) - Add auto pre-release creation on .version changes - Add security scanning (gosec for Go, bandit for Python, npm audit for Node.js) - Create comprehensive docs/WORKFLOWS.md - Update docs/STANDARDS.md with CI/CD section - Update CLAUDE.md with CI/CD section and pre-commit checklist 🤖 Generated with Claude Code Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Backend: - Add DHCP models (pools, leases, reservations, servers) with PyDAL - Add Time models (servers, sync logs, clients, config) with PTP/NTP support - Add full REST API for DHCP pool management and lease tracking - Add full REST API for time server management and sync operations - Fix RBAC middleware: add check_team_access function - Fix team model: remove unsupported PyDAL unique constraint Frontend: - Add TypeScript types for DHCP and Time services - Add DHCPPools management component with utilization visualization - Add TimeServers management component with PTP/NTP protocol selection - Integrate new tabs into Management page Go Client: - Add NTP client package with automatic server failover - Add NTP forwarder to intercept OS time requests (port 123) - Add time configuration with environment variable support - Add CLI commands: time query, time forward, time status - Add comprehensive unit and integration tests Documentation: - Update API.md with DHCP and Time API endpoints - Update ARCHITECTURE.md with network services diagrams - Update STANDARDS.md with service-specific guidelines 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Restructure from server-side templates to a three-container architecture: - Flask backend serves JSON-only REST API with JWT + Flask-Login auth - React WebUI (Vite + TypeScript + Tailwind) as separate nginx container - DNS server unchanged Flask changes: add Flask-JWT-Extended, Flask-CORS, Flask-Limiter; convert all dashboard routes to /api/v1/ JSON endpoints; add safe PyDAL row serialization (exact type checks to exclude RecordUpdater/LazySet); enforce auth on all API endpoints. Docker: add Dockerfile.api for Flask, dns-webui service with nginx reverse proxy, update docker-compose.yml with new services. Tests: add comprehensive smoke test suite (130 tests) covering container health, API endpoints, JWT auth flows, pagination, and response formats. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes Dependabot alerts #33 and #34: - CVE: Next.js HTTP request deserialization DoS (high) - CVE: Next.js Image Optimizer DoS (medium) Also fixes js-yaml prototype pollution via npm audit fix. Remaining 2 moderate alerts (eslint stack overflow, Next.js PPR memory) require breaking major version upgrades and have no stable patches. quic-go alerts (#28-30) are already resolved on this branch (v0.59.0 includes all patches). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The root Dockerfile's dns-server and dns-client stages both COPY docs/ into the image. The .dockerignore was excluding the docs directory, which would cause the CI docker-multi-build job to fail. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit addresses all identified security vulnerabilities across Python, Node.js, and Go components, and adds extensive smoke test coverage for both alpha (local) and beta (K8s) environments to ensure production readiness. Security Fixes: - Python: Update cryptography (41→44), PyMySQL (1.1.0→1.1.1), Pillow (10→11), aiohttp (3.9→3.11.11), add defusedxml for XXE prevention - Node.js: Update react-router-dom (6→7), eslint (8→9), vite (5→7), next (15→16) to fix XSS, DoS, and stack overflow vulnerabilities - Go: Fix integer overflow in NTP time handling, replace math/rand with crypto/rand, enforce TLS 1.2+ minimum, update quic-go to v0.59.0 New Features: - Add version management script (scripts/version/update-version.sh) supporting major/minor/patch/build version increments - Add 150 mock tests for isolated component testing (DNS client/server, Manager API, database operations) - Add 79 comprehensive smoke tests covering security, edge cases, integration, and user experience for alpha and beta environments Test Coverage: - Security: 32 tests (auth, authorization, input validation, injection prevention, CORS, TLS, data isolation) - Edge Cases: 37 tests (boundaries, special chars, concurrency, large datasets, error recovery) - Integration: 39 tests (component communication, data flow, K8s integration, monitoring) - User Experience: 24 tests (page loads, forms, navigation, error messages) - Mock Tests: 150 tests (fast, isolated, no network/database dependencies) Version: Bump to v2.1.1.1770072428 (patch increment for security fixes) Resolves: Critical PyMySQL SQL injection, XSS in react-router, integer overflow in Go NTP client, weak RNG in HTTP backoff, TLS misconfiguration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Install shared/react_libs dependencies before webui build to resolve zod module import error during vite build. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update dal2-beta cluster deployment to use the pre-existing penguintech-wildcard-tls certificate from cert-manager instead of generating a new certificate. Changes: - Create comprehensive K8s manifests in k8s/manifests/ - Configure ingress for squawk.penguintech.io with proper TLS - Use pre-existing Cloudflare wildcard certificate (penguintech-wildcard-tls) - Deploy DNS server, Flask API, WebUI, and Valkey services - Add health checks, resource limits, and HA configuration (2 replicas) - Configure proper routing: / → webui, /api/v1 → flask-api, /dns-query → dns-server Ingress Configuration: - Host: squawk.penguintech.io - TLS Secret: penguintech-wildcard-tls (Cloudflare wildcard cert) - SSL redirect enabled - NGINX ingress class Deployed to: dal2-beta cluster, squawk namespace Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove restrictive security contexts and add emptyDir volumes for directories that need write access (database logs, nginx cache). Fixes: - flask-api: Health check path /api/v1/health → /health - flask-api: Add volume mount for /app/flask_app/databases - webui: Add volume mounts for nginx cache and run directories - All: Remove runAsUser/runAsNonRoot to allow containers to start properly Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change @penguin/react_libs reference from local file path to GitHub repository to ensure consistent versioning across all PenguinTech projects. - Update package.json to reference github:penguintechinc/penguin-libs#main - Install @penguintechinc/react-libs from main branch - Remove dependency on local shared/react_libs copy Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace local copies of shared libraries with git-referenced dependencies from github.com/penguintechinc/penguin-libs. This ensures consistent versioning across all PenguinTech projects. - Remove entire shared/ directory (react_libs, node_libs, go_libs, py_libs, licensing, database) including duplicate shared/shared/ nested copy - Update Dockerfile to remove COPY shared/react_libs build steps - Update tailwind.config.js content path to node_modules dist - Update all imports from @penguin/react_libs to @penguintechinc/react-libs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…olation) The new test relied on dns-server/tests/conftest.py importing the manager's schema.py to populate the shared token table; in CI's job layout that import path isn't reachable, conftest silently falls back to empty metadata, and the test hit TableNotFoundError: 'token'. The test now creates the minimal post-migration token table (token_hash, no plaintext token) directly against db_engine with checkfirst=True and only cleans rows on teardown. Production code untouched. Verified passing twice in an isolated copy with no manager/ sibling (the exact CI condition). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(manager): close broken-access-control vulns (zones, client-config, users, DHCP/Time, SCIM, DPoP)
After merging v2.1.x (access-control PR #91) into this branch, the access-control test fixture still seeded deployment_domain.jwt_token, which this PR renamed to jwt_token_hash. Updated the fixture to insert the hashed column. Full manager suite: 441 passed (1 pre-existing env-only observability failure). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(manager,dns-server): hash DNS tokens/join-keys at rest, encrypt jwt_secret
Pre-existing functional break, confirmed across penguin-dal 0.1.0-0.4.1 and by reading penguin_dal/query.py: Row has no update_record() and TableProxy has no __delitem__, so every PUT route (`row.update_record(**f)`) and every DELETE route (`del db.<table>[id]`) 500'd at runtime. It shipped because these routes had zero test coverage. Replaced all 28 call sites across 12 files with the supported QuerySet API: - `row.update_record(**f)` -> `db(db.<table>.id == row.id).update(**f)` (18 sites) - `del db.<table>[id]` -> `db(db.<table>.id == id).delete()` (9 sites) - config_service heartbeat + whois_cache update (2 service-internal sites) Files: ioc_feeds, time, dhcp, auth, sso_admin, teams, zones, users, dns_servers, tokens blueprints + config_service, whois_service. Tests: new test_dal_update_delete_regression.py (12 tests) exercises each PUT/ DELETE route end-to-end and re-fetches to assert the change persisted (these fail against the old code, pass now). Also corrected 4 access-control tests that had encoded the bug via pytest.raises(AttributeError/TypeError) to assert real success. Manager suite: 453 passed (1 pre-existing env-only observability fail). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(manager): repair every PUT/DELETE route (penguin-dal API misuse)
…t-time, key perms) MED/LOW hardening from the security audit: - init_db no longer prints the admin password to stdout (persisted in container/CI logs); prints a masked note instead. - MFA recovery codes: entropy raised from 32 bits (token_hex(4)) to 64 bits (token_hex(8)); bcrypt storage/verify unchanged. - ProductionConfig now enforces a minimum length (>=32) on the secret that keys the MFA/SSO Fernet encryption, failing startup if too weak (dev/test untouched). - whois + ioc_feeds error paths no longer return str(e) to clients — detail logged server-side, generic message returned (via responses.internal_error). - Browser-binding SHA-256 comparisons in sso_service/saml_service switched from != to hmac.compare_digest. - dns-server cert_manager writes key files atomically at mode 0600 (os.open O_CREAT|O_WRONLY) instead of write-then-chmod, closing the readable race window. - Removed a pre-existing unused import (datetime.timedelta) in sso_service.py. Tests: 6 new suites (password hygiene, recovery entropy, key-strength, const-time binding, error-response hygiene, cert key perms). manager 468 passed, dns-server 155 passed (1 pre-existing env-only observability failure). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(security): crypto & secret hygiene (password print, entropy, const-time, key perms)
A security product must record security-relevant events. The audit subsystem existed (@audit_log on ~60 routes) but the auth lifecycle was uninstrumented. Extended coverage using the existing decorator to: - token refresh, machine/NHI token grants (client_credentials + token-exchange, incl. failed client auth), MFA verify (success+failure), SSO/OIDC login, SAML login/ACS, SCIM user CRUD + token mint/revoke, deployment-domain JWT rollover. - Fixed the actor-attribution gap: user_login rows were written with actor_id=NULL (get_current_user() is None during login), so brute force was untraceable to the targeted account. login now records the TARGET account id + source IP for success and failure. Mechanism: audit_log now honors optional g.audit_actor_id/tenant/resource_type/ resource_id set by the view for pre-auth events, falling back to the existing JWT/kwarg inference; added a defensive per-invocation reset of those g attrs. Tests: test_audit_logging_coverage.py (8 tests) — refresh, failed/successful/ unknown-username login attribution, MFA-verify failure, SSO login, SCIM create, domain-JWT rollover. flake8 clean; 461 passed (1 pre-existing env-only failure). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(manager): close audit-logging gaps on auth-lifecycle events
Audit found the CI security gates masked (can't fail) and no real CodeQL despite
the org ruleset requiring code-scanning.
- Add .github/workflows/codeql.yml: real CodeQL analysis (python, go,
javascript-typescript), SHA-pinned action, on push/PR to main+v2.1.x+release/**
plus weekly cron. Provides the code-scanning results the org ruleset requires
and adds the previously-absent JS/TS SAST coverage.
- Unmask bandit (build.yml, version-monitor.yml): JSON-report run kept, followed
by a real gating run on the same inputs (no || true, continue-on-error: false),
scoped to app source.
- Unmask gosec: replaced the -no-fail masked action with a gating
`gosec ./...` on squawk-client-go; also fixed 4 ineffective //nolint:gosec
suppressions to gosec-native `// #nosec` syntax (they were silently unsuppressed).
- Resolved the real findings the now-gating scanners surfaced:
* ntp-server binds: `# nosec B104` justified (network daemon must listen broadly)
* dhcp.py/schema.py listen_address DB defaults: `# nosec B104` (config data, not
a socket bind)
* squawk-client k8s-client.py: added missing requests timeout (B113, real bug)
- Coverage gate (--cov-fail-under=98 || true) left non-gating with a rationale
comment — actual coverage is below threshold; raising it is a separate effort.
Verified: bandit gating command exits 0, gosec exits 0, go build passes, all
workflow YAML valid, actions SHA-pinned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…deql chore(ci): add CodeQL analysis + unmask bandit/gosec security gates
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Release is gated on 90% coverage; the dns-server coverage check was masked (--cov-fail-under=98 || true) and hid that real coverage was ~45%. It was also missing a test-only dependency (`responses`), so two suites failed collection and the gate would have errored even if unmasked. - Added 14 unit-test suites covering the previously-thin modules to ~99-100%: cert_manager, selective_dns_routing, selective_router, manager_client, resilience, config, main, http3_serving, dns_resolver, cache_manager, prometheus_metrics, metrics_reporter, observability, grpc_server. Real behavior + edge/error paths, mocked externals. - Fixed a real DoS bug found while testing: prometheus_metrics.py imported `prometheus_client.Counter` over `collections.Counter` (name shadow), so the top_domains cap trim always raised (silently swallowed) and the dict was never bounded. Aliased to CollectionsCounter; the cap now actually enforces. - CI (build.yml, server-release.yml): install requirements-dev.txt in the coverage step (provides `responses`), remove the `|| true` mask, and set --cov-fail-under=90 so the gate is real. Local lower bound (excluding the 2 responses-dependent suites): 99% (1951 stmts, 25 missing). flake8 clean on all files; workflows valid. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… hardening) dns-webui kept access + refresh JWTs in localStorage, so any JavaScript (app or a compromised dependency) could exfiltrate them. Moved to HttpOnly, Secure, SameSite=Strict cookies that JS cannot read. Change is ADDITIVE — bearer-token clients (manager/frontend, Go client) are unaffected. Backend (manager): - app/services/cookie_auth.py (new): set/clear HttpOnly+Secure+SameSite=Strict access/refresh cookies + a JS-readable csrf_token cookie; double-submit CSRF check (secrets.compare_digest). - middleware/auth.py: token_required accepts the cookie as a fallback to the Authorization header; enforces CSRF on cookie-sourced mutating requests only (header clients exempt). - blueprints/auth.py (login/refresh/logout), mfa.py (mfa_verify): set cookies alongside the existing JSON tokens; logout clears cookies + revokes the cookie-sourced refresh token. - config.py: COOKIE_SECURE/COOKIE_DOMAIN (dev/test default to non-Secure). Frontend (dns-webui): - Removed ALL localStorage token read/write; api uses withCredentials:true and auto-attaches the CSRF header on mutating requests; 401 -> cookie-based refresh. - useAuth tracks auth state via /auth/me, never holding the JWT in JS. Tests: 16 backend cookie/CSRF tests + updated frontend Login test. Backend suite 492 passed (1 pre-existing env-only observability failure); dns-webui vitest 9 passed. flake8 clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two CI failures on the coverage gate: - test_manager_client_coverage.py uses the `requests_mock` fixture from the requests-mock package, which was installed locally but not declared — CI errored with "fixture 'requests_mock' not found". Added requests-mock to dns-server/requirements-dev.txt (installed by the coverage step). - CodeQL raised 2 high false positives (py/incomplete-url-substring-sanitization) on membership checks in the new test files (`"host" in dict`, `"name" in list` — not URL sanitization). Added .github/codeql/codeql-config.yml with paths-ignore for test/vendored code and wired it into codeql.yml init; production code is still scanned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…kies fix(dns-webui): store JWTs in HttpOnly cookies, not localStorage (XSS hardening)
CI (stricter env) surfaced 3 test failures on the coverage run: - cert_manager.create_server_cert used the host FQDN as the X.509 CommonName with no length cap; a >64-char hostname (e.g. the CI runner's) raised ValueError during cert generation -- a real bug for long-hostname hosts. CN is now truncated to 64 chars; the full hostname still goes in the SAN. - prometheus_metrics system-metrics tests require psutil (an optional guarded import in the module); added psutil to requirements-dev.txt so the psutil path runs under test. Coverage gate itself already passed (97.86% >= 90%). All 134 cert/prometheus tests pass locally; flake8 clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 40 selective-routing coverage tests passed in the pip test env but failed in CI's Docker-image test run (`docker run ... pytest tests/`) with TableNotFoundError for dns_group / dns_routing_zone / user_group_assignment / group_zone_access. Like the token-hash test before it, they relied on conftest importing the manager schema to create those tables, which silently no-ops when manager/ isn't checked out alongside (the Docker case). Added an autouse fixture that creates the four tables directly against db_engine (checkfirst=True, row-only teardown), mirroring test_selective_dns_routing_ token_hash.py. Also fixed one test that used a token but hadn't declared the token_table fixture. Verified against a fresh copy with no manager/ sibling (60 passed, twice); coverage still 100%. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
test(dns-server): raise coverage to ~99% and enforce the 90% gate
…ns didn't exist)
saml.py acs() issued tokens via AuthService.create_tokens(user_id=, email=,
global_role=, sso=), a method that does not exist — so every successful SAML
login raised AttributeError and 500'd. It was invisible because no test reached
the success path (real signature verification needs xmlsec1). Replaced with the
same two-call pattern the working OIDC SSO callback uses:
create_access_token(id, username, global_role, {}) + create_refresh_token(id).
Added a regression test (test_saml_acs_success_issues_tokens) that patches the
assertion-validation + JIT-provisioning seams and asserts acs() returns 200 with
access/refresh tokens (it 500'd before this fix). Full SAML suite: 27 passed;
flake8 clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(manager): repair SAML login success path (create_tokens didn't exist)
…ning docs: reflect v2.1.x security hardening in README + docs
Enabling CodeQL/scanning surfaced pre-existing dependency debt. Bumped all critical + high advisories to their first-patched versions and verified builds: - Go (squawk-client-go): google.golang.org/grpc 1.82.1->1.83.1; golang.org/x/ crypto pinned 0.52.0 (CRITICAL floor); quic-go already >=0.54.1. build/vet/test green. - Python (manager/backend): cryptography ->50.0.1 (>=49 floor), PyJWT ->2.13.0. 493 tests pass. - npm manager/frontend: react-router-dom 7.18.1->7.18.3; override bumps for brace-expansion/js-yaml/postcss; audit fix cascaded nanoid/browserslist/vite/ minimatch. 9 high -> 0; vite build + 23 tests pass. - npm website + root: 0 high/critical. Root puppeteer 24->25 (required — extract-zip has no patched release; 25.x drops it); overrides for js-yaml/ip-address/ brace-expansion/nanoid/postcss. website next build passes (9/9 pages). Notes: shared/go_libs/go.mod and shared/node_libs do not exist on this branch (migrated to penguin-libs) — those alerts are moot. requirements.txt hash regeneration + penguin-* version drift deferred to a separate pass (out of CVE scope). The 10 CodeQL code-scanning alerts were verified false positives and dismissed separately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(deps): remediate critical+high dependency CVEs (Go/Python/npm)
…in logs)
CodeQL py/clear-text-logging-sensitive-data: posthog_client logged
`api_key={self.api_key[:8]}...` at init. Even a prefix is partial secret
material in logs (violates token-hygiene). Now logs only the host.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(manager): stop logging PostHog API-key prefix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request merges the v2.1.x branch into main, representing a significant modernization and consolidation of the Squawk codebase.
Key Changes
1. Architectural Consolidation
2. Frontend Modernization
3. Testing & Quality Assurance
4. Infrastructure & Deployment
5. Documentation & Standards