diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ac1cda..294ea75 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,8 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ## [Unreleased] +- Evaluate security-cohort R3 staleness at the receipt observation time; report clock-only arrivals as pending security coverage and covered clock departures explicitly. Preserve metadata-only R3 failure diagnostics beside publication outputs. + - Clarify locked contributor setup, optional test dependencies, and credential-free fixture verification without workstation discovery. ### Added diff --git a/docs/portfolio-cohort-transition-contract.md b/docs/portfolio-cohort-transition-contract.md index b4bf5f0..0371fdc 100644 --- a/docs/portfolio-cohort-transition-contract.md +++ b/docs/portfolio-cohort-transition-contract.md @@ -25,13 +25,13 @@ receipt: required = prospective outgoing = prior − prospective collection = required ∪ outgoing ↓ -producer independently recomputes the candidate cohort +producer independently recomputes the candidate cohort as of receipt produced_at ↓ R1 receipt == required ∪ outgoing R2 required ∩ outgoing == ∅ R3 candidate == required (exact set equality) R4 outgoing ∩ candidate == ∅ -R5 final ⊆ receipt +R5 final ⊆ receipt ∪ explicitly pending clock-only arrivals R6 every departure positively explained ↓ transactional publication @@ -43,6 +43,63 @@ hand-maintained integer. That is strictly stronger on membership — a same-size swap passed the old count tripwire silently and now fails R3 with both sides named. +## Receipt observation clock and pending coverage + +R3 evaluates time-relative activity staleness at the receipt's timezone-aware +`produced_at` observation time. It uses the inputs observed by the current run: +activity timestamp, classification, and prior security evidence. Only the clock +changes; inputs are not restored from history. A moved HEAD, reclassification, +or real coverage gap that changes membership at that observation time still +fails the exact set comparison. Missing, invalid, or timezone-less receipt time +fails closed. Receipt freshness and publication guards still use the run/live +clock. + +The same observed inputs are also evaluated at the run clock. Repositories that +enter attention only because of elapsed time are listed as **pending security +coverage**, remain in current attention, and are not counted as collected or +admitted security evidence. Covered repositories that leave attention only +because of elapsed time are recorded without failing. R1, R2, and R4 remain +strict; R4 checks the current candidate as before, so declared outgoing members +cannot become pending. Declared-outgoing departure evidence is unchanged. R6 +accepts a clock-only departure only when both the pre-security and post-security +views show that departure from changing the clock alone. + +`portfolio-cohort-transition-latest.json` adds these fields under `transition`: + +- `receipt_observed_at` +- `pending_security_coverage_count` and `pending_security_coverage_repositories` +- `clock_departed_count` and `clock_departed_repositories` + +For runs with clock differences, the same lists and counts appear under +`portfolio-truth-latest.json` → `source_summary.security_coverage_clock`, with +`receipt_observed_at` and `evaluated_at`. The portfolio Markdown report names +each pending repository and covered clock departure. These are coverage +annotations; the existing attention-state vocabulary remains unchanged. + +On R3 publication failure, a unique +`portfolio-cohort-r3-failure--.json` is written in the +requested output directory before the error is returned. The error points to +it. It contains receipt identity/time, receipt and run evaluation clocks, +required/collected and both derived cohorts, and differing repositories' +activity timestamp/source, classification, local HEAD, and attention states. +It contains metadata only and is outside the publication transaction, so the +prior truth and reports remain untouched. In-memory snapshot builds carry the +same diagnostics on `CohortR3Error` without writing files. + +Operator-scripts integration: advance `release-pins.json`'s +`auditor_truth_ref.value` to the reviewed producer commit containing this fix, +then regenerate and activate the governed release manifest. Both wrappers read +that shared pin through release preflight; keep one pin. Extend +`portfolio-maintenance.sh`'s `read_cohort_transition()` JSON projection with the +five new transition fields listed above, so its terminal-state/history +`cohort_transition` summary retains the pending repositories and clock departures. +Any digest or summary consumer should display +`pending_security_coverage_repositories` as pending (never covered/clean) and +`clock_departed_repositories` as already covered. Read the lists from the +transition outcome or the optional truth summary; absent lists mean empty for +older outputs. Exit status remains the failure gate. No new CLI flag, changed +receipt schema, attention-state enum, or hand-maintained cohort count is needed. + ## Surfaces | Component | Role | @@ -59,8 +116,9 @@ two equivalent algorithms would silently drift apart and reproduce the deadlock. ## Departure evidence -A repository may leave the published default-attention cohort only with positive -evidence, through one of three branches: +A source or security change may remove a repository from the published +default-attention cohort only with positive evidence, through one of three +branches (covered clock departures are handled separately above): 1. **observed resolution** — prior high+critical > 0, fresh receipt, Dependabot `observed`, current high = critical = 0. @@ -99,8 +157,9 @@ Both are overridable per run (`--max-cohort-size`, `--max-cohort-delta`). ## Backward compatibility A receipt without the transition block is legacy-shaped: reconcile falls back to -`required = collected`, `outgoing = ∅`, which reproduces the pre-transition -semantics verbatim, size tripwire included. `--portfolio-truth-require-cohort-transition` +`required = collected`, `outgoing = ∅`, retaining the pre-transition partition +and size tripwire. Receipt-time recency evaluation applies in this mode too. +`--portfolio-truth-require-cohort-transition` turns "a stale collector silently reverted us to a deadlock" into a named refusal. `--portfolio-truth-security-cohort-count` is retained as a deprecated alias for one release so a stale deployed wrapper cannot hard-error on an unknown diff --git a/src/github_repo_auditor/portfolio_truth_publish.py b/src/github_repo_auditor/portfolio_truth_publish.py index 5b5ea57..1011c1a 100644 --- a/src/github_repo_auditor/portfolio_truth_publish.py +++ b/src/github_repo_auditor/portfolio_truth_publish.py @@ -11,6 +11,7 @@ from datetime import datetime from pathlib import Path from typing import Iterable, Iterator +from uuid import uuid4 from github_repo_auditor.github_security_coverage import ( DEFAULT_ATTENTION_STATES, @@ -23,6 +24,7 @@ COHORT_TRANSITION_PROTOCOL, ) from github_repo_auditor.portfolio_truth_reconcile import ( + CohortR3Error, CohortTransitionOutcome, build_portfolio_truth_snapshot, load_prior_notion_context, @@ -129,9 +131,12 @@ def _load_prior_security_alerts( current_security_metadata.get("produced_at"), field="Current security receipt produced_at", ) - if evidence.generated_at > current_produced_at and not _is_same_bound_security_receipt( - evidence.prior_security_metadata or {}, - current_security_metadata, + if ( + evidence.generated_at > current_produced_at + and not _is_same_bound_security_receipt( + evidence.prior_security_metadata or {}, + current_security_metadata, + ) ): raise PortfolioTruthPublishError( "Prior PortfolioTruth was generated after the current security receipt." @@ -499,10 +504,7 @@ def _publish_portfolio_truth_locked( and security_alerts_by_name is not None else None ) - if ( - prior_security_evidence is not None - and security_cohort_transition is not None - ): + if prior_security_evidence is not None and security_cohort_transition is not None: declared_prior_truth_sha256 = security_cohort_transition.get( "prior_truth_sha256" ) @@ -513,35 +515,55 @@ def _publish_portfolio_truth_locked( f"receipt={declared_prior_truth_sha256!r}; " f"observed={prior_security_evidence.content_sha256!r}." ) - build_result = build_portfolio_truth_snapshot( - workspace_root=workspace_root, - catalog_path=catalog_path, - legacy_registry_path=legacy_registry_path, - include_notion=include_notion, - notion_context_fallback=notion_context_fallback, - release_count_by_name=release_count_by_name, - degraded_dimensions_by_name=degraded_dimensions_by_name, - security_alerts_by_name=security_alerts_by_name, - security_coverage_metadata=security_coverage_metadata, - prior_security_alerts_by_name=( - prior_security_evidence.alerts_by_full_name - if prior_security_evidence is not None - else None - ), - prior_security_cohort_repositories=( - prior_security_evidence.final_cohort_repositories - if prior_security_evidence is not None - else None - ), - security_cohort_required=security_cohort_required, - security_cohort_outgoing=security_cohort_outgoing, - security_cohort_transition=security_cohort_transition, - require_cohort_transition=require_cohort_transition, - repo_status_by_name=repo_status_by_name, - producer=producer_evidence.to_dict() if producer_evidence else {}, - prior_notion_generated_at=prior_notion_generated_at, - now=now, - ) + try: + build_result = build_portfolio_truth_snapshot( + workspace_root=workspace_root, + catalog_path=catalog_path, + legacy_registry_path=legacy_registry_path, + include_notion=include_notion, + notion_context_fallback=notion_context_fallback, + release_count_by_name=release_count_by_name, + degraded_dimensions_by_name=degraded_dimensions_by_name, + security_alerts_by_name=security_alerts_by_name, + security_coverage_metadata=security_coverage_metadata, + prior_security_alerts_by_name=( + prior_security_evidence.alerts_by_full_name + if prior_security_evidence is not None + else None + ), + prior_security_cohort_repositories=( + prior_security_evidence.final_cohort_repositories + if prior_security_evidence is not None + else None + ), + security_cohort_required=security_cohort_required, + security_cohort_outgoing=security_cohort_outgoing, + security_cohort_transition=security_cohort_transition, + require_cohort_transition=require_cohort_transition, + repo_status_by_name=repo_status_by_name, + producer=producer_evidence.to_dict() if producer_evidence else {}, + prior_notion_generated_at=prior_notion_generated_at, + now=now, + ) + except CohortR3Error as exc: + stamp = now.strftime("%Y-%m-%dT%H%M%SZ") if now else "unknown-clock" + diagnostics_path = ( + output_dir / f"portfolio-cohort-r3-failure-{stamp}-{uuid4().hex}.json" + ) + payload = {**exc.diagnostics, "error": str(exc)} + try: + staged = _stage_text( + diagnostics_path, json.dumps(payload, indent=2, sort_keys=True) + "\n" + ) + staged.replace(diagnostics_path) + _fsync_directory(output_dir) + except OSError as write_error: + raise PortfolioTruthPublishError( + f"{exc} R3 diagnostics could not be written to {diagnostics_path}: {write_error}" + ) from exc + raise CohortR3Error( + f"{exc} R3 diagnostics: {diagnostics_path}", exc.diagnostics + ) from exc validate_truth_snapshot( build_result.snapshot, security_max_age_hours=security_max_age_hours, diff --git a/src/github_repo_auditor/portfolio_truth_reconcile.py b/src/github_repo_auditor/portfolio_truth_reconcile.py index e016368..78b266f 100644 --- a/src/github_repo_auditor/portfolio_truth_reconcile.py +++ b/src/github_repo_auditor/portfolio_truth_reconcile.py @@ -22,7 +22,9 @@ group_entry_for_path, load_portfolio_catalog, ) -from github_repo_auditor.portfolio_context_contract import has_substantive_readme_support +from github_repo_auditor.portfolio_context_contract import ( + has_substantive_readme_support, +) from github_repo_auditor.portfolio_pathing import build_operating_path_entry from github_repo_auditor.portfolio_repository_state import observe_repository_state from github_repo_auditor.portfolio_truth_coverage import build_coverage_envelope @@ -196,7 +198,7 @@ class CohortTransitionOutcome: """Planned and actual outcome of one governed cohort transition. ``protocol`` is ``None`` for a legacy receipt that carries no transition - block; those runs reproduce the pre-transition semantics exactly. + block; those runs retain the legacy partition and size tripwire. """ protocol: str | None @@ -211,6 +213,9 @@ class CohortTransitionOutcome: departed: tuple[str, ...] retained_due_security: tuple[str, ...] final: tuple[str, ...] + pending_security_coverage: tuple[str, ...] = () + clock_departed: tuple[str, ...] = () + receipt_observed_at: str | None = None def to_dict(self) -> dict[str, Any]: """Return the inspectable transition record carried by producer evidence.""" @@ -235,9 +240,106 @@ def to_dict(self) -> dict[str, Any]: "retained_due_security": list(self.retained_due_security), "final_count": len(self.final), "final_repositories": list(self.final), + "pending_security_coverage_count": len(self.pending_security_coverage), + "pending_security_coverage_repositories": list( + self.pending_security_coverage + ), + "clock_departed_count": len(self.clock_departed), + "clock_departed_repositories": list(self.clock_departed), + "receipt_observed_at": self.receipt_observed_at, } +class CohortR3Error(ValueError): + """A refused candidate cohort, with metadata for durable failure evidence.""" + + def __init__(self, message: str, diagnostics: dict[str, Any]) -> None: + super().__init__(message) + self.diagnostics = diagnostics + + +def _r3_diagnostics( + *, + candidate_projects: list[PortfolioTruthProject], + current_projects: list[PortfolioTruthProject], + required: tuple[str, ...], + receipt: tuple[str, ...], + metadata: dict[str, Any] | None, + now: datetime | None, + observed_at: datetime | None, +) -> dict[str, Any]: + as_of = {p.identity.repo_full_name: p for p in candidate_projects} + current = {p.identity.repo_full_name: p for p in current_projects} + candidate = _cohort_repositories_for(candidate_projects, label="R3 diagnostics") + current_cohort = _cohort_repositories_for( + current_projects, label="R3 diagnostics now" + ) + differences = (set(required) ^ set(candidate)) | ( + set(candidate) ^ set(current_cohort) + ) + rows = [] + for repository in sorted(differences, key=str.lower): + project = current.get(repository) or as_of.get(repository) + activity = ( + getattr(project.derived, "last_meaningful_activity_at", None) + if project + else None + ) + declared = getattr(project, "declared", None) + provenance = getattr(project, "provenance", {}) + rows.append( + { + "repository": repository, + "activity_timestamp": activity.isoformat() if activity else None, + "activity_source": provenance.get( + "derived.last_meaningful_activity_at", {} + ).get("source"), + "classification": { + field: getattr(declared, field, None) + for field in ( + "lifecycle_state", + "operating_path", + "category", + "criticality", + "doctor_standard", + ) + }, + "archived": getattr(project.derived, "archived", None) + if project + else None, + "path_override": getattr(project.derived, "path_override", None) + if project + else None, + "attention_as_of_receipt": as_of[repository].derived.attention_state + if repository in as_of and observed_at + else None, + "attention_now": current[repository].derived.attention_state + if repository in current + else None, + "local_head": getattr(project, "repository_state", {}) + .get("local", {}) + .get("head"), + } + ) + metadata = metadata or {} + return { + "schema_version": "PortfolioCohortR3DiagnosticsV1", + "receipt": { + key: metadata.get(key) + for key in ("receipt_id", "content_sha256", "produced_at") + }, + "evaluation_clock": observed_at.isoformat() if observed_at else None, + "run_clock": now.isoformat() if now else None, + "required_repositories": list(required), + "collection_repositories": list(receipt), + "candidate_as_of_receipt_repositories": list(candidate) + if observed_at + else None, + "candidate_now_repositories": list(current_cohort), + "differing_repositories": rows, + } + + @dataclass(frozen=True) class PortfolioTruthBuildResult: snapshot: PortfolioTruthSnapshot @@ -295,6 +397,11 @@ def _validate_security_receipt_cohort_identity( outgoing_repositories: tuple[str, ...] | None = None, transition: dict[str, Any] | None = None, require_cohort_transition: bool = False, + current_candidate_projects: list[PortfolioTruthProject] | None = None, + security_coverage_metadata: dict[str, Any] | None = None, + receipt_observed_at: datetime | None = None, + now: datetime | None = None, + receipt_final_projects: list[PortfolioTruthProject] | None = None, ) -> CohortTransitionOutcome: """Bind receipt membership to attention derived before the new receipt. @@ -305,15 +412,20 @@ def _validate_security_receipt_cohort_identity( * R3 ``candidate == required`` (exact set equality; the fail-closed control is re-pointed, never relaxed) * R4 ``outgoing intersect candidate == empty`` - * R5 ``final subset of receipt`` + * R5 ``final subset of receipt union explicitly pending clock arrivals`` * R6 every departure carries positive evidence A receipt without a transition block falls back to ``required = receipt`` - and ``outgoing = ()``, which reproduces the pre-transition semantics - verbatim, size tripwire included. + and ``outgoing = ()``, retaining the legacy size tripwire. Receipt-backed + builds evaluate R3 recency as of the receipt in either partition mode. """ if candidate_projects is None: candidate_projects = projects + current_candidate_projects = ( + current_candidate_projects + if current_candidate_projects is not None + else candidate_projects + ) prior_security_alerts_by_name = prior_security_alerts_by_name or {} receipt_repositories = tuple(sorted(security_alerts_by_name, key=str.lower)) @@ -389,14 +501,27 @@ def _validate_security_receipt_cohort_identity( expected_count=len(receipt_repositories), ) except SecurityCoverageError as exc: - raise ValueError( + raise CohortR3Error( "PortfolioTruth GitHub security receipt cohort cannot match freshly " - f"derived default attention: {exc}." + f"derived default attention: {exc}.", + _r3_diagnostics( + candidate_projects=candidate_projects, + current_projects=current_candidate_projects, + required=required, + receipt=receipt_repositories, + metadata=security_coverage_metadata, + now=now, + observed_at=receipt_observed_at, + ), ) from exc # R4 — a declared departure may not still be in derived default attention. + current_candidate_repositories = _cohort_repositories_for( + current_candidate_projects, label="current pre-security candidate cohort" + ) still_attended = sorted( - set(outgoing) & set(candidate_repositories), key=str.lower + set(outgoing) & set(current_candidate_repositories), + key=str.lower, ) if still_attended: raise ValueError( @@ -408,20 +533,53 @@ def _validate_security_receipt_cohort_identity( receipt_only = sorted(set(required) - set(candidate_repositories), key=str.lower) derived_only = sorted(set(candidate_repositories) - set(required), key=str.lower) if receipt_only or derived_only: - raise ValueError( + raise CohortR3Error( "PortfolioTruth GitHub security receipt cohort differs from freshly " "derived pre-security default attention: " - f"receipt_only={receipt_only}; derived_only={derived_only}." + f"receipt_only={receipt_only}; derived_only={derived_only}.", + _r3_diagnostics( + candidate_projects=candidate_projects, + current_projects=current_candidate_projects, + required=required, + receipt=receipt_repositories, + metadata=security_coverage_metadata, + now=now, + observed_at=receipt_observed_at, + ), ) + pending = tuple( + sorted( + set(current_candidate_repositories) + - set(candidate_repositories) + - set(receipt_repositories), + key=str.lower, + ) + ) + clock_departed = tuple( + sorted( + set(candidate_repositories) - set(current_candidate_repositories), + key=str.lower, + ) + ) final_repositories = _cohort_repositories_for( projects, label="post-receipt security cohort", ) + receipt_final_repositories = ( + _cohort_repositories_for( + receipt_final_projects, label="post-security receipt-time cohort" + ) + if receipt_final_projects is not None + else final_repositories + ) + covered_clock_departures = set(clock_departed) & ( + set(receipt_final_repositories) - set(final_repositories) + ) - # R5 — published attention may never exceed collected evidence. + # R5 — any uncollected attention must be an explicit clock-only arrival. final_only = sorted( - set(final_repositories) - set(receipt_repositories), + set(final_repositories) - set(receipt_repositories) - set(pending), key=str.lower, ) if final_only: @@ -442,7 +600,8 @@ def final_project_is_archived(repository: str) -> bool: ] return len(matches) == 1 and matches[0].derived.archived is True - # R6 — every departure carries positive evidence. + # R6 — source/security departures still require positive evidence; elapsed + # time may explain only a covered departure in both security views. non_collectable = [ repository for repository in departed @@ -459,7 +618,8 @@ def final_project_is_archived(repository: str) -> bool: unresolved_departures = [ repository for repository in departed - if not _is_verified_security_cohort_departure( + if repository not in covered_clock_departures + and not _is_verified_security_cohort_departure( prior_security_alerts_by_name.get(repository), security_alerts_by_name.get(repository), final_project_archived=final_project_is_archived(repository), @@ -476,9 +636,7 @@ def final_project_is_archived(repository: str) -> bool: # An outgoing member whose fresh evidence keeps it in default attention is a # valid outcome: security evidence outranks lifecycle classification (I5). retained_due_security = tuple( - repository - for repository in outgoing - if repository in set(final_repositories) + repository for repository in outgoing if repository in set(final_repositories) ) planned_incoming = tuple( @@ -549,6 +707,11 @@ def final_project_is_archived(repository: str) -> bool: departed=departed, retained_due_security=retained_due_security, final=final_repositories, + pending_security_coverage=pending, + clock_departed=tuple(sorted(covered_clock_departures, key=str.lower)), + receipt_observed_at=receipt_observed_at.isoformat() + if receipt_observed_at + else None, ) @@ -615,6 +778,7 @@ def _is_verified_security_cohort_departure( declared_source_departure = declared_outgoing and current_clean return observed_resolution or observed_archive or declared_source_departure + def _candidate_prior_security_alerts( prior_security_alerts_by_name: dict[str, dict], prior_security_cohort_repositories: tuple[str, ...] | None, @@ -691,6 +855,47 @@ class CandidateCohort: repositories: tuple[str, ...] +def _projects_at_clock( + projects: list[PortfolioTruthProject], *, now: datetime +) -> list[PortfolioTruthProject]: + """Re-evaluate recency without re-reading or changing any observed inputs.""" + result = [] + for project in projects: + declared, derived = project.declared, project.derived + activity_status = _activity_status_for( + derived.last_meaningful_activity_at, now=now + ) + admission = derive_security_admission(project.security.to_dict()) + risk, attention = build_project_decision( + display_name=project.identity.display_name, + operating_path=declared.operating_path, + path_override=derived.path_override, + context_quality=derived.context_quality, + activity_status=activity_status, + archived=derived.archived, + lifecycle_state=declared.lifecycle_state, + category=declared.category, + criticality=declared.criticality, + doctor_standard=declared.doctor_standard, + known_risks_present=derived.known_risks_present, + run_instructions_present=derived.run_instructions_present, + security_coverage_state=admission.effective_coverage_state, + security_high_alerts=admission.total_open_high, + security_critical_alerts=admission.total_open_critical + + admission.total_open_secrets, + ) + result.append( + replace( + project, + derived=replace( + derived, activity_status=activity_status, attention_state=attention + ), + risk=RiskFields(**risk), + ) + ) + return result + + def build_materialization_context( *, workspace_root: Path, @@ -855,9 +1060,46 @@ def build_portfolio_truth_snapshot( ) cohort_transition: CohortTransitionOutcome | None = None if reconcile_receipt: + assert candidate is not None + assert security_coverage_metadata is not None + assert security_alerts_by_name is not None + receipt_time_value = security_coverage_metadata.get("produced_at") + try: + receipt_time = datetime.fromisoformat( + str(receipt_time_value).replace("Z", "+00:00") + ) + if receipt_time.tzinfo is None: + raise ValueError("timezone missing") + # A receipt cannot observe the future; evaluating as of a later clock + # could reclassify a near-boundary repo as a clock-only arrival. + if now is not None and now.tzinfo is not None and receipt_time > now: + raise ValueError("observation time is after the run clock") + except (ValueError, TypeError) as exc: + raise CohortR3Error( + "PortfolioTruth R3 requires a parseable timezone-aware security receipt observation time (produced_at) that is not after the run clock.", + _r3_diagnostics( + candidate_projects=candidate.projects, + current_projects=candidate.projects, + required=security_cohort_required + if security_cohort_required is not None + else tuple(sorted(security_alerts_by_name)), + receipt=tuple(sorted(security_alerts_by_name)), + metadata=security_coverage_metadata, + now=now, + observed_at=None, + ), + ) from exc + receipt_candidate_projects = _projects_at_clock( + candidate.projects, now=receipt_time + ) cohort_transition = _validate_security_receipt_cohort_identity( projects=projects, - candidate_projects=candidate.projects if candidate else None, + candidate_projects=receipt_candidate_projects, + current_candidate_projects=candidate.projects, + security_coverage_metadata=security_coverage_metadata, + receipt_observed_at=receipt_time, + now=now, + receipt_final_projects=_projects_at_clock(projects, now=receipt_time), security_alerts_by_name=security_alerts_by_name, prior_security_alerts_by_name=prior_security_alerts, required_repositories=security_cohort_required, @@ -865,6 +1107,19 @@ def build_portfolio_truth_snapshot( transition=security_cohort_transition, require_cohort_transition=require_cohort_transition, ) + # Pending attention is deliberately uncollected, never synthetic coverage. + projects = [ + replace( + project, + security=replace( + project.security, cohort_member=False, cohort_policy="" + ), + ) + if project.identity.repo_full_name + in cohort_transition.pending_security_coverage + else project + for project in projects + ] projects.sort( key=lambda item: ( item.identity.section_marker.lower(), @@ -882,6 +1137,21 @@ def build_portfolio_truth_snapshot( notion_context_carried_forward=context.notion_context_carried_forward, checkout_collisions=context.checkout_collisions, ) + if cohort_transition is not None and ( + cohort_transition.pending_security_coverage or cohort_transition.clock_departed + ): + source_summary["security_coverage_clock"] = { + "receipt_observed_at": cohort_transition.receipt_observed_at, + "evaluated_at": now.isoformat(), + "pending_security_coverage_count": len( + cohort_transition.pending_security_coverage + ), + "pending_security_coverage_repositories": list( + cohort_transition.pending_security_coverage + ), + "clock_departed_count": len(cohort_transition.clock_departed), + "clock_departed_repositories": list(cohort_transition.clock_departed), + } warnings = build_warnings( catalog_errors=source_summary["catalog_errors"], catalog_warnings=source_summary["catalog_warnings"], @@ -928,6 +1198,7 @@ def build_portfolio_truth_snapshot( cohort_transition=cohort_transition, ) + def _cataloged_supplementary_projects( *, catalog_data: dict[str, Any], now: datetime ) -> list[dict[str, Any]]: diff --git a/src/github_repo_auditor/portfolio_truth_render.py b/src/github_repo_auditor/portfolio_truth_render.py index e40c5f1..456620d 100644 --- a/src/github_repo_auditor/portfolio_truth_render.py +++ b/src/github_repo_auditor/portfolio_truth_render.py @@ -250,6 +250,15 @@ def render_portfolio_report_markdown( lines.append("") lines.extend(["## Security Posture", ""]) + clock_coverage = snapshot.source_summary.get("security_coverage_clock", {}) + for repository in clock_coverage.get("pending_security_coverage_repositories", []): + lines.append( + f"- **Pending security coverage**: `{repository}` entered default attention after the receipt observation because it became stale." + ) + for repository in clock_coverage.get("clock_departed_repositories", []): + lines.append( + f"- Covered by the receipt; left default attention since observation due to the clock: `{repository}`." + ) attention = _security_attention_items(snapshot.projects) scanned_count = security_overview["scanned_count"] if attention: diff --git a/src/github_repo_auditor/portfolio_truth_validate.py b/src/github_repo_auditor/portfolio_truth_validate.py index aa92d9a..5e56497 100644 --- a/src/github_repo_auditor/portfolio_truth_validate.py +++ b/src/github_repo_auditor/portfolio_truth_validate.py @@ -10,6 +10,7 @@ from typing import Any, Union, get_args, get_origin, get_type_hints from github_repo_auditor.github_security_coverage import ( + DEFAULT_ATTENTION_STATES, DEFAULT_COHORT_POLICY, GITHUB_SECURITY_RECEIPT_SCHEMA_VERSION, PROVIDER_NAMES, @@ -20,7 +21,9 @@ _validate_remote_repository, validate_normalized_security_provider, ) -from github_repo_auditor.portfolio_checkout_authority import validate_checkout_authority_envelope +from github_repo_auditor.portfolio_checkout_authority import ( + validate_checkout_authority_envelope, +) from github_repo_auditor.portfolio_pathing import ( VALID_MATURITY_PROGRAMS, VALID_OPERATING_PATHS, @@ -41,7 +44,9 @@ build_warnings, ) from github_repo_auditor.portfolio_truth_precedence import PRECEDENCE_MATRIX -from github_repo_auditor.portfolio_truth_provenance import REQUIRED_PROJECT_PROVENANCE_KEYS +from github_repo_auditor.portfolio_truth_provenance import ( + REQUIRED_PROJECT_PROVENANCE_KEYS, +) from github_repo_auditor.portfolio_truth_render import registry_project_labels from github_repo_auditor.portfolio_truth_sources import WORKSPACE_EXCLUSION_REASONS from github_repo_auditor.portfolio_truth_types import ( @@ -537,6 +542,68 @@ def _validate_snapshot_metadata( notion_context_carried_forward=notion_context_carried_forward, checkout_collisions=summary["checkout_collisions"]["groups"], ) + if "security_coverage_clock" in summary: + clock = summary["security_coverage_clock"] + security_input = snapshot.inputs.get("github_security") or {} + if not isinstance(clock, dict) or set(clock) != { + "receipt_observed_at", + "evaluated_at", + "pending_security_coverage_count", + "pending_security_coverage_repositories", + "clock_departed_count", + "clock_departed_repositories", + }: + raise ValueError("PortfolioTruth security coverage clock is invalid.") + if clock[ + "evaluated_at" + ] != snapshot.generated_at.isoformat() or _parse_datetime( + clock["receipt_observed_at"], "security_coverage_clock.receipt_observed_at" + ) != _parse_datetime( + security_input.get("produced_at"), "inputs.github_security.produced_at" + ): + raise ValueError( + "PortfolioTruth security coverage clock binding is invalid." + ) + projects_by_repo = { + project.identity.repo_full_name: project for project in snapshot.projects + } + for prefix in ("pending_security_coverage", "clock_departed"): + repositories = clock[f"{prefix}_repositories"] + count = clock[f"{prefix}_count"] + if ( + not isinstance(repositories, list) + or any( + not isinstance(repo, str) or repo not in projects_by_repo + for repo in repositories + ) + or repositories != sorted(set(repositories), key=str.lower) + or type(count) is not int + or count != len(repositories) + ): + raise ValueError( + "PortfolioTruth security coverage clock repositories are invalid." + ) + for repo in repositories: + project = projects_by_repo[repo] + if prefix == "pending_security_coverage": + if ( + project.security.cohort_member + or project.derived.attention_state + not in DEFAULT_ATTENTION_STATES + ): + raise ValueError( + "PortfolioTruth pending security coverage is invalid." + ) + elif ( + not project.security.cohort_member + or project.derived.attention_state in DEFAULT_ATTENTION_STATES + ): + raise ValueError("PortfolioTruth clock departure is invalid.") + if set(clock["pending_security_coverage_repositories"]) & set( + clock["clock_departed_repositories"] + ): + raise ValueError("PortfolioTruth security coverage clock sets overlap.") + expected_summary["security_coverage_clock"] = clock if summary != expected_summary: raise ValueError("PortfolioTruth source summary differs from producer facts.") expected_warnings = build_warnings( diff --git a/src/github_repo_auditor/weekly_command_center.py b/src/github_repo_auditor/weekly_command_center.py index 3296328..9b6c3bf 100644 --- a/src/github_repo_auditor/weekly_command_center.py +++ b/src/github_repo_auditor/weekly_command_center.py @@ -491,6 +491,12 @@ def render_weekly_command_center_markdown(digest: dict[str, Any]) -> str: "- Security overlay not run for this snapshot " "(re-run with `--portfolio-truth-include-security`)." ) + pending = list(security_posture.get("pending_coverage_repositories") or []) + if pending: + lines.append( + f"- Security coverage PENDING for {len(pending)} repo(s) that entered attention " + f"after the last receipt ({', '.join(pending)}); do not treat them as clear." + ) lines.extend(["", "## Movement"]) lines.append( @@ -722,8 +728,15 @@ def _build_security_summary(portfolio_truth: dict[str, Any]) -> dict[str, Any]: total_secrets += admission.total_open_secrets if admission.has_findings: repos_with_open += 1 + # Repos that entered attention after the last security receipt are not + # cohort members yet, so the loop above cannot see them. Carry them + # explicitly so the digest never reports "all clear" over uncovered repos. + clock = _mapping(_mapping(portfolio_truth.get("source_summary")).get("security_coverage_clock")) + pending = [str(name) for name in clock.get("pending_security_coverage_repositories") or []] return { "scanned_count": scanned, + "pending_coverage_count": len(pending), + "pending_coverage_repositories": pending, "repos_with_open_high_critical": repos_with_open, "repos_with_blocking_findings": repos_with_open, "unadmitted_count": unadmitted, diff --git a/tests/test_portfolio_cohort_transition.py b/tests/test_portfolio_cohort_transition.py index 4096ad2..236642a 100644 --- a/tests/test_portfolio_cohort_transition.py +++ b/tests/test_portfolio_cohort_transition.py @@ -33,6 +33,7 @@ publish_portfolio_truth, ) from github_repo_auditor.portfolio_truth_reconcile import ( + CohortR3Error, _validate_security_receipt_cohort_identity, ) @@ -1133,3 +1134,287 @@ def test_case_19_interrupted_publication_is_recovered_from_the_journal( now=NOW, ) assert latest.read_bytes() != before + + +# Receipt-time regression cases use real Git activity observations and publication. +RECEIPT_AT = datetime(2026, 10, 11, 8, 31, 57, tzinfo=timezone.utc) +RUN_AT = datetime(2026, 10, 11, 9, 0, 5, tzinfo=timezone.utc) +ACTIVITY_AT = datetime(2026, 9, 10, 8, 59, 40, tzinfo=timezone.utc) +CLOCK_REPO = "d/agent-permission-diff-bot" + + +def _commit_activity(project: Path, at: datetime) -> None: + import os + + subprocess.run( + [ + "git", + "-c", + "user.name=Fixture", + "-c", + "user.email=fixture@example.invalid", + "commit", + "--allow-empty", + "-m", + "fixture activity", + ], + cwd=project, + capture_output=True, + check=True, + env={ + **os.environ, + "GIT_AUTHOR_DATE": at.isoformat(), + "GIT_COMMITTER_DATE": at.isoformat(), + }, + ) + + +def _clock_workspace( + transition_workspace: tuple[Path, Path, Path], + *, + operating_path: str = "finish", + category: str = "vanity", +) -> tuple[Path, Path, Path]: + workspace, catalog, output = transition_workspace + _git_project(workspace, "agent-permission-diff-bot", CLOCK_REPO) + _commit_activity(workspace / "agent-permission-diff-bot", ACTIVITY_AT) + with catalog.open("a") as handle: + handle.write( + f"\n agent-permission-diff-bot:\n lifecycle_state: maintenance\n operating_path: {operating_path}\n category: {category}\n" + ) + return workspace, catalog, output + + +def _clock_publish( + workspace: Path, + catalog: Path, + output: Path, + required: tuple[str, ...], + *, + metadata: dict | None = None, +): + metadata = _metadata(RECEIPT_AT) if metadata is None else metadata + metadata["cohort_repository_count"] = len(required) + alerts = validate_security_coverage_receipt( + _sign(_legacy_receipt_payload(required, now=RECEIPT_AT)), + max_age_hours=24, + expected_cohort_count=len(required), + now=RUN_AT, + ).entries_by_full_name + return _publish( + workspace, + catalog, + output, + security_alerts_by_name=alerts, + security_coverage_metadata=metadata, + security_cohort_required=required, + security_cohort_outgoing=(), + security_cohort_transition=_transition(kind="steady", prior_truth_sha256=None), + now=RUN_AT, + ) + + +def test_receipt_clock_crossing_is_pending_and_is_published( + transition_workspace: tuple[Path, Path, Path], +) -> None: + workspace, catalog, output = _clock_workspace(transition_workspace) + _clock_publish(workspace, catalog, output, ("d/Alpha",)) + truth = json.loads((output / "portfolio-truth-latest.json").read_text()) + clock = truth["source_summary"]["security_coverage_clock"] + assert clock["pending_security_coverage_repositories"] == [CLOCK_REPO] + assert clock["pending_security_coverage_count"] == 1 + project = next( + p for p in truth["projects"] if p["identity"]["repo_full_name"] == CLOCK_REPO + ) + assert project["derived"]["attention_state"] == "decision-needed" + assert project["security"]["cohort_member"] is False + assert project["security"]["coverage_state"] == "unknown" + assert ( + "**Pending security coverage**: `d/agent-permission-diff-bot`" + in (workspace / "PORTFOLIO-AUDIT-REPORT.md").read_text() + ) + transition = json.loads( + (output / PORTFOLIO_COHORT_TRANSITION_FILENAME).read_text() + )["transition"] + assert transition["pending_security_coverage_repositories"] == [CLOCK_REPO] + assert not list(output.glob("portfolio-cohort-r3-failure-*.json")) + # The additive summary remains readable as prior truth for the next planner/run. + from github_repo_auditor.portfolio_truth_publish import load_prior_security_evidence + + prior = load_prior_security_evidence( + output / "portfolio-truth-latest.json", security_max_age_hours=24 + ) + assert CLOCK_REPO in prior.final_cohort_repositories + assert CLOCK_REPO not in prior.alerts_by_full_name + + +@pytest.mark.parametrize("failure", ["moved-head", "gap", "reclassification"]) +def test_non_clock_r3_failure_writes_metadata_and_preserves_truth( + transition_workspace: tuple[Path, Path, Path], + failure: str, +) -> None: + workspace, catalog, output = _clock_workspace(transition_workspace) + _publish(workspace, catalog, output, now=RECEIPT_AT - timedelta(hours=1)) + latest = output / "portfolio-truth-latest.json" + before = latest.read_bytes() + import hashlib + + metadata = _metadata(RECEIPT_AT) + # Binding the prior truth is independently strict; pass the actual digest. + transition = _transition( + kind="steady", prior_truth_sha256=hashlib.sha256(before).hexdigest() + ) + if failure == "moved-head": + _commit_activity( + workspace / "agent-permission-diff-bot", RECEIPT_AT - timedelta(minutes=1) + ) + required = ( + "d/Alpha", + CLOCK_REPO, + ) # collection planned when this input was stale + expected_side = "receipt_only" + else: + if failure == "reclassification": + catalog.write_text( + catalog.read_text() + .replace("category: vanity\n", "category: infrastructure\n") + .replace("operating_path: finish", "operating_path: maintain") + .replace("lifecycle_state: maintenance", "lifecycle_state: active") + ) + else: + _commit_activity( + workspace / "agent-permission-diff-bot", + ACTIVITY_AT - timedelta(hours=1), + ) + required = ("d/Alpha",) + expected_side = "derived_only" + metadata["cohort_repository_count"] = len(required) + alerts = validate_security_coverage_receipt( + _sign(_legacy_receipt_payload(required, now=RECEIPT_AT)), + max_age_hours=24, + expected_cohort_count=len(required), + now=RUN_AT, + ).entries_by_full_name + with pytest.raises(CohortR3Error, match=expected_side) as caught: + _publish( + workspace, + catalog, + output, + security_alerts_by_name=alerts, + security_coverage_metadata=metadata, + security_cohort_required=required, + security_cohort_outgoing=(), + security_cohort_transition=transition, + now=RUN_AT, + ) + assert latest.read_bytes() == before + artifacts = list(output.glob("portfolio-cohort-r3-failure-*.json")) + assert len(artifacts) == 1 + assert str(artifacts[0]) in str(caught.value) + diagnostic = json.loads(artifacts[0].read_text()) + assert diagnostic["receipt"]["receipt_id"] == metadata["receipt_id"] + assert diagnostic["receipt"]["produced_at"] == RECEIPT_AT.isoformat() + assert diagnostic["evaluation_clock"] == RECEIPT_AT.isoformat() + assert diagnostic["run_clock"] == RUN_AT.isoformat() + assert diagnostic["required_repositories"] == sorted(required, key=str.lower) + row = next( + row + for row in diagnostic["differing_repositories"] + if row["repository"] == CLOCK_REPO + ) + assert row["activity_source"] == "git" + assert row["local_head"] is not None + assert row["classification"]["operating_path"] in {"finish", "maintain"} + if failure == "moved-head": + assert ( + row["activity_timestamp"] == (RECEIPT_AT - timedelta(minutes=1)).isoformat() + ) + assert row["attention_as_of_receipt"] == row["attention_now"] == "manual-only" + else: + assert row["attention_as_of_receipt"] in {"decision-needed", "active-infra"} + + +def test_clock_departure_is_covered_and_recorded( + transition_workspace: tuple[Path, Path, Path], +) -> None: + workspace, catalog, output = _clock_workspace( + transition_workspace, operating_path="maintain", category="commercial" + ) + _clock_publish( + workspace, + catalog, + output, + tuple(sorted(("d/Alpha", CLOCK_REPO), key=str.lower)), + ) + truth = json.loads((output / "portfolio-truth-latest.json").read_text()) + clock = truth["source_summary"]["security_coverage_clock"] + assert clock["clock_departed_repositories"] == [CLOCK_REPO] + assert clock["pending_security_coverage_repositories"] == [] + transition = json.loads( + (output / PORTFOLIO_COHORT_TRANSITION_FILENAME).read_text() + )["transition"] + assert ( + transition["clock_departed_repositories"] + == transition["departed_repositories"] + == [CLOCK_REPO] + ) + assert ( + "Covered by the receipt; left default attention" + in (workspace / "PORTFOLIO-AUDIT-REPORT.md").read_text() + ) + + +@pytest.mark.parametrize( + "receipt_time", + # The last case is after RUN_AT yet inside publication's 3-minute skew allowance. + [None, "broken", "2026-10-11T08:31:57", "2026-10-11T09:02:00+00:00"], +) +def test_invalid_receipt_time_fails_closed_with_diagnostics( + transition_workspace: tuple[Path, Path, Path], + receipt_time: str | None, +) -> None: + workspace, catalog, output = _clock_workspace(transition_workspace) + metadata = _metadata(RECEIPT_AT) + if receipt_time is None: + metadata.pop("produced_at") + else: + metadata["produced_at"] = receipt_time + with pytest.raises(CohortR3Error, match="R3 requires.*observation time") as caught: + _clock_publish(workspace, catalog, output, ("d/Alpha",), metadata=metadata) + artifacts = list(output.glob("portfolio-cohort-r3-failure-*.json")) + assert len(artifacts) == 1 + assert str(artifacts[0]) in str(caught.value) + assert json.loads(artifacts[0].read_text())["evaluation_clock"] is None + assert not (output / "portfolio-truth-latest.json").exists() + + +def test_clock_entering_declared_outgoing_still_fails_r4() -> None: + with pytest.raises(ValueError, match="outgoing repositories that are still"): + _validate_security_receipt_cohort_identity( + projects=_projects(("d/Alpha", CLOCK_REPO)), + candidate_projects=[ + _project("d/Alpha"), + _project(CLOCK_REPO, state="manual-only"), + ], + current_candidate_projects=_projects(("d/Alpha", CLOCK_REPO)), + security_alerts_by_name=_receipt(("d/Alpha", CLOCK_REPO)), + required_repositories=("d/Alpha",), + outgoing_repositories=(CLOCK_REPO,), + transition=_transition(kind="shrink", outgoing=(CLOCK_REPO,)), + ) + + +def test_current_security_retention_is_not_reported_as_clock_departure() -> None: + outcome = _validate_security_receipt_cohort_identity( + projects=[_project(CLOCK_REPO, state="decision-needed")], + candidate_projects=[_project(CLOCK_REPO, state="active-product")], + current_candidate_projects=[_project(CLOCK_REPO, state="parked")], + receipt_final_projects=[_project(CLOCK_REPO, state="decision-needed")], + security_alerts_by_name=_receipt((CLOCK_REPO,), **{CLOCK_REPO: _entry(high=1)}), + required_repositories=(CLOCK_REPO,), + outgoing_repositories=(), + transition=_transition(kind="steady"), + ) + assert outcome.final == (CLOCK_REPO,) + assert outcome.clock_departed == () + assert outcome.departed == () diff --git a/tests/test_portfolio_truth.py b/tests/test_portfolio_truth.py index 5c885e1..a68fb94 100644 --- a/tests/test_portfolio_truth.py +++ b/tests/test_portfolio_truth.py @@ -3392,7 +3392,7 @@ def test_empty_security_metadata_cannot_bypass_cohort_identity( catalog_path = tmp_path / "portfolio-catalog.yaml" catalog_path.write_text("repos: {}\n") - with pytest.raises(ValueError, match="expected 1, observed 0"): + with pytest.raises(ValueError, match="R3 requires.*observation time"): build_portfolio_truth_snapshot( workspace_root=workspace, catalog_path=catalog_path, diff --git a/tests/test_weekly_command_center.py b/tests/test_weekly_command_center.py index 87b4cc8..df80352 100644 --- a/tests/test_weekly_command_center.py +++ b/tests/test_weekly_command_center.py @@ -516,6 +516,40 @@ def test_security_posture_reports_clean_when_scanned_and_no_open_alerts() -> Non assert "All 2 admitted repos are clear" in rendered +def test_security_posture_names_pending_coverage_beside_all_clear() -> None: + # A repo that went stale after the last security receipt is pending + # coverage, not a cohort member; "all clear" must not stand alone. + portfolio_truth = { + "projects": [ + _security_project("CleanA", "baseline", _sec(True, 0, 0)), + _security_project("CleanB", "baseline", _sec(True, 0, 0)), + ], + "source_summary": { + "security_coverage_clock": { + "pending_security_coverage_count": 1, + "pending_security_coverage_repositories": ["saagpatel/StaleC"], + } + }, + } + digest = _digest_for(portfolio_truth) + assert digest["security_posture"]["pending_coverage_count"] == 1 + assert digest["security_posture"]["pending_coverage_repositories"] == ["saagpatel/StaleC"] + + rendered = render_weekly_command_center_markdown(digest) + assert "All 2 admitted repos are clear" in rendered + assert ( + "Security coverage PENDING for 1 repo(s) that entered attention after the " + "last receipt (saagpatel/StaleC); do not treat them as clear." + ) in rendered + + +def test_security_posture_has_no_pending_line_without_pending_repos() -> None: + portfolio_truth = {"projects": [_security_project("CleanA", "baseline", _sec(True, 0, 0))]} + digest = _digest_for(portfolio_truth) + assert digest["security_posture"]["pending_coverage_count"] == 0 + assert "PENDING" not in render_weekly_command_center_markdown(digest) + + def test_security_posture_ignores_projects_outside_security_cohort() -> None: out_of_cohort = _sec(False) out_of_cohort["cohort_member"] = False