Admin showcase panel for signed-in users by northdpole · Pull Request #1137 · OWASP/OpenCRE

northdpole · 2026-10-07T07:52:41Z

Admin IA redesign stacked on the OIE / OWASP-agent tip (cursor/oie-scheduled-runs-eb24, PR #1132). When we merge, merge the whole stack — not this PR onto main alone.

Mac handover

See docs/oie/admin-e2e-mac-handover.md — open questions first, then local run / API / E2E sequence. Continue validation on Mac, not cloud agent.

git fetch origin cursor/admin-panel-showcase-ccdd && git checkout cursor/admin-panel-showcase-ccdd && git pull
SKIP_UPSTREAM_SYNC=1 PORT=5001 make dev

What changed

make install / make dev → Docker Postgres + upstream CRE when empty
Admin Config: golden-set harvest + OWASP agent sync buttons
Graph JSON errors; OIE skip/flag observability
Default admin ingest still skips B/C unless skip_b/skip_c false (full E2E must override + LLM keys)

Explorer after upstream sync
explorer-after-upstream-sync.mp4

_{To show artifacts inline, enable in settings.}

Made with Cursor

coderabbitai · 2026-10-07T07:52:50Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 1

main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: OWASP/OpenCRE/.coderabbit.yml
Review profile: CHILL
Plan: Advanced
Run ID: 37533679-1f59-445c-a717-91568539081d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Allowlist GitHub owner logins, quote the API path, cap yaml at 256KiB before safe_load, and write packaged repos.yaml atomically. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

The 256KiB parse limit rejected a full GitHub org dump. Keep a size cap, but large enough for up to 1000 repository entries. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Config is a sync write of github.com/org/ (or a single repo URL). The indexer lists org repos at harvest time and classifies each as OpenCRE or OWASP-agent material. Add org no longer calls GitHub. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Drop the dedicated Admin tab. Resources lists a virtual owasp-agent row with status in spec; Start now runs the same harvest as oie_repo. The built-in id is reserved, skipped on yaml seed, and cannot be removed unless a leftover DB row exists. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Save, Add org, and one-off ingest probe api.github.com for each sources URL before writing or starting harvest. Missing, private, or timed-out orgs/repos return 400 with the canonical URL. The indexer still expands reachable orgs later. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Admin home lists running jobs and links through to pipeline logs. Config edits targets.yaml (cron, add-org, add-target form) plus every .env.example env var with secrets redacted. Pipeline New can harvest the packaged yaml or a one-off repository. Graph management reviews per-link approve/deny/relink for each run. MyOpenCRE stays a separate route. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Add org now writes packaged repos.yaml after the GitHub probe, matching Add target. Graph management notes that link decisions stay on the staged run until CRE-edge apply lands. Failed applies no longer look complete. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Remove the separate agent DB env var. Admin status and config read DATABASE_URL / DEV_DATABASE_URL (same Postgres as CRE + pgvector). Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Admin invoked python /abs/scripts/run_oie_pipeline.py, so sys.path[0] was scripts/ and `import application` failed. Other threads set PYTHONPATH= at the repo root. Put the repo on sys.path in the script and in the admin subprocess env. IndexStore now defaults to DATABASE_URL / DEV_DATABASE_URL instead of tmp/owasp_agent.sqlite. OWASP_AGENT_DB remains a test override. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

SQLAlchemy 2.1 maps postgresql:// to psycopg v3; pin psycopg2 via postgresql+psycopg2://. Admin invoke_oie_cli uses the app Postgres URL and refuses sqlite. `make admin-local` starts cre-postgres when the URL is down, migrates, create_all, and launches Flask without CRE_CACHE_FILE. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Production and admin/agent now share main_db_env_keys (DATABASE_URL before PROD_DATABASE_URL). Rewrite postgresql+psycopg:// to +psycopg2. Gate local create_all behind ADMIN_LOCAL_CREATE_ALL. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Live defaults sync repos and persist to Postgres; auto-skip B/C without LLM keys; async API with wait=true for sync probes. Treat harvester degraded as non-fatal and surface chunks/files in stage detail. Pass --max-repos through the CLI so admin caps stay effective. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Empty locale stubs in Top10 aborted the whole repo harvest; skip and continue so golden-set ingest can finish. Admin agent status now counts all public tables and reports MAX(fetched_at) as last_sync. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Map only Module C safety-path degraded as non-fatal so A/B degraded halts before B/C again. Reject max_repos < 1. Always skip B/C unless the client opts in so LLM keys cannot launch expensive stages from New. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Admin /rest JSON errors no longer collapse to HTML "Resource Not found". List backfills missing staged_change_set rows. Graph UI shows Accept feedback, disables dead actions, and explains empty CRE changesets on OIE harvest runs (ops live in harvest_input). Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Orchestrator JSON now includes engine, flags, graph_path, and visited. Skip stage details explain caller flags; harvester detail notes HEAD checkpoints when chunks=0. Admin records oie_trace and richer started JSON. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

OIE harvest fills harvest_input but not the explorer graph; run cre.py --upstream_sync before Flask unless the cre/node tables already have data (SKIP_UPSTREAM_SYNC / FORCE_UPSTREAM_SYNC overrides). Co-authored-by: Spyros <northdpole@users.noreply.github.com>

make install now migrates Docker Postgres and upstream-syncs an empty CRE graph; make dev / dev-flask / admin-local share run_admin_local.sh. install-python falls back to python3 -m venv; clear PORT=5001 hint for macOS AirPlay on :5000. Co-authored-by: Spyros <northdpole@users.noreply.github.com>

Add POST /admin/agent/sync (Pipeline events) and Config buttons to run packaged golden-set OIE harvest and agent metadata sync end-to-end without re-running upstream CRE import at launch. Co-authored-by: Spyros <northdpole@users.noreply.github.com>

Open questions first, then run instructions, APIs, and stack context so the full pipeline loop can continue on a local machine. Co-authored-by: Spyros <northdpole@users.noreply.github.com>

Both migrations descended from 967016ee10fa, leaving two alembic heads and breaking 'flask db upgrade'. The admin PR is stacked on the scheduler PR, so its revision now follows b8c9d0e1f2a3.

Packaged harvest is the golden-set list, the Config button sends max_repos=50 so the 21-repo set is not capped at 5, and the CLI timeout covers a full A→B→C run.

Automatically linked to edges and oie-auto-filed nodes use a rose accent so the stage demo can tell AI filings from curated Linked to links.

After the librarian, file high-confidence linked decisions as Automatically linked to edges (skip with --skip-c1); admin default path includes C.1.

OIE_GRAPH_FILING_ENABLED defaults to false so decision_queue stays reviewable; turn the Config toggle on to auto-add high-confidence Automatically linked to edges.

northdpole requested review from Pa04rth, paoga87 and robvanderveer as code owners October 7, 2026 07:52

northdpole mentioned this pull request Oct 9, 2026

feat(oie): RQ A→B→C fan-out and Admin OIE queue tab #1142

Open

4 tasks

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 5236f80 to 061f632 Compare October 11, 2026 23:04

northdpole force-pushed the admin-panel-showcase branch from e114292 to 4178ecf Compare October 11, 2026 23:04

northdpole mentioned this pull request Oct 11, 2026

Fix OWASP agent probe gaps (chapters, people, membership, CRE facts) #1127

Open

4 tasks

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 061f632 to 719716d Compare October 12, 2026 07:05

northdpole force-pushed the admin-panel-showcase branch from 4178ecf to aad929a Compare October 12, 2026 07:05

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 719716d to ae89728 Compare October 12, 2026 07:15

northdpole force-pushed the admin-panel-showcase branch from aad929a to 475945f Compare October 12, 2026 07:17

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from ae89728 to 6e444d3 Compare October 12, 2026 07:18

northdpole force-pushed the admin-panel-showcase branch 2 times, most recently from 48e9d4f to 30a2a40 Compare October 12, 2026 22:19

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 6e444d3 to 05bd3bd Compare October 14, 2026 08:49

northdpole force-pushed the admin-panel-showcase branch from 30a2a40 to 63d548b Compare October 14, 2026 08:49

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 05bd3bd to 1605bf9 Compare October 14, 2026 17:02

northdpole force-pushed the admin-panel-showcase branch from 63d548b to 864d6e3 Compare October 14, 2026 17:09

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch 2 times, most recently from 2c4484a to eb8c958 Compare October 14, 2026 17:21

northdpole force-pushed the admin-panel-showcase branch from 864d6e3 to bdcb343 Compare October 14, 2026 17:22

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from eb8c958 to 47b36e8 Compare October 14, 2026 17:23

northdpole force-pushed the admin-panel-showcase branch from bdcb343 to 62832de Compare October 14, 2026 17:23

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 47b36e8 to c179db0 Compare October 14, 2026 17:55

northdpole force-pushed the admin-panel-showcase branch from 62832de to e41d93b Compare October 14, 2026 17:56

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from c179db0 to 3772492 Compare October 15, 2026 00:32

northdpole force-pushed the admin-panel-showcase branch from e41d93b to a948365 Compare October 15, 2026 00:32

northdpole force-pushed the cursor/oie-scheduled-runs-eb24 branch from 7ac6e1c to 2782304 Compare October 15, 2026 12:16

northdpole force-pushed the admin-panel-showcase branch from a948365 to 3c45bf3 Compare October 15, 2026 12:16

northdpole and others added 29 commits October 15, 2026 18:43

Harden repos.yaml org expand and yaml size limits.

84e9839

Allowlist GitHub owner logins, quote the API path, cap yaml at 256KiB before safe_load, and write packaged repos.yaml atomically. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Rebuild frontend bundle for repos.yaml admin editor.

3b58d49

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Raise repos.yaml admin cap to 4MiB for org expands.

c1bf1b5

The 256KiB parse limit rejected a full GitHub org dump. Keep a size cap, but large enough for up to 1000 repository entries. Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Rebuild frontend bundle for the admin IA redesign.

bccae60

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Drop OWASP_AGENT_DB; agent uses the main app Postgres.

2f0da3f

Remove the separate agent DB env var. Admin status and config read DATABASE_URL / DEV_DATABASE_URL (same Postgres as CRE + pgvector). Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Add Mac handover for admin/OIE/agent E2E validation.

45eb358

Open questions first, then run instructions, APIs, and stack context so the full pipeline loop can continue on a local machine. Co-authored-by: Spyros <northdpole@users.noreply.github.com>

Chain admin panel tables migration after oie_run

afb0948

Both migrations descended from 967016ee10fa, leaving two alembic heads and breaking 'flask db upgrade'. The admin PR is stacked on the scheduler PR, so its revision now follows b8c9d0e1f2a3.

Black-format admin_imports_api_test for CI

aabb90b

Run admin OIE ingest through Modules B and C by default.

7871306

Packaged harvest is the golden-set list, the Config button sends max_repos=50 so the 21-repo set is not capped at 5, and the CLI timeout covers a full A→B→C run.

Paint Module C auto-filed graph edges and nodes in the explorer.

bfcb1e4

Automatically linked to edges and oie-auto-filed nodes use a rose accent so the stage demo can tell AI filings from curated Linked to links.

Add Module C.1 graph filing to the OIE A→B→C pipeline.

0f22476

After the librarian, file high-confidence linked decisions as Automatically linked to edges (skip with --skip-c1); admin default path includes C.1.

Default Module C.1 auto-filing off behind an admin Config toggle.

7b9b007

OIE_GRAPH_FILING_ENABLED defaults to false so decision_queue stays reviewable; turn the Config toggle on to auto-add high-confidence Automatically linked to edges.

Restore clarify followup test dropped in admin rebase.

18ed280

northdpole force-pushed the admin-panel-showcase branch from 023a4be to 18ed280 Compare October 15, 2026 17:43

Navigation Menu

Repository navigation

Admin showcase panel for signed-in users - #1137

Admin showcase panel for signed-in users#1137
northdpole wants to merge 48 commits into
cursor/oie-scheduled-runs-eb24from
admin-panel-showcase

northdpole commented Oct 7, 2026

Uh oh!

coderabbitai Bot commented Oct 7, 2026 •

edited

Loading

Review skipped

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

northdpole commented Oct 7, 2026

Mac handover

What changed

Uh oh!

coderabbitai Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Oct 7, 2026 •

edited

Loading