Admin showcase panel for signed-in users

.cursor/skills/brevity-bot/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: brevity-bot
+description: >-
+  Review a diff like a long-tenured greybeard coworker: shorten AI blurbs,
+  simplify code, keep correctness/security/Bugbot fixes. Use after Bugbot and
+  Security reviews, or when asked for /review-brevity.
+---
+
+# Brevity bot
+
+You are the ponytailed, greybearded, Linux-using god of coworker who has been
+with the company since the beginning. You can shorten or simplify any codebase.
+Your job is to dramatically increase readability and quality and shorten AI
+blurbs while preserving correctness, quality, and security.
+
+You simplify code **without dropping the fixes** from Bugbot or Security
+review. If a check, error path, redaction, auth gate, or test exists because
+those reviews required it, keep it. Prefer a shorter form of the same fix.
+
+## Diff contract (same as Bugbot)
+
+The parent gives:
+
+```text
+Full Repository Path: <absolute path>
+Diff: <branch changes | uncommitted changes>
+```
+
+Review that diff only. Do not invent drive-by refactors outside it.
+
+## What to cut
+
+- Restated comments, essay docstrings, "as an AI" padding, changelog comments
+- Needless wrappers, duplicate branches, unused helpers, speculative abstraction
+- Tests that only restate the same assertion five ways
+
+## What to keep
+
+- Auth, allowlists, redaction, abort codes, CSRF/login gates
+- Bugbot/Security fixes (narrower code is fine; weaker checks are not)
+- Names and structure that match this repo (`application/tests/`, Makefile)
+
+## Output
+
+Either:
+
+1. **Patches** — file path, short why, and the smaller replacement, or
+2. **`NO_COMMENTS`** — nothing left to shorten without harming clarity, tests,
+   or security.
+
+End with exactly one of:
+
+- `VERDICT: NO_COMMENTS`
+- `VERDICT: COMMENTS` (and the patch list)
+
+Do not praise the diff. Do not recap the plan.

.cursor/skills/brevity-bot/examples.md

@@ -0,0 +1,40 @@
+# Brevity bot fixtures
+
+## AI blurb → short
+
+Before:
+
+```python
+def is_enabled() -> bool:
+    """Return True if the feature flag is enabled.
+
+    This function checks the environment variable and compares it against
+    a set of truthy values so that operators can enable the feature.
+    """
+    value = os.getenv("FLAG", "")
+    normalized = value.strip().lower()
+    if normalized in {"1", "true", "yes"}:
+        return True
+    return False
+```
+
+After:
+
+```python
+def is_enabled() -> bool:
+    return os.getenv("FLAG", "").strip().lower() in {"1", "true", "yes"}
+```
+
+## Do not drop a security fix
+
+Security required redacting secrets. Illegal brevity (drops the fix):
+
+```python
+return {"GEMINI_API_KEY": os.environ.get("GEMINI_API_KEY")}
+```
+
+Legal brevity (keeps redaction):
+
+```python
+return {"GEMINI_API_KEY": "***"} if os.environ.get("GEMINI_API_KEY") else {}
+```

.cursor/skills/review-brevity/SKILL.md

@@ -0,0 +1,24 @@
+---
+name: review-brevity
+description: Launch Brevity bot on the current diff after Bugbot and Security.
+---
+
+# Review Brevity
+
+When asked to run `/review-brevity` or Brevity bot:
+
+Launch **three** independent `generalPurpose` subagents (`run_in_background: false`)
+with the contents of `.cursor/skills/brevity-bot/SKILL.md` as instructions.
+
+Prompt each with:
+
+```text
+Full Repository Path: <absolute repository path>
+Diff: branch changes
+```
+
+Default to `branch changes`. Use `uncommitted changes` only if asked.
+
+Collect the three `VERDICT:` lines. Majority `NO_COMMENTS` (2 of 3) is clean.
+If majority is `COMMENTS`, apply agreed simplifications that do **not** revert
+Bugbot or Security fixes, then re-run.

.env.example

@@ -5,6 +5,10 @@
 # Never commit your real `.env` file to version control.
 # ------------------------------------------------------------------------------
 # Database
+# Local Docker (make docker-postgres / make admin-local):
+#   postgresql://cre:password@127.0.0.1:5432/cre
+# `make admin-local` starts that container when the URL is unreachable and
+# never falls back to SQLite for the admin showcase.
 
 DEV_DATABASE_URL=postgresql://user:password@localhost:5432/opencre
 
@@ -42,14 +46,12 @@ CRE_ENABLE_MYOPENCRE=false
 CRE_ENABLE_LOGIN=false
 
 # Experimental OIE OWASP metadata agent (chat + MCP only — never CRE graph UI).
-# Off by default. When enabled, community/meta questions are answered from
-# the local owasp_agent index instead of CRE RAG. Concepts/tags stay in
-# OWASP_AGENT_DB and are reachable only from /completion and MCP owasp_meta_* tools.
+# Off by default. Uses the main app Postgres (DEV_DATABASE_URL / DATABASE_URL)
+# for the agent index + pgvector. Reachable from /completion and MCP owasp_meta_* tools.
 OWASP_AGENT_ENABLED=false
 # Nest REST API key (https://nest.owasp.org — create via GraphQL while logged in)
 NEST_API_KEY=
 NEST_API_BASE=https://nest.owasp.org/api/v0
-OWASP_AGENT_DB=tmp/owasp_agent.sqlite
 # Set to 1 to crawl all www-chapter / www-project repos during sync (slow).
 OWASP_AGENT_CRAWL_REPOS=0
 

.gitignore

@@ -49,6 +49,8 @@ v/
 .cursor/*
 !.cursor/rules/
 !.cursor/rules/**
+!.cursor/skills/
+!.cursor/skills/**
 .claude/
 
 ### Frontend 
@@ -67,6 +69,7 @@ standards_cache.sqlite
 ### Docs
 *.md
 !AGENTS.md
+!.cursor/skills/**/*.md
 !docs/faq.md
 !docs/api/mcp.md
 !docs/Mid_eval_blog_gsoc2026/module_B_mideval_blog.md
@@ -94,6 +97,8 @@ neo4j/
 .mypy_cache
 tmp/
 
+import_telemetry.json
+
 ### CREs dir
 cres/*
 ### Local project management tooling

Makefile

@@ -1,6 +1,6 @@
 .ONESHELL:
 
-.PHONY: run test covers install-deps dev docker lint frontend clean all e2e e2e-db owasp-agent-sync owasp-agent-merge-concepts owasp-agent-synth-eval
+.PHONY: run test covers install-deps dev dev-flask docker lint frontend clean all e2e e2e-db admin-local owasp-agent-sync owasp-agent-merge-concepts owasp-agent-synth-eval
 
 prod-run:
 	gunicorn cre:app --log-file=-
@@ -74,11 +74,19 @@ start-worker:
 	. ./venv/bin/activate && FLASK_APP=`pwd`/cre.py python cre.py --start_worker
 
 upstream-sync:
-	. ./venv/bin/activate && python cre.py --upstream_sync
+	. ./venv/bin/activate &&\
+	python cre.py --upstream_sync --cache_file "$(or $(DEV_DATABASE_URL),$(CRE_CACHE_FILE),$(CURDIR)/standards_cache.sqlite)"
 
 PORT?=5000
 
-dev-flask:
+# Local app against Docker Postgres: migrate, upstream-sync if cre/node empty, Flask.
+# Same entrypoint as admin showcase / OIE local (no SQLite). macOS AirPlay often
+# holds :5000 — use PORT=5001 make dev if bind fails.
+dev dev-flask admin-local:
+	bash ./scripts/run_admin_local.sh --port $(PORT) $(ADMIN_LOCAL_ARGS)
+
+# Bare Flask without Postgres bootstrap (legacy / CI helpers). Prefer `make dev`.
+dev-flask-sqlite:
 	. ./venv/bin/activate && INSECURE_REQUESTS=1 FLASK_APP=`pwd`/cre.py  FLASK_CONFIG=development flask run --port $(PORT)
 
 dev-flask-docker:
@@ -136,7 +144,14 @@ install-deps-typescript:
 install-deps: install-deps-python install-deps-typescript
 
 install-python:
-	virtualenv -p python3 venv
+	@if [ ! -d "./venv" ]; then \
+		if command -v virtualenv >/dev/null 2>&1; then \
+			virtualenv -p python3 venv; \
+		else \
+			echo "virtualenv not on PATH — using python3 -m venv"; \
+			python3 -m venv venv; \
+		fi; \
+	fi
 	. ./venv/bin/activate &&\
 	make install-deps-python &&\
 	playwright install  # Python embeddings/scraping (prompt_client); NOT frontend e2e — keep when migrating to Cypress
@@ -148,7 +163,9 @@ install-python:
 # `make install`. Rebuild with `make frontend` when you change the UI.
 install-typescript: install-deps-typescript
 
-install: install-typescript install-python migrate-upgrade
+# Deps + Docker Postgres schema + CRE upstream graph when empty (see run_admin_local.sh).
+install: install-typescript install-python
+	bash ./scripts/run_admin_local.sh --migrate-only $(ADMIN_LOCAL_ARGS)
 
 docker-dev:
 	docker build -f Dockerfile-dev -t opencre-dev:$(shell git rev-parse HEAD) .

README.md

@@ -49,44 +49,33 @@ If you want to develop on OpenCRE or docker is not available in your environment
 ### Command Line
 
 To run outside of Docker you need to install OpenCRE.
-To install this application you need Python 3.11.9, Node.js, Yarn Classic (the repository uses a Yarn v1 lockfile), and `virtualenv`. The Python version is pinned in `.python-version`; the repository does not currently pin a Node.js version.
+To install this application you need Python 3.11.9, Node.js, Yarn Classic (the repository uses a Yarn v1 lockfile), and Docker (local Postgres). `virtualenv` is optional — `make install-python` falls back to `python3 -m venv`. The Python version is pinned in `.python-version`; the repository does not currently pin a Node.js version.
 
 Clone the repository:
 
 ```bash
 git clone https://github.com/OWASP/OpenCRE
 ```
 
-`make install` creates and uses the project virtual environment. Do not create a second environment with `python3 -m venv`; install the prerequisites above, then let the Makefile manage `venv`.
-
-Install dependencies:
+`make install` creates the project `venv` (if missing), installs deps, starts Docker Postgres when needed, migrates, and pulls the CRE graph from opencre.org when the local DB is empty.
 
 ```bash
 make install
+make dev                 # same as make dev-flask / make admin-local
 ```
 
-`make install` creates the local schema as part of installation. Download the latest CRE graph from upstream with:
-
-```bash
-make upstream-sync
-```
+Then open http://127.0.0.1:5000/admin and http://127.0.0.1:5000/explorer. If port 5000 is busy (common on macOS with AirPlay Receiver), use `PORT=5001 make dev`. To reuse an existing DB without re-pulling the CRE graph: `SKIP_UPSTREAM_SYNC=1 PORT=5001 make dev`.
 
 For contributors, this is the supported local data bootstrap path. You do
 **not** need access to the internal OpenCRE Google Sheet to work on the
-project.
+project. From **Admin → Config**, use **Run golden-set harvest** and **Sync OWASP agent** (progress under Pipeline), or `make owasp-agent-sync` from the CLI.
 
 You can precompute local gap-analysis cache after imports with:
 
 ```bash
 make backfill-gap-analysis
 ```
 
-To run CRE locally then you can do:
-
-```bash
-make dev-flask
-```
-
 To run the CLI application, you can run:
 
 ```bash

application/config.py

@@ -4,9 +4,30 @@
 
 import os
 
+from application.utils.postgres_url import (
+    is_postgres_url,
+    main_db_env_keys,
+    sqlalchemy_postgres_url,
+)
+
 basedir = os.path.abspath(os.path.dirname(__file__))
 
 
+def _sqlalchemy_uri_from_env(raw: str) -> str:
+    """Bind Postgres env URLs to psycopg2; leave sqlite and other schemes alone."""
+    if is_postgres_url(raw):
+        return sqlalchemy_postgres_url(raw)
+    return raw
+
+
+def _first_env_url(*keys: str) -> str:
+    for key in keys:
+        raw = (os.environ.get(key) or "").strip()
+        if raw:
+            return raw
+    return ""
+
+
 class Config:
     SQLALCHEMY_TRACK_MODIFICATIONS = False
     SQLALCHEMY_RECORD_QUERIES = False
@@ -18,8 +39,8 @@ class DevelopmentConfig(Config):
     DEBUG = True
     ENVIRONMENT = "DEVELOPMENT"
     CACHE_TYPE = "SimpleCache"
-    SQLALCHEMY_DATABASE_URI = (
-        os.environ.get("DEV_DATABASE_URL")
+    SQLALCHEMY_DATABASE_URI = _sqlalchemy_uri_from_env(
+        _first_env_url(*main_db_env_keys("development"))
         or f'sqlite:///{os.path.join(basedir, "../standards_cache.sqlite")}'
     )
 
@@ -35,8 +56,8 @@ class ProductionConfig(Config):
     ENVIRONMENT = "PRODUCTION"
     CACHE_TYPE = "SimpleCache"
     CACHE_DEFAULT_TIMEOUT = 3000
-    SQLALCHEMY_DATABASE_URI = (
-        os.environ.get("PROD_DATABASE_URL")
+    SQLALCHEMY_DATABASE_URI = _sqlalchemy_uri_from_env(
+        _first_env_url(*main_db_env_keys("production"))
         or f'sqlite:///{os.path.join(basedir, "../standards_cache.sqlite")}'
     )
 
@@ -46,11 +67,12 @@ class CMDConfig(Config):
 
     def __init__(self, db_uri: str):
         if "://" in db_uri:
-            # Heroku and some tools still emit ``postgres://``; SQLAlchemy 2 expects
-            # ``postgresql://`` for the psycopg dialect name.
-            if db_uri.startswith("postgres://"):
-                db_uri = "postgresql://" + db_uri[len("postgres://") :]
-            self.SQLALCHEMY_DATABASE_URI = db_uri
+            # Heroku still emits ``postgres://``; SQLAlchemy 2.1 maps bare
+            # ``postgresql://`` to psycopg v3 — pin psycopg2 (requirements.txt).
+            if is_postgres_url(db_uri):
+                self.SQLALCHEMY_DATABASE_URI = sqlalchemy_postgres_url(db_uri)
+            else:
+                self.SQLALCHEMY_DATABASE_URI = db_uri
         else:
             # Flask-SQLAlchemy 3+ resolves non-absolute sqlite URLs against
             # app.instance_path, not the shell cwd. CLI --cache_file should