A retrospective analysis tool for auditing police enforcement patterns using open public datasets from San Francisco. Built for civic oversight analysts, investigative journalists, and academic researchers.
The thesis: police incident records do not capture crime — they capture police contact. A heavily-patrolled neighborhood generates more reports, which makes it look like a hotspot, which has historically justified more patrol. Lens makes that feedback loop visible. It normalizes the same incident data three ways (raw count, per-capita using ACS census population, and officer enforcement ratio) so the same data tells three different stories — and surfaces the differences explicitly rather than hiding them.
What this looks like in practice: after Mayor Lurie took office in January 2025, South of Market's officer enforcement ratio rose +79 points compared to the prior period — roughly 4–6× its historical year-over-year baseline. Mission rose +43 points (5× baseline). Tenderloin, by contrast, was lower than its historical average, suggesting a geographic reallocation of enforcement rather than a citywide increase. Lens surfaces this finding in two clicks: select the Lurie preset, read the ranked list.
The same tool applied to the 2026 World Cup shows which neighborhoods are seeing enforcement shift during the event window — a live example of the same analysis applied to a current policy question.
What's next (v2.0): cross-references enforcement data with auxiliary sources — 311 service requests (unmet civic need), business density (explains raw counts in commercial areas), building violations (housing disinvestment), and public health indicators — to further separate genuine need from patrol intensity. An analyst will walk away with a finding like: "District X has 2.1× the citywide discretionary enforcement rate but 0.7× the victim-reported serious crime rate, and burglaries there resolve in a recorded arrest about half as often as the citywide rate."
Lens 1 — Per-capita enforcement density Incident counts normalized by ACS census population per neighborhood. Strips out the trivial fact that more people generate more of everything.
Lens 2 — Officer enforcement ratio Officer-initiated incidents per 100 victim-reported serious crimes, per neighborhood. Separates patrol behavior from underlying crime. Validated against 50+ crime categories using at-filing arrest rates and Coplogic shares to classify officer-initiated vs. victim-reported.
Compare mode Side-by-side Lens 2 view across two user-defined date windows. Surfaces enforcement shifts tied to policy events. Ships with two presets: Mayor Lurie's anti-homelessness campaign (Jan–Sep 2025 vs. the prior 9 months) and the 2026 World Cup event window.
Data ~746,000 deduplicated SF incidents, Jan 2018 – July 2026, loaded from DataSF via Socrata. Raw dataset is ~1M report rows; deduplicated to unique incidents before analysis.
What's not in v1.0 Lens 3 (resolution gap — clearance rate by crime type vs. citywide median) is built and validated but held pending assault-category surgery. It will ship in v2.0.
Lens 1 (per-capita) — South of Market, 2018–2026
Lens 2 (officer enforcement ratio) — South of Market in compare mode: first 9 months of Mayor Lurie's anti-homelessness campaign vs. the 9 months directly preceding
| Layer | Technology |
|---|---|
| Backend | Python / FastAPI |
| Database | PostgreSQL + PostGIS |
| Migrations | Alembic |
| Frontend | Next.js + TypeScript |
| Map | Leaflet / react-leaflet |
| Charts | Recharts |
| Data pipeline | Python / Pandas / GeoPandas |
| Dev environment | Docker Compose |
Prerequisites: Docker and Docker Compose.
docker compose upThe frontend will be available at http://localhost:3000. The backend API will be available at http://localhost:8000. Migrations run automatically on backend startup — no manual step required.
docker compose up db backend# API health
curl http://localhost:8000/health
# Lens 1 data (should return 41 neighborhoods)
curl "http://localhost:8000/lens/1?start=2024-01-01&end=2025-01-01"
# PostGIS extension
docker compose exec db psql -U lens -d lens -c "SELECT PostGIS_Version();"The full docker compose up runs a production build — every CSS or component change requires a full rebuild. For active frontend work, run the frontend outside Docker so changes reload instantly:
# Terminal 1 — DB + backend only
docker compose up db backend
# Terminal 2 — frontend dev server (hot reload)
cd frontend
npm run devTo avoid setting the env var every time, create frontend/.env.local once:
NEXT_PUBLIC_API_TARGET=http://localhost:8000
Switching back to full Docker (e.g. to test the production build or share via ngrok):
# Stop just the frontend dev server (Ctrl+C in its terminal), then:
docker compose up --build frontendSwitching back to local dev from full Docker:
docker compose stop frontend # DB and backend keep running
cd frontend && npm run dev# Backend
pip install -r backend/requirements.txt
DATABASE_URL=postgresql://lens:lens@localhost:5432/lens uvicorn app.main:app --reload
# Frontend
cd frontend
npm install
npm run dev# Backend + pipeline
pytest
# Frontend
cd frontend && npm testlens/
├── backend/
│ ├── app/
│ │ ├── api/ # route handlers: lens, compare, neighborhoods
│ │ └── main.py # FastAPI app entry point
│ ├── alembic/ # migrations — never hand-edit schema
│ │ └── versions/
│ ├── tests/ # API endpoint + lens calculation tests
│ └── requirements.txt
├── pipeline/
│ ├── adapters/
│ │ └── sf/ # SF-specific field mappings and ingest script
│ ├── analysis/ # exploratory scripts used to generate spike findings
│ ├── sources/ # Socrata API client
│ └── tests/ # transform unit tests
├── frontend/
│ ├── app/ # page.tsx — main layout and data fetching
│ ├── components/ # Map, Controls, LensPanel, NeighborhoodPanel, RankingsPanel
│ ├── lib/ # API helpers, preset events
│ └── types/ # TypeScript types
├── docs/
│ ├── adr/ # architecture decision records
│ ├── spikes/ # spike findings: what we learned, therefore what we did
│ └── methodology.md # lens definitions, flag definitions, denominators, limitations
├── docker-compose.yml
└── pytest.ini
The app runs on a shared instance — any teammate can access real SF data without cloning the repo or running anything locally.
URL: https://parky-efren-nondynastically.ngrok-free.app
Credentials: none required to view the app. The database uses local dev credentials (lens / lens) and is never exposed outside the host machine.
docker compose up on the host machine starts the database, backend, and frontend in the correct order, verified by health checks.
Migrations run automatically. The backend startup command chains alembic upgrade head && uvicorn — every time the container starts, any pending migrations are applied before the server accepts traffic.
The frontend runs a production build (next build + next start) rather than the dev server, avoiding the HMR/hydration issues that come with sharing a next dev instance.
The URL is served via ngrok rather than a bare local IP. A local IP only works on the same network, and campus eduroam blocks device-to-device traffic via client isolation. ngrok exposes the instance over a public HTTPS URL that works from anywhere.