Core Evaluation Platform

Core engine for a modular competition-evaluation platform. Users submit tools and benchmarks; the code runs on a provisioned worker, and results are collected, scored, and displayed. Django REST backend + React/TypeScript (Vite) frontend.

This is the core component powering the evaluation platforms of VNN-COMP and ARCH-COMP.

Individual competitions are plugins in their own repos (vnn-eval-platform, arch-eval-platform, …).

Getting started

Clone either variant and follow the Getting Started guide there.

If you want to use this core component for a new submission platform, it also might be best to check out how this was done for each of the existing variants. Don't hesitate to reach out for any questions regarding this!

Requirements

• Docker + Docker Compose (Docker Desktop on macOS/Windows). The dev backend mounts the host Docker socket to run worker containers.
• Git.

Tests

Frontend tests cover shared status/result formatting, API list handling, authentication state, protected-route access, and toolkit and benchmark submission workflows:

cd frontend
npm ci
npm run check

Use npm run lint to check code without changing it, or npm run lint:fix to apply ESLint's safe automatic fixes. npm run check matches the frontend CI sequence: lint, test, and production build.

npm ci also installs a pre-commit hook. It runs ESLint with safe fixes on staged TypeScript files and blocks the commit if violations remain. GitHub Actions still checks the complete frontend after every push and pull request update.

Backend tests can be run with:

docker run --rm -v "$PWD:/core" -w /core python:3.11-slim \
  sh -c "pip install -q -e '.[dev]' && pytest"

Continuous integration

The GitHub Actions workflow in .github/workflows/ci.yml runs for pushes to every branch, pull requests, and manual dispatches. It runs the core Python tests and tests and builds the shared frontend.

Documentation

Current architecture and operations documentation starts at docs/index.md.