Skip to content

Repository files navigation

📚 Atlas

A self-collecting, source-grounded, learn-anything study engine. Bring a PDF textbook or YouTube series → highlight while reading → get grounded answers → build cards → review them on a spaced-repetition schedule.

status license llm storage rag


Architecture (ingest → RAG → cards → SM-2)

flowchart LR
  subgraph ingest [Ingest]
    PDF[PDF]
    YT[YouTube]
    URL[URL / Wiki / HN]
  end

  subgraph store [Local-first store]
    SQ[(SQLite)]
    Emb[Embeddings<br/>nomic-embed-text]
    Jobs[Job queue]
  end

  subgraph study [Study loop]
    HL[Highlights]
    RAG[RAG retrieve]
    Card[Card templates]
    SM2[SM-2 review]
  end

  PDF & YT & URL --> Jobs --> SQ
  SQ --> Emb
  HL --> Card
  Emb --> RAG --> Card --> SM2
Loading

Key design decisions:

  • SQLite-first — single-user local study doesn't need a cloud DB; lower ops cost, portable backup. Postgres is optional for the job queue when you want concurrent workers.
  • Highlight → card, not chat → forget — NotebookLM optimizes for Q&A; Atlas optimizes for long-term retention via SM-2. Trade-off: more structured UX, less free-form chat.
  • Pluggable LLMs — teacher / syllabus / card / research can each use a different provider. Trade-off: more config surface; win is cost control (local Ollama for embeddings, paid model for card quality).

🆚 How is Atlas different from NotebookLM?

Google's NotebookLM is a great product. Atlas is not a clone — it's a different tool for a different workflow.

Dimension NotebookLM (Google) Atlas
Primary loop Upload sources → chat with them → generate summaries Read sources → highlightturn highlights into study cardsreview on a spaced-repetition schedule
End goal Understand a document set Remember a domain long-term (interviews, a new job, a textbook)
Source discovery You upload everything by hand Self-collecting — Atlas builds search queries from your topic and scrapes Wikipedia, YouTube, Reddit, arXiv, HN, GitHub, StackExchange in parallel
Spaced repetition ❌ None — it's a chat tool SM-2 (the algorithm Anki uses) with muscle / instinct / struggled / forgot ratings
Card quality N/A (no cards) Every card is validated (core ≥ 80 chars, < 50% empty fields, no near-duplicate hooks). Degraded cards are honestly badged ⚠ Low quality — ↺ regenerate.
Card templates N/A 4 templates: interview (STAR war story + anti-pattern), concept, skill, history
RAG Yes (proprietary) Yes (open, local) — nomic-embed-text (768-dim), brute-force cosine over Float32Array, page-cited context
PDF reading Yes Yes — pdf.js with page-aware highlights that persist with bounding rects and re-render on scroll/zoom
Export Limited JSON backup, Markdown (Obsidian-friendly), CSV, Anki TSV (File → Import in Anki desktop)
LLM Gemini only Ollama, OpenAI, Anthropic, Gemini — per-feature defaults (teacher / syllabus / card / research / critique / embed can each use a different model)
Where it runs Google's cloud Your machine — local-first, single-user, no telemetry
Job queue / observability Hidden Exposed — Postgres-backed job queue with a live Jobs dashboard, event timeline, retry/cancel, stuck-job reaper
Price Free (Google account) Free (your compute)

One-line summary

NotebookLM answers questions about your sources. Atlas turns your sources into a study deck you'll still be reviewing next month.


🎯 Real use-cases

1. "I have a Staff Engineer system-design interview in 4 weeks"

  • Create a roadmap: "Distributed messaging with Apache Kafka", 14 days, advanced, interview template.
  • Atlas generates a syllabus (4–8 topics with outcomes, depth, est. minutes, prerequisites) and auto-collects sources for each topic from Wikipedia, YouTube, arXiv, HN, GitHub, StackExchange.
  • You read the Wikipedia article on Apache Kafka, highlight the line about Kafka being a distributed log, label it #concept.
  • You click Make card → Atlas generates a card with hook, core, analogy, trade-offs, STAR war story, and anti-pattern (because you're on the interview template).
  • The card queues for review tomorrow. SM-2 schedules it. You review 3 cards a day for 14 days. You walk into the interview having actually internalized the material.

2. "I'm watching a 12-hour YouTube course on distributed systems but I retain nothing"

  • Paste each video URL as a source. Atlas pulls the transcript via yt-dlp.
  • Highlight the parts that matter. Ask @teacher follow-up questions grounded in the transcript (not the first 2000 chars — Atlas retrieves the top-k chunks by embedding similarity and feeds them to the LLM with timestamps).
  • Turn the best highlights into cards. Review them on SM-2. By video 12, you remember video 1.

3. "I'm reading the Designing Data-Intensive Applications PDF"

  • Upload the PDF. Atlas extracts text via pdftotext and persists the binary so the page viewer renders real scrollable pages.
  • Highlight on page 47. The highlight stores its page number + bounding rects, so it re-renders correctly when you zoom or scroll back.
  • Ask @teacher: "Why does the author say eventual consistency is a usability problem?" — Atlas retrieves the chunks around your highlight + the most similar chunks elsewhere in the book, formats them with [p.47] citations, and the LLM answers with page references.
  • Export to Markdown → drop into Obsidian. Export to Anki TSV → import as a deck.

4. "I'm a hiring manager and I want to build an onboarding curriculum for new hires"

  • Create a roadmap: "Our internal platform — service mesh, deploy pipeline, on-call", 30 days, intermediate, concept template.
  • Paste internal docs / runbooks as sources. Atlas chunks and embeds them.
  • Generate cards. Export to JSON. Hand each new hire the JSON — they import it and start reviewing.

📸 What it looks like

Roadmaps dashboard

Each roadmap is a structured curriculum. Create one with a learning goal, duration, level, and template.

Roadmaps dashboard

Roadmap detail — topics, sources, cards

A roadmap breaks into topics. Each topic has sources (auto-collected or manual) and cards. Buttons to regenerate the syllabus, run research, review due cards, and export.

Roadmap detail

Source reader — highlight, ask @teacher, make cards

Read a source. Highlight text (yellow). Each highlight becomes a sidebar entry you can label, ask @teacher about, or turn into a card. @teacher answers are grounded in retrieved chunks with page citations.

Source reader

Review session — SM-2 spaced repetition

One card at a time. Tap to reveal. Rate your recall: muscle / instinct / struggled / forgot. SM-2 schedules the next review.

Review session

Cards view — with honest quality badges

Cards are listed per roadmap. Duplicates and shallow cards are badged ⚠ Low quality — ↺ regenerate. Never silently passed off as good.

Cards view

Jobs dashboard — Postgres-backed observability

Live view of LLM + research jobs. Stuck/failed jobs surface here with full event timelines. Retry, cancel, watch live via SSE.

Jobs dashboard

LLM providers — per-feature defaults

Configure which provider/model handles each feature (teacher, syllabus, card, research, critique, embed). API keys are read from env vars — never stored in the DB.

LLM config

New roadmap form

New roadmap form


What is Atlas?

Atlas is a local-first study app for self-learners entering a new domain. You bring the material (a PDF textbook, a YouTube series, a blog post, a Wikipedia article), Atlas helps you actually learn it instead of passively reading:

  • Highlight while you read — select text on any page, label it (#concept, #tradeoff), and ask follow-up questions grounded in the surrounding context.
  • Get grounded answers — when you ask @teacher a question about a highlight, Atlas retrieves the most relevant chunks from the whole document (not just the first 2000 chars) and feeds them to the LLM with page citations.
  • Build study cards — turn a highlight into a flashcard with hook, core explanation, analogy, trade-offs, STAR war story, and anti-pattern. Cards are template-aware: interview prep gets STAR framing, concept mastery gets simpler field sets.
  • Review with spaced repetition — cards queue up for review on a SM-2 schedule (ease factor + interval). One card at a time, tap to reveal, rate your recall (muscle / instinct / struggled / forgot), auto-advance.
  • Export anywhere — JSON backup, Markdown notes, CSV, or Anki TSV (File → Import in Anki desktop).

It is not a note-taking app, a chat app, or a YouTube downloader. It is the loop between those: read → highlight → understand → remember.

Who it's for

  • Engineers preparing for system design interviews (the interview template ships STAR war story + anti-pattern framing by default)
  • Self-learners working through a textbook or a long PDF
  • Anyone who watches YouTube lectures but never retains anything

What it is not

  • Not a hosted product. Single-user, local-first, runs on your machine. No auth system beyond a bearer token that gates /api/run (which executes arbitrary code).
  • Not a note sync service. No multi-device sync. Export to JSON/Markdown/Anki and import elsewhere.
  • Not a chat frontend. The LLM is a tool in the loop, not the interface. Cards and review are the point.

How it works — a real walkthrough

This is the actual corpus that's in the repo right now (the dev DB has 13 sources, 3 highlights, 4 cards, 41 embedded chunks).

1. Create a roadmap

curl -X POST http://localhost:5178/api/roadmaps?token=$TOK \
  -H "Content-Type: application/json" \
  -d '{"goal":"Distributed messaging with Apache Kafka","duration_days":30,"level":"intermediate","template":"interview"}'

Atlas calls the LLM once to generate a syllabus — 4 to 8 topics with outcomes, depth, estimated minutes, and prerequisites. Each topic becomes a row in the topic table.

2. Collect sources (two ways)

Manual — paste a URL, upload a PDF, or paste text. Atlas scrapes via yt-dlp (YouTube transcripts), pdftotext (PDFs), or direct fetch (Wikipedia, docs, blogs). Reddit goes through JSON or old.reddit.com HTML.

Auto-discovery — Atlas builds 3 search queries from the topic title, then searches Wikipedia, YouTube, Reddit, arXiv, HN Algolia, GitHub, and StackExchange in parallel. MDN is gated to web-related topics only (so "exactly-once semantics" doesn't return MathML <semantics> anymore). Every candidate title is then scored against the topic title using the embedder; below-threshold items are dropped.

curl -X POST http://localhost:5178/api/topics/2/research?token=$TOK

For the Kafka roadmap, this returned the Wikipedia article on Apache Kafka (11,725 chars), a YouTube talk on Kafka Transactions (2,951 chars), and a few others. Ingest runs with a concurrency cap of 3 so 20 topics don't take 10+ minutes serially.

3. Read and highlight

Open a source in the reader. PDFs render as real scrollable pages (pdf.js) with a zoom toolbar and page indicator. Articles render as text. Select any text on the page:

  • Label#concept, #tradeoff, #example, anything you want
  • Ask @teacher — opens a thread on that highlight; the server retrieves top-k chunks by similarity to highlight.text + question plus the chunks physically adjacent to the highlight span, formatted with page numbers, and feeds them to the LLM
  • Make card — turns that highlight into a study card (see step 4)

Highlights persist with page number and bounding rects, so they re-render correctly on scroll and zoom. Click ↗ in the sidebar to scroll the viewer to a highlight and flash it.

4. Ask @teacher a question

curl -X POST http://localhost:5178/api/teacher/questions?token=$TOK \
  -H "Content-Type: application/json" \
  -d '{"question":"Why does Kafka keep messages instead of deleting them like a queue?","highlight_id":2,"source_id":4,"topic_id":2}'

The server:

  1. Embeds question + highlight.text with nomic-embed-text (768-dim)
  2. Retrieves the top-3 chunks from the source by cosine similarity
  3. Also pulls the chunks adjacent to the highlight span (so the model sees what came before and after)
  4. Formats the context with page numbers (e.g. [p.3] ...) so the LLM can cite
  5. Calls the LLM with the enriched context and returns the answer

If embeddings aren't available yet (e.g. the embed job hasn't run), it gracefully falls back to a 2000-char slice around the highlight.

5. Generate a study card

From a highlight (the human-driven path):

curl -X POST http://localhost:5178/api/highlights/2/cards?token=$TOK -d '{}'

The LLM gets the highlight text + surrounding context and returns a JSON card. The server validates it:

  • core must be at least 80 chars
  • fewer than half the requested fields can be empty
  • the hook can't be a near-duplicate of an existing card on the same topic

If validation fails, the server retries once with a stricter prompt. If it still fails, the card is inserted with quality = 'degraded' and the UI shows a ⚠ Low quality — ↺ regenerate badge. The card is never silently passed off as a good card.

A real card from the Kafka corpus (card 2, generated from highlight 2):

hook:      "Kafka is not a message queue — it's a distributed, durable,
            replayable log. The mental model shift: instead of 'messages sent
            to subscribers', think 'events written to an infinite tape that
            any reader can seek to any position at any time'."

core:      "In a traditional queue (RabbitMQ, SQS), messages are deleted
            once consumed — they're ephemeral work units. Kafka keeps every
            record on disk for a configurable retention period (default 7
            days). Each record has an offset — a permanent address in a
            partition. Consumers track their own offset, so they can replay,
            rewind, or fast-forward independently..."

analogy:   "Kafka is like a shared Google Doc revision history: every change
            is permanently recorded with a timestamp, anyone can read it at
            any time, and multiple people reading it don't interfere with
            each other — unlike a whiteboard that's erased after each meeting."

tradeoffs: "Durability vs Latency: acks=all waits for all ISR replicas →
            higher durability but +2-5ms latency vs acks=1
            Throughput vs Ordering: more partitions = more parallelism, but
            strict ordering only within a single partition
            Retention vs Storage cost: longer retention = full replay
            capability but higher disk cost"

war_story: "S: At [fintech], our payment events pipeline used RabbitMQ —
            one consumer failure meant events were lost.
            T: Migrate to a system where failures were replayable.
            A: Migrated to Kafka, set retention=7d, min.insync.replicas=2.
            R: Recovered a 4-hour backlog after a pod crash by simply
            restarting consumers — zero data loss."

anti_pattern: "The naive approach treats Kafka like a queue with
            acknowledgements — deleting records after reading. This fails
            because: (1) you lose replay capability, (2) you can't have
            multiple independent consumers, (3) a slow consumer delays
            everyone behind it."

drills:    ["If consumer A is at offset 1000 and consumer B is at offset 500,
            does consumer B slow down consumer A?",
            "Why does Kafka guarantee ordering within a partition but not
            across partitions — and how do you design around this?",
            "With replication factor=3, min.insync.replicas=2, and acks=all —
            how many broker failures can you tolerate?"]

gap_check: "Can you explain why Kafka can have multiple consumers on the same
            topic without using the word 'offset'?"

6. Review with SM-2

Cards without a review session are notes. Every card gets a review row on creation with due_on = today. Open the Review view:

  • One card at a time, hook shown
  • Tap to reveal the full card
  • Four recall buttons: muscle / instinct / struggled / forgot
  • Auto-advance to the next due card
  • Session summary at the end

The SM-2 algorithm (the same one Anki uses):

Rating Quality Ease change Interval Rung
muscle q=5 +0.1 advances (1 → 6 → 6×ease → ...) +1
instinct q=4 no change advances +1
struggled q=2 -0.2 reset to 1 reset to 0
forgot q=1 -0.2 reset to 1 reset to 0

A "N due" badge in the nav shows how many cards are waiting.

7. Export

From the roadmap view, click ⬇ Export:

  • JSON backup — full fidelity (roadmaps, topics, sources, highlights, cards, review state) — for re-import
  • Markdown — one file per topic, cards as headed sections, highlights with page refs — for reading in Obsidian
  • CSV — flat card dump — for spreadsheets
  • Anki TSVhook as front, assembled body as back — Anki desktop: File → Import → select the .txt → map Field 1 to Front, Field 2 to Back

Quick start

export STUDY_TRACKER_TOKEN=$(openssl rand -hex 24)
# Optional but recommended: Postgres for the job queue + event timeline
createdb atlas_jobs
export ATLAS_PG_URL=postgres://localhost:5432/atlas_jobs
npm install   # installs pg
node --experimental-sqlite server.mjs
# open http://localhost:5178

The server refuses to start if STUDY_TRACKER_TOKEN is not set. This is intentional — /api/run executes arbitrary code on your machine.

Postgres is optional. If ATLAS_PG_URL is unreachable, Atlas falls back to the legacy SQLite pollers — but you lose the Jobs dashboard, retry/stuck detection, and the per-job event timeline.

Environment variables

Var Default Purpose
STUDY_TRACKER_TOKEN Required. Bearer token for all mutation/exec endpoints.
ATLAS_PORT 5178 HTTP port.
ATLAS_DB atlas.db SQLite DB path.
ATLAS_PG_URL postgres://localhost:5432/atlas_jobs Postgres URL for the job queue.
OLLAMA_URL http://127.0.0.1:11434 Ollama endpoint.
OPENAI_API_KEY OpenAI (or compatible) API key.
OPENAI_BASE_URL https://api.openai.com/v1 Override for Groq/Together/vLLM.
ANTHROPIC_API_KEY Anthropic API key.
GEMINI_API_KEY Google Gemini API key.
STUDY_TEACHER_PROVIDER / STUDY_TEACHER_MODEL ollama / qwen3.5:9b @teacher Q&A.
STUDY_SYLLABUS_PROVIDER / STUDY_SYLLABUS_MODEL ollama / qwen3.5:9b Syllabus generation.
STUDY_CARD_PROVIDER / STUDY_CARD_MODEL ollama / qwen2.5:1.5b Card synthesis.
STUDY_RESEARCH_PROVIDER / STUDY_RESEARCH_MODEL ollama / qwen2.5:1.5b Research query generation.
STUDY_CRITIQUE_PROVIDER / STUDY_CRITIQUE_MODEL ollama / qwen3.5:9b Interview critique.
STUDY_EMBED_PROVIDER / STUDY_EMBED_MODEL ollama / nomic-embed-text Embeddings (RAG). Run ollama pull nomic-embed-text (274MB).
GITHUB_TOKEN Optional, raises GitHub rate limit in research discovery.

Per-feature defaults can also be changed at runtime via POST /api/llm/defaults (stored in SQLite llm_default; DB wins over env). API keys are never stored in the DB.

Architecture

server.mjs          — HTTP server, DB migrations, all CRUD endpoints
syllabus.mjs        — syllabus generator (one LLM call → topic rows)
ingest.mjs          — multi-source scraping (Wikipedia, YouTube, Reddit, MDN, arXiv, HN, GitHub, StackExchange, PDF, paste)
research.mjs        — research job helpers (discover + ingest per topic, with embedding re-rank)
index.html          — SPA UI (roadmap → topic → source → reader → cards + review + Jobs + LLM)
llm/                — provider dispatcher + ollama/openai/anthropic/gemini adapters (complete + embed)
rag/                — chunker, embedder, retrieve (brute-force cosine), backfill
jobs/               — Postgres queue (db, queue, worker) — teacher_question/research/critique/embed
content/            — templates (cardPrompt, highlightCardPrompt, getTemplate) + seed-roadmaps
vendor/             — vendored study-tracker modules (teacher-worker, teacher-llm, interview-llm, tts, stt, code-run, ngrok-url)
files/              — persisted PDF binaries (<source_id>.pdf), gitignored

Data model

roadmap → topic → source → highlight → card → review
                                              chunk (source_id, seq, text, span_start, span_end, page_number, embedding BLOB)

Tables: roadmap, topic, source, highlight, card, review, chunk, teacher_question, interview_session, interview_turn, interview_critique, llm_default, llm_provider, research_job. All created idempotently on first start.

RAG (real, not faked)

Teacher Q&A and card generation are grounded in retrieved chunks, not a naive content.slice(0, 2000) from the front of the document.

  • Chunkingrag/chunker.mjs splits source text into ~1000-char paragraph-aware chunks with 200-char overlap. For PDFs, page numbers are inferred from the \f form-feeds pdftotext emits.
  • Embeddingrag/embedder.mjs calls llm/provider.embed() (default Ollama nomic-embed-text, 768-dim) and stores each chunk's embedding as a Float32Array BLOB in the chunk table.
  • Retrievalrag/retrieve.mjs does brute-force cosine similarity over Float32Array rows, scoped to one source by default. At corpus scale (a few thousand chunks) this is well under 10ms per query — no native extension needed.
  • Teacher enrichment — when a highlight is present, the server retrieves top-k chunks by similarity to highlight.text + question, plus the chunks physically adjacent to the highlight span, formatted with page numbers so the model can cite. Falls back to a content slice when embeddings are absent.
  • Backfillnode rag/backfill.mjs embeds all existing sources. Use --all to re-embed everything.

Cards worth studying

  • Template-aware promptscontent/templates.mjs ships four templates (interview, concept, skill, history). Only interview asks for STAR war story and anti-pattern.
  • Grounded generation — card prompts are built from retrieved chunks, not content.slice(0, 14000), so cards from a long PDF cover the whole document.
  • Validation gate — rejects cards with core under 80 chars, >50% empty fields, or near-duplicate hooks. Retries once with a stricter prompt, then marks quality = 'degraded'.
  • Honest degraded cardscard.quality is llm or degraded. The UI shows a "⚠ Low quality — ↺ regenerate" badge on degraded cards.
  • No auto-highlighting — cards from a source do NOT fabricate highlights. The only path to a card-with-highlight is a human selecting text and choosing "make card".
  • Card CRUDPATCH /api/cards/:id, DELETE /api/cards/:id, POST /api/cards/:id/regenerate.

Review loop (SM-2)

Cards without a review session are notes. Atlas uses SM-2 (ease factor + interval), replacing the earlier Leitner ladder.

  • GET /api/cards/due?limit=&roadmap_id= — cards where due_on <= date('now')
  • POST /api/cards/:id/review with { recall: 'muscle' | 'instinct' | 'struggled' | 'forgot' }
  • Review session UI — one card at a time, hook shown, tap to reveal, four recall buttons, auto-advance, session summary.
  • "N due" nav badge updates on view switch.

Highlight CRUD and export

  • PATCH /api/highlights/:id (label, note) and DELETE /api/highlights/:id
  • hl.note rendered in the sidebar with edit/delete controls.
  • GET /api/export?format=&roadmap_id= supports:
    • json — full fidelity backup
    • markdown — one file per topic, cards as headed sections
    • csv — flat card dump
    • anki — TSV with hook as front and assembled body as back (Anki: File → Import → select the .txt file)

Supported source types

Type How Notes
Article / Web Paste URL Wikipedia, docs, blogs that allow fetch. Medium often blocks (403).
YouTube Paste video URL Transcript via yt-dlp (auto-captions).
Reddit Paste post URL JSON API when available; else HTML scrape via old.reddit.com.
PDF Upload file or paste URL Text extraction via pdftotext (poppler) + binary persisted for page viewer.
Paste text Manual Use for X/Twitter, Facebook, Instagram — login walls block scraping.

PDF page viewer with highlights and @teacher Q&A

PDF sources are rendered as real scrollable pages (pdf.js) with page-aware highlights and source-grounded Q&A.

  • Continuous vertical scroll through rendered pages (lazy-loaded via IntersectionObserver)
  • Toolbar: zoom in/out, fit-width, current page indicator
  • Select text on any page → floating menu: Label / Ask @teacher / Make card
  • Highlights persist with page number + bounding rects — overlays re-render on scroll and zoom
  • Click ↗ in sidebar to scroll the viewer to that highlight and flash it
  • Ask @teacher on a highlight — server retrieves top-k chunks by similarity to highlight.text + question plus adjacent chunks, formatted with page numbers

PDF binaries are stored at files/<source_id>.pdf (gitignored).

AI research jobs (self-collecting pipeline)

  1. Build search queries — LLM generates 3 targeted queries from topic + roadmap goal
  2. Discover — searches Wikipedia, YouTube, Reddit, arXiv, HN Algolia, GitHub, StackExchange in parallel; MDN is gated to web-related topics only
  3. Re-rank — every candidate title is scored against the topic title using the embedder; below-threshold items are dropped
  4. Ingest — fetches content via ingest.mjs, writes source rows (skips duplicates, concurrency cap 3)

API

Method Path Body Response
POST /api/roadmaps/:id/syllabus { research: true } { topics, research_job_ids }
POST /api/roadmaps/:id/research { only_empty: true } { queued, job_ids }
POST /api/topics/:id/research research job object
GET /api/research/:id job status + progress JSON
GET /api/research?roadmap_id= list jobs for roadmap

LLM providers

Atlas routes every LLM call through llm/provider.mjs — a single complete() and embed() dispatcher that routes to the right adapter by provider name.

Provider Adapter Notes
ollama llm/ollama.mjs Default. /api/generate + /api/embed. No API key needed.
openai llm/openai.mjs OPENAI_API_KEY env. OPENAI_BASE_URL override for Groq/Together/vLLM.
anthropic llm/anthropic.mjs ANTHROPIC_API_KEY env. No embedding API — throws clear error.
gemini llm/gemini.mjs GEMINI_API_KEY env.

Provider config API

Method Path Purpose
GET /api/llm/providers List providers with configured boolean + available models
GET /api/llm/models?provider= List models for one provider
GET /api/llm/defaults Current per-feature defaults
POST /api/llm/defaults { feature, provider, model } — set default
POST /api/llm/test { provider, model, prompt } — returns latency + chars

Postgres job queue

When ATLAS_PG_URL is reachable, Atlas moves job state from SQLite pollers into Postgres. Two tables auto-create on boot:

  • job — one row per queued unit of LLM work (teacher_question, research, critique, embed)
  • job_event — append-only timeline (queued, claimed, llm_start, llm_end, retry, done, failed, stuck)

Why Postgres

  • FOR UPDATE SKIP LOCKED gives atomic, race-free claiming
  • LISTEN/NOTIFY wakes the worker the instant a job is enqueued
  • A real event timeline makes stuck/failed jobs debuggable
  • The reaper marks jobs stuck after 2 min without heartbeat and re-enqueues them

Jobs API

Method Path Purpose
GET /api/jobs?kind=&status=&limit= List jobs
GET /api/jobs/:id Job detail
GET /api/jobs/:id/events Full event timeline
POST /api/jobs/:id/retry Manually re-enqueue a failed/stuck job
POST /api/jobs/:id/cancel Cancel a running job
GET /api/jobs/events (SSE) Live stream of job events

License

Private. All rights reserved.

About

Atlas — self-collecting, source-grounded study engine (PDF/URL/YouTube → highlight → RAG → cards → SM-2 review)

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages