Skip to content

Fix piped output truncation, harden scraping, add queue/search/harvest - #3

Open
eppser wants to merge 1 commit into
csima:masterfrom
eppser:fix-piped-output-and-harvest
Open

eppser wants to merge 1 commit into
csima:masterfrom
eppser:fix-piped-output-and-harvest

Conversation

@eppser

@eppser eppser commented Sep 21, 2026

Copy link
Copy Markdown

Ran this against a live event (5 evaluations, ~500 submissions) and hit three bugs. Fixes plus the tooling I needed to review at that volume.

Fixes

1. Piped output truncates

bin/sessionize.js calls process.exit() right after console.log(JSON.stringify(...)). Writes to a pipe are async, so the process exits before the buffer flushes:

$ sessionize session show | wc -c
512
$ sessionize session show > out.json; wc -c < out.json
7139

Every jq example in the README was silently getting truncated JSON. Fixed by exiting from the write callback.

2. Extraction assumed one event's conventions

Titles were found by matching a [Track N] prefix, description and outline by slicing innerText between guessed markers, speaker by a Firstname Lastname regex, and rating criteria were hardcoded to Practical/Originality/Relevance/Clarity. On an event without those conventions you get title: null, description: "", and a speaker name lifted from a form label (mine returned "Media Release").

Extraction now reads the server-rendered hooks that are stable across events: #Session_Id, .ibox-title h3, p.es-description, the .es-categories field table, .es-speakers, and the dt / input.rating-value pairs. Per-event criteria and custom fields come through as data rather than assumptions. Speaker gains tagline, bio, photo, and speaker-level custom fields.

3. Ratings never reached the posted fields

setRatings drove only the jQuery rateYo widget and left the hidden input.rating-value fields (Criteria[i].Rating) untouched. It now writes the input first, then syncs the widget so the UI agrees.

New

  • session queue — the full queue via Sessionize's own tabboxes endpoint, paged, with rating state per session. session list only sees the ~20 sessions linked on the rendered page while reporting its count as if complete; it now says so in its output rather than changing behaviour.
  • session search <query> — session search scoped to the evaluation.
  • session harvest — walks the queue and caches every submission to local JSON (~/.sessionize-cli-harvest/<eventId>-<evaluationId>.json by default). Resumable, and reconnects if a tab or the CDP connection drops mid-walk. Reviewing a few hundred submissions against a file beats a browser round trip each time.
  • session rate --dry-run — fills the form and reports what would post without saving. rate also refuses to run without --comment when the evaluation marks it required, instead of submitting a form the server will reject.

Testing

All commands exercised against a live evaluation: queue returned 125/166/92/64/54 across five tracks, exactly matching the counts on the event page; harvest pulled 501 submissions with zero failures; search returned sensible hits; rate --dry-run confirmed the hidden inputs carry the scores and the save button enables, then reloaded without saving. No evaluation was submitted.

One caveat: I drove this through a local Chrome over CDP rather than Browserbase, so the Browserbase connect path is unexercised here. lib/browser.js and lib/auth.js are untouched by this PR.

Not included

extractSessionDetails now returns comment (the editable field's current value) plus commentRequired, where it previously returned a comments string scraped from the page body. Happy to keep the old key as an alias if you'd rather not break that shape.

https://claude.ai/code/session_0183YGef9Gm7vbVSB6HSNRnq

Three fixes and one new capability, found while running this against a live
event with five evaluations and ~500 submissions.

Fixes:
- bin/sessionize.js exited immediately after console.log. stdout to a pipe is
  async, so every piped invocation was truncated at the first buffer chunk
  (512 bytes here) -- which silently breaks the jq examples in the README.
  Now writes and exits in the write callback.
- Session extraction assumed the author's own event conventions: titles were
  matched by a `[Track N]` prefix, description/outline by slicing innerText
  between guessed markers, speaker by a `Firstname Lastname` regex, and rating
  criteria were hardcoded to Practical/Originality/Relevance/Clarity. On an
  event without those conventions this yields null titles, empty descriptions
  and a speaker name lifted from a form label. Extraction now reads the stable
  server-rendered hooks (#Session_Id, .ibox-title h3, p.es-description, the
  custom-field tables, .es-speakers, and the dt / input.rating-value pairs), so
  per-event criteria and custom fields come through as data.
- setRatings drove only the jQuery rateYo widget and never wrote the hidden
  input.rating-value fields that the form actually posts. It now writes the
  input first, then syncs the widget.

New:
- session queue: the full evaluation queue via Sessionize's own tabboxes
  endpoint, paged, with rating state. session list only sees the ~20 sessions
  linked on the rendered page while reporting its count as if complete.
- session search <query>: session search within an evaluation.
- session harvest: walks the queue and caches every submission to local JSON.
  Resumable, and reconnects if a tab or CDP connection drops mid-walk.
- session rate --dry-run: fills the form and reports what would post without
  saving, and rate now refuses to run without --comment when the evaluation
  requires one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant