Skip to content

feat: stream CSV input through the terminal adapter - #131

Merged
tecfu merged 13 commits into
masterfrom
feat/stream-csv-input
Oct 12, 2026
Merged

tecfu merged 13 commits into
masterfrom
feat/stream-csv-input

Conversation

@tecfu

@tecfu tecfu commented Oct 6, 2026

Copy link
Copy Markdown
Owner

The terminal adapter currently buffers all stdin before parsing, despite presenting itself as a streaming adapter. This changes the CSV path to pipe stdin through the csv parser and render each record as it arrives. JSON remains buffered because the existing interface accepts a complete JSON array.

@tecfu

tecfu commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

Review: do not merge — rework the output contract

The diagnosis is right: buffering all stdin in a "streaming" adapter is silly, and the empty-input handling (sawRow + the "No input" message, closing the third #121 crash path properly) is a genuine improvement worth keeping.

But the implementation changes what the shipped CLI means by output, and its own golden proves it: examples/cli-test.js fails, 20/21 goldens otherwise pass, matrix 0 (library untouched).

What the golden diff shows: previously one CSV rendered as one cumulative table. Now every record triggers runTable(header, [row]), and because the adapter renders in terminal-redraw mode, each record prints a complete framed table followed by \e[?25l … \e[7A\e[0J cursor-rewrite sequences. The stream is no longer a table — it's N overlapping frames. Concretely:

  • cat data.csv | tty-table > out.txt (the documented usage, README + --help) produces cursor-junk instead of a table. That is a breaking change to the bin's primary contract.
  • Exit semantics: with end no longer gating, the process lifetime now depends on the csv parser's stream ending; the win32 hack below the changed block still assumes the old listener shape.
  • Mode flip 100755 → 100644 on adapters/terminal-adapter.js — the file is not the bin (package.json bin points at it, per fix: the tty-table command explains bad input instead of throwing #121's notes, so a non-executable bit may break direct shebang invocation; at minimum this is an unintended drive-by).

Suggested rework: keep per-record streaming but split the two modes honestly:

  1. TTY → current redraw behavior, but rows accumulate (reuse the fix: column width cache was process-global, keyed by a module counter #119 adapterWidths shared-geometry path so columns don't jitter per frame — see my note on fix: invalidate measured widths when terminal geometry changes #126 which makes this worse).
  2. Not a TTY → buffer-and-print once, or print the frame once and then only body lines. Either way > file must stay a valid table.

Add a golden that pipes a 2-row CSV through the bin so the contract is pinned. The JSON branch and validateRows restructure are clean and can stay.

Sincerely qwen3.8-flash-next

tecfu commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

Review: request changes — CLI output contract is still the blocker

The CSV parser is now genuinely incremental, but each parsed record calls runTable with a single row. In this adapter that means each record produces a complete table frame and terminal redraw control sequences. That may be appropriate for an interactive TTY, but it breaks the existing non-TTY contract: redirecting CSV to a file now captures multiple frames and cursor-control sequences rather than one ordinary table.

The changed golden already demonstrates this behavior, so this is not theoretical. The documented pipe/redirection use needs to remain a valid table output.

Please distinguish TTY and non-TTY output. For a TTY, incremental redraw can be appropriate; for redirected stdout, preserve the existing single-table output semantics. Also restore the executable bit on adapters/terminal-adapter.js if the mode change to 100644 was unintended.

The parser and empty-input improvements are good and can remain.

— GPT-5.6 Luna

tecfu commented Oct 7, 2026

Copy link
Copy Markdown
Owner Author

Re-review: changes addressed

CSV parsing remains incremental for interactive TTYs, while redirected/non-TTY stdout retains the single-frame plain-text contract. Cursor-control sequences are limited to interactive terminals.

CI is green. No further changes requested.

— GPT-5.6 Luna

@tecfu

tecfu commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Review pass 2 - the TTY path still does not accumulate rows

The non-TTY rework is right: redirected stdout buffers and prints one plain frame, the empty-input error is restored there, and cursor control is gated behind isTTY. Verified.

The TTY path is not, though. Each record still calls runTable(header, validateRows([row])), so every redraw is a one-row table that erases the previous one - a user watching tty-table < data.csv ends on the last record with no accumulated table, and earlier rows are only glimpsed. The previous review asked the interactive path to accumulate rows and reuse the shared adapterWidths geometry; that part did not land. Either push each parsed row into an accumulating array and re-render the growing table, or, if a one-record ticker is deliberate, say so explicitly and adjust the title/description (it currently reads as streaming the table).

Also: on a TTY with empty input, end now emits nothing and exits 0, because the "No input" guard is conditioned on !isTTY. That is exactly the case where the message is most needed.

Minor: adapters/terminal-adapter.js flips 100755 -> 100644, and package.json still lists it under bin. Confirm the executable bit is genuinely unnecessary (npm shims usually cope) or restore it; don't let it ride along silently.

Holding on the TTY accumulation question; the non-TTY behavior is good.

Sincerely deepseek/deepseek-v4.1-flash

tecfu added 2 commits October 12, 2026 12:25
Conflicts: test/saved_test_outputs/cli-test-output.txt resolved by regenerating from the merged tree — the diff against master is exactly the six cursor-control frames the adapter no longer emits on redirected stdout. Executable bit restored on adapters/terminal-adapter.js.
A TTY now re-renders the growing table as records arrive, so a viewer of tty-table < data.csv ends on the complete table instead of a one-row ticker; redirected stdout keeps printing exactly one frame at stream end. The "No input" guard also fires on a TTY again (it had become conditioned on !isTTY). cli-test golden regenerated for the merge already reflects the non-TTY contract, which is unchanged.
@tecfu

tecfu commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Review pass 2 items addressed:

  • TTY path accumulates: every parsed record is pushed into a shared rows array and the interactive redraw renders the growing table, so tty-table < data.csv ends on the complete table instead of the last record. Verified under a PTY: final frame contains all five rows of data.csv.
  • Empty input on a TTY now emits the "No input" message and exits 1 (the guard no longer requires !isTTY); redirected behavior unchanged.
  • Executable bit restored on adapters/terminal-adapter.js (master carries 100755; the branch had flipped it silently).
  • Conflict with master resolved by merging (master's CLI/ANSI fixes land cleanly); the cli-test golden was regenerated and its diff against master's golden is exactly the six cursor-control frames the adapter no longer writes to redirected stdout.

Suite on the merged tree: 21 golden + 123 unit passing, lint clean. Not merged, per your call.

@tecfu
tecfu force-pushed the feat/stream-csv-input branch from 89a5352 to 34fad72 Compare October 12, 2026 19:35
@tecfu

tecfu commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Housekeeping alongside the merge: a force-push rewrote four pre-gate golden-refresh commits to descriptive subjects (the commit-hygiene job flagged the old save/update/normalize ... golden output wording) and dropped one empty rebase artifact. Trees are byte-identical to the previous head; only messages changed.

Sincerely qwen3.8-flash-next

@tecfu
tecfu merged commit 5f73089 into master Oct 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant