Skip to content

fix(cli): read piped stdin to EOF instead of the first line (#166) - #167

Merged
penta2himajin merged 1 commit into
mainfrom
claude/issue166-stdin-eof
Aug 1, 2026
Merged

fix(cli): read piped stdin to EOF instead of the first line (#166)#167
penta2himajin merged 1 commit into
mainfrom
claude/issue166-stdin-eof

Conversation

@penta2himajin

Copy link
Copy Markdown
Owner

Summary

cat file | qwisp chat and qwisp chat < file answered using only the first line of the input — no error, no warning, exit 0 — while both the help text and the usage line advertise piping as an input path. main.swift used readLine() (one line) for every non-argv prompt, with no TTY check, so the interactive case (where one line is what the user means) and the piped case shared the same call. Now stdin is read to EOF when it is not a TTY; readLine() stays for the interactive case, where reading to EOF would block until ^D.

Found while building #162's A/B harness: a 126KB (~35K-token) file was ingested as 52 tokens and reported prompt 52 tok (94 tok/s) · gen 32 tok (96.4 tok/s) — a completely healthy-looking run that measured nothing. Same silent-failure class as #151.

Related issue

Closes #166

Verification

Gates run on a tree containing only this change (the in-progress #162 knob was stashed and both schemes rebuilt first, so the green below is evidence for this commit and not a superset):

  • scripts/test_raw.shRAWTESTS 98/98 PASS
  • scripts/test_completion.shCOMPTEST 97/97 PASS + [stdin] piped multi-line read to EOF: 1-line=11 tok < 2-line=23 tok ok
  • scripts/test_bench_batch.shBENCHBATCHTEST PASS

The new regression check is GPU-free (QWISP_FAKE=1 uses FakeBackend: no model weights, only the tokenizer this gate already requires). It asserts a strict increase in prompt tokens between one-line and two-line stdin rather than an exact count, so it does not encode the tokenizer's segmentation.

Verified RED before the fix (both arms reported 11 tok on the pre-fix binary) and GREEN after (11 < 23).

  • Tests pass
  • Zero warnings
  • Related docs updated (n/a — help text already documented the intended behaviour; the code now matches it)

Notes for reviewer

  • Behaviour change is confined to the piped/redirected case. Interactive qwisp chat with no argument still takes one line, deliberately.
  • The prompt is trimmed of surrounding whitespace/newlines so a trailing newline in a piped file does not change tokenization.

`cat file | qwisp chat` and `qwisp chat < file` answered using only the first
line of the input, with no error and exit 0 — while both the help text and the
usage line advertise piping as an input path.

Cause: main.swift used readLine(), which returns a single line, for every
non-argv prompt. There was no TTY check, so the interactive case (where one
line is what the user means) and the piped case shared the same call.

Read stdin to EOF when it is not a TTY; keep readLine() interactively, where
reading to EOF would block until ^D.

Found while building #162's A/B harness: a 126KB (~35K-token) file was ingested
as 52 tokens and reported `prompt 52 tok (94 tok/s)` — a healthy-looking run
that measured nothing. Same silent-failure class as #151.

Gate: test_completion.sh gains a GPU-free regression check (QWISP_FAKE=1 needs
no model weights, only the tokenizer the gate already requires). It asserts a
strict increase in prompt tokens between one-line and two-line stdin rather than
an exact count, so it does not encode the tokenizer's segmentation. Verified RED
on the pre-fix binary (both arms 11 tok) and GREEN after (11 < 23).

Closes #166

Co-Authored-By: Claude <noreply@anthropic.com>
@penta2himajin
penta2himajin merged commit f6da277 into main Aug 1, 2026
1 check passed
@penta2himajin
penta2himajin deleted the claude/issue166-stdin-eof branch August 1, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chat: piped stdin is truncated to the first line (cat file | qwisp chat silently answers on line 1)

1 participant