fix(preview): max_tokens 4096→8192 + detect truncation explicitly
All checks were successful
Deploy to Production / deploy (push) Successful in 1m24s

Root cause of repeat 422s: 4096 was too tight for ambitious prompts
(Marco's research-assistant prompt produces ~12kB of JSON before the
model gets cut off mid-string). The error then surfaced as an opaque
"Unterminated string in JSON" zod failure instead of pointing the user
at the real problem.

Two fixes:
- maxTokens back to 8192 (the original) for all Claude tiers, 4096 for
  GLM. Timeouts bumped to 95s — Sonnet 4.6 at ~130 tok/s does 8192 in
  ~63s, ~30s headroom for cold starts, still under Cloudflare's 100s
  edge cap.
- Detect stop_reason === 'max_tokens' on the Anthropic response BEFORE
  parsing and throw the new SpecTruncatedError. /preview catches it
  and returns 422 spec_too_large with a clear "split the prompt"
  message instead of leaking the zod parse failure.
This commit is contained in:
Marco Sadjadi
2026-05-28 19:34:40 +02:00
parent 979d1abfca
commit d2b19a5439
2 changed files with 53 additions and 14 deletions

View File

@@ -14,6 +14,7 @@ import {
import {
BannedPatternError,
SpecTimeoutError,
SpecTruncatedError,
SpecValidationError,
generateSpec,
pickPreviewModel,
@@ -153,6 +154,21 @@ export async function serverRoutes(app: FastifyInstance): Promise<void> {
detail: 'Spec generation took too long. Try a shorter, more specific prompt.',
});
}
if (err instanceof SpecTruncatedError) {
app.log.warn(
{
reason: err.message,
prompt: parsed.data.prompt.slice(0, 200),
model: choice.displayName,
},
'preview_spec_truncated',
);
return reply.code(422).send({
error: 'spec_too_large',
detail:
'The spec for this prompt exceeded the maximum response size. Split it into fewer tools or describe one capability per prompt.',
});
}
app.log.error(err);
return reply.code(500).send({ error: 'preview_failed', detail: (err as Error).message });
}