feat(preview): SSE-streamed generation, no CF 100s edge cap
All checks were successful
Deploy to Production / deploy (push) Successful in 1m27s
All checks were successful
Deploy to Production / deploy (push) Successful in 1m27s
Architectural fix for "spec_too_large" / preview_timeout — the sync endpoint had to fit the whole model run into Cloudflare's ~100s edge window, which made the system fragile against any prompt that produced a verbose spec. The new streaming path pipes Anthropic's token deltas as Server-Sent Events; every chunk resets CF's idle timer and a 15s keepalive comment guarantees activity even during slow first-token windows. @bmm/llm: new streamSpecFromAnthropic() exposes the SDK's .stream() flow with the same typed-error contract as generateSpec — same SpecTruncatedError / SpecValidationError / SpecTimeoutError raised from the relevant moment. API: POST /v1/servers/preview/stream returns text/event-stream with events 'text' (deltas), 'spec' (final success payload, same shape as the sync endpoint), 'error' (typed). Anthropic-only — GLM/hobby falls back to the sync route via 409 streaming_unavailable. Frontend: apiSseStream() handles the POST + ReadableStream + SSE parser. The wizard's analyze() prefers the stream and only uses the sync endpoint on the explicit 409 fallback. nginx (api.buildmymcpserver.com): the /v1/builds/ location block (which already had proxy_buffering off + 600s read timeout for the WS build stream) now also matches /v1/servers/preview/stream so the SSE response isn't buffered.
This commit is contained in:
@@ -33,3 +33,96 @@ export function apiWebSocketURL(path: string): string {
|
||||
const wsBase = httpBase.replace(/^http/, 'ws');
|
||||
return `${wsBase}${path}`;
|
||||
}
|
||||
|
||||
export interface SseHandlers {
|
||||
onEvent: (event: string, data: unknown) => void;
|
||||
onError?: (err: Error) => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* POST with SSE response. Used by the streaming preview where Cloudflare's
|
||||
* sync edge timeout (~100s) is fatal — every server-side chunk resets it,
|
||||
* so as long as the model emits text we never hit the cap.
|
||||
*
|
||||
* Native EventSource doesn't support POST or auth-cookie credentials, hence
|
||||
* the manual fetch + ReadableStream consumer. Caller controls aborting via
|
||||
* the supplied AbortSignal (we attach it to the underlying fetch).
|
||||
*/
|
||||
export async function apiSseStream(
|
||||
path: string,
|
||||
body: unknown,
|
||||
handlers: SseHandlers,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(`${API_BASE}${path}`, {
|
||||
method: 'POST',
|
||||
credentials: 'include',
|
||||
cache: 'no-store',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'text/event-stream',
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
signal,
|
||||
});
|
||||
} catch (err) {
|
||||
handlers.onError?.(err as Error);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
let detail: unknown = undefined;
|
||||
try {
|
||||
detail = await res.json();
|
||||
} catch {}
|
||||
const err = new Error(`api_error_${res.status}`);
|
||||
(err as unknown as { detail?: unknown }).detail = detail;
|
||||
(err as unknown as { status?: number }).status = res.status;
|
||||
handlers.onError?.(err);
|
||||
return;
|
||||
}
|
||||
|
||||
const reader = res.body?.getReader();
|
||||
if (!reader) {
|
||||
handlers.onError?.(new Error('no_response_body'));
|
||||
return;
|
||||
}
|
||||
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = '';
|
||||
try {
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
// SSE events are separated by a blank line. Parse one event at a time.
|
||||
let idx = buffer.indexOf('\n\n');
|
||||
while (idx >= 0) {
|
||||
const raw = buffer.slice(0, idx);
|
||||
buffer = buffer.slice(idx + 2);
|
||||
let event = 'message';
|
||||
const dataLines: string[] = [];
|
||||
for (const line of raw.split('\n')) {
|
||||
if (line.startsWith(':')) continue; // SSE comment
|
||||
if (line.startsWith('event:')) event = line.slice(6).trim();
|
||||
else if (line.startsWith('data:')) dataLines.push(line.slice(5).trim());
|
||||
}
|
||||
if (dataLines.length > 0) {
|
||||
const data = dataLines.join('\n');
|
||||
let parsed: unknown = data;
|
||||
try {
|
||||
parsed = JSON.parse(data);
|
||||
} catch {
|
||||
// Treat as a raw string event.
|
||||
}
|
||||
handlers.onEvent(event, parsed);
|
||||
}
|
||||
idx = buffer.indexOf('\n\n');
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
handlers.onError?.(err as Error);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user