Repository navigation
[Bug]: Assistant messages stay marked as streaming after the server restarts mid-response #14572
Description
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
- Start a turn whose provider streams a multi-paragraph response (default "paragraph" streaming mode).
- As soon as the first paragraph appears, stop the T3 server: Ctrl+C, an app update, or a crash. SIGTERM and SIGKILL behave the same.
- Start the server again and open the thread.
Expected behavior
Startup reconciliation already marks the orphaned session as failed ("Provider session did not survive a server restart. Send a new message to continue."). The partial assistant message it left behind should be completed too, keeping its text.
Actual behavior
The partial message keeps streaming: true (projection_thread_messages.is_streaming = 1). Its timestamp stays hidden and the Run buttons on its code blocks stay disabled. The flag never clears: it survives later turns that complete normally and further clean restarts.
Cause: ProviderRuntimeIngestion completes the messages it streams from in-memory state (finalizeAssistantMessage / finalizeActiveSegmentForTurn), and that state dies with the process. reconcileProviderSessions in apps/server/src/serverRuntimeStartup.ts only dispatches thread.session.set (status error, or starting when the thread continues after an update). No thread.message.assistant.complete or thread.message.reasoning.complete is sent for rows already projected as streaming. Reading the code, the continue-after-update path leaves the old message streaming too; I reproduced only the error path end to end.
Related: #14433 fixes the same symptom in orchestrator V2's ProviderRuntimeRecoveryService. This report is about the startup reconciliation that ships on main.
I have a focused fix with tests and can open a PR if this is triaged.
Impact
Minor bug or occasional failure
Version or commit
main @ 0cf482b (2026-09-30); the reconciliation code is unchanged on main as of 2026-10-01
Environment
Linux, Node 24.21.0, server run from source. Reproduced with the repo's ACP mock agent (scripts/acp-mock-agent.ts in content-then-hang mode, with the streamed text containing a paragraph break) behind the Cursor provider. Reconciliation does not depend on the provider.
Logs or stack traces
before stop: is_streaming=1 text="First paragraph of the answer.\n\n"
after restart: is_streaming=1 (unchanged; still 1 after the next turn and after another clean restart)Screenshots, recordings, or supporting files
No response
Workaround
None. Sending a new message does not clear the old flag.
Activity
juliusmarminge commentedon Oct 1, 2026
Note
Grok responding on behalf of Julius.
Triage
Thanks for the clear repro and diagnosis. I confirmed this is a real bug on
main, and #14433 (open PR) doesn't cover it.Startup reconciliation already settles the orphaned provider session:
reconcileProviderSessionsinapps/server/src/serverRuntimeStartup.tsonly dispatchesthread.session.set(errorwith "Provider session did not survive a server restart. Send a new message to continue.", orstartingon the continue-after-update path).thread.session-setdoesn't touch messages, and the command read model used there always hasmessages: [], so the reconciler never even sees the streaming rows.The
streamingflag gets set bythread.message.assistant.delta/thread.message.reasoning.deltaand is only cleared by the matching*.completecommand. Those completes come fromProviderRuntimeIngestion(finalizeAssistantMessage/finalizeActiveSegmentForTurn), and its segment state and text buffer live in process memory, so they die with the server. Paragraph mode makes this easy to hit: a finished paragraph is sent as a delta (is_streaming = 1, text kept), and it isn't completed until the segment finalizes. After a restart the row stays streaming. A later turn uses a new message id, and once the session iserrorwith no active turn, later restarts never look at it again. The client hides the timestamp, the copy control, and code-block Run whilemessage.streamingis set, which matches what you saw.#14433 clears the same flag in Orchestrator V2's
ProviderRuntimeRecoveryServiceont3code/codex-turn-mapping, but it doesn't change this startup path.What should happen: when an orphaned session is reconciled, any assistant and reasoning rows for that turn that are already projected as streaming should be completed, keeping their projected text.
*.completealready does that, because an empty complete text leaves the existing text in place. This should cover both the error settle and the continue-after-update path. Text that never left the in-memory buffer is gone with the process, so there's no need to try to rebuild it.You mentioned you have a fix with tests. A focused PR that links this issue and adds coverage for the flag in the existing reconcile tests is very welcome.
t3dotgg commentedon Oct 2, 2026
Note
🤖 GPT-6.1-Sol responding on behalf of Theo
This issue should be resolved in the next nightly build by Orchestrator v2 (#2829).
Please try that nightly. If the problem still exists, open a new issue with the nightly version you tested and steps to reproduce it.