feat(standard-tests): validate tool call chunks during streaming - #34707
Conversation
Merging this PR will not alter performance
|
|
Local smoke-test summary for the provider follow-up: Checked whether a streamed tool call produces at least one per-chunk Passed:
Did not pass this PR's exact
This PR remains scoped to enabling the flag for OpenAI and Anthropic only; the broader provider list can be follow-up work after profile decisions/translator fixes. |
|
Scope update: based on the smoke-test results above, this PR now also opts in the smoke-passing providers instead of leaving them for follow-up. Added
MistralAI remains excluded because it does not currently satisfy this PR's exact |
|
Follow-up to the scope update: Perplexity is now left out of this PR. Perplexity Responses passed the smoke test, but the generated Perplexity profile data currently covers the |
As a LangChain user streaming a tool-calling model, I expect each streamed chunk to expose structured
tool_call_chunkcontent blocks so I can render or process tool calls live, instead of waiting for the final aggregated message.This adds
tool_call_streamingtoModelProfileand uses it in the standard chat-model tool-calling tests. When a model profile opts in,test_tool_callingandtest_tool_calling_asyncnow validate that at least one streamed chunk includes atool_call_chunkblock viacontent_blocks, while preserving the existing final-message validation.This keeps the contract profile-gated so providers can opt in once their streaming chunk shape is verified. This PR opts in the providers verified by smoke testing with straightforward profile coverage: OpenAI, Anthropic, Fireworks, HuggingFace, OpenRouter, DeepSeek, and xAI. The generated profile artifacts are refreshed so runtime profiles expose the new capability flag.
Perplexity Responses also passed the smoke test, but its current profile data is for the
sonarfamily while the Responses smoke path used a routed model string. That profile strategy is left as follow-up. MistralAI currently streams.tool_call_chunks, but its content-block translator exposes a completetool_callblock instead oftool_call_chunk, so it also stays out of this flag until that integration is fixed.