Streaming Chat
The chat completion endpoint provides a way to generate chat completions based on provided input messages. This endpoint supports streaming via Server-Sent Events (SSE), allowing for partial message deltas to be sent back to the client.
Request
For details on the parameters for chat completion, refer to the chat-completion-parameters documentation.
stream parameter is set to true.Response
The response is streamed back to the client using Server-Sent Events (SSE). Each event corresponds to a partial completion message or an error message.
event: completion: Indicates a partial completion message.data: Contains the JSON object representing the partial completion message.
event: error: Indicates an error occurred.data: Contains the JSON object representing the error message.
event: done: Indicates the completion of the streaming process.data: Contains[DONE].trace_id: Contains the unique identifier for tracing purposes.document_chunks: Contains the list of document chunks processed during the streaming.
Data
For event: completion, the data field is a JSON object containing a chunk of the completion message.
The unique identifier of the chunk
The model associated with the chunk
The object associated with the chunk
The creation date of the chunk
An array of choices made for the chunk
Examples
Completion Event
event: completion
data: {
"id": "<id>",
"model": "gpt-3.5",
"object": "chat_completion",
"created": "2024-04-04T12:00:00Z",
"choices": [
{
"delta": {
"content": "Hello, how can I assist you today?",
"role": "assistant"
},
"finish_reason": "complete"
}
]
}
Error Event
event: error
data: {
"error": "Internal Server Error",
"message": "An internal server error occurred while processing the request."
}
Done Event
event: done
data: "[DONE]"
trace_id: <trace_id>
document_chunks: ["chunk1", "chunk2", "chunk3"]