Streaming replies

An AI reply is a stream: after you start it, the platform writes a "generating" AI message from the bot to the user; you write deltas in order and the platform pushes them to the user in real time; finally you finish it or report a failure. Concepts and limits are in AI conversations.

With the server SDKs you do not need to call these APIs directly: the stream returned by StartStream batches, numbers and retries writes and chooses the write method, and Pipe* turns OpenAI, Anthropic and Gemini streams into replies.

Calling the APIs directly, a whole reply goes like this. User zhangsan has just sent the bot assistant a message whose seq in ai.message_received is 356:

# 1. 开始回复，用户的客户端立即显示“生成中”
STREAM_ID=$(curl -s -X POST "$IM_API/$ORG/$APP/ai/streams" \
  -H "Authorization: Bearer $APP_TOKEN" -H "Content-Type: application/json" \
  -d '{"bot":"assistant","to":"zhangsan","reply_to_seq":356}' | jq -r .stream.stream_id)

# 2. 把模型生成的内容写入两段，用户看到文字逐段出现
curl -s -X POST "$IM_API/$ORG/$APP/ai/streams/$STREAM_ID/deltas:batch" \
  -H "Authorization: Bearer $APP_TOKEN" -H "Content-Type: application/json" \
  -d '{"deltas":[
    {"seq":1,"ops":[{"op":"append","index":0,"type":"text","text":"明天上海"}]},
    {"seq":2,"ops":[{"op":"append","index":0,"type":"text","text":"**多云**。"}]}
  ]}'

# 3. 结束回复，平台保存最终消息并发离线推送
curl -s -X POST "$IM_API/$ORG/$APP/ai/streams/$STREAM_ID/finish" \
  -H "Authorization: Bearer $APP_TOKEN" -H "Content-Type: application/json" \
  -d '{"last_seq":2,"model":"gpt-4.1"}'

You want to	API
Start a reply	Start a reply
Write what the model generates	Write deltas: Persistent write or Batch write
Finish normally	Finish a reply
Finish when the model fails	Report a failure
See a reply's status and what's been written	Get a stream

Stream object

{
  "stream_id": "1841000000000000123",
  "bot": "assistant",
  "to": "zhangsan",
  "conversation_id": "1840098765432101",
  "message_id": "1841000000000000124",
  "seq": 358,
  "client_stream_id": "req-7f3a",
  "reply_to_seq": 356,
  "status": "streaming",
  "last_seq": 21,
  "finish_reason": null,
  "error": null,
  "model": null,
  "usage": null,
  "content_bytes": 0,
  "first_delta_ms": 420,
  "started_at": "2026-11-02T08:00:00.000Z",
  "finished_at": null
}

Field	Type	Description
stream_id	String	Stream ID
bot, to	String	The bot and the user receiving the reply
conversation_id, message_id, seq	—	Conversation, message ID and sequence number of this AI message in the conversation
client_stream_id	String	Idempotency key given when starting
reply_to_seq	Number	The message in the conversation this reply answers
status	String	streaming, completed, stopped, failed
last_seq	Number	Sequence number of the last accepted delta; unrelated to seq
finish_reason, error	—	Finish reason and failure details, see Finish reasons
model, usage	—	Model name and usage given when finishing
content_bytes	Number	Size of the saved content when finished
first_delta_ms	Number	Milliseconds from start to the first delta
started_at, finished_at	String	Timestamps
content	Array	Only from Get a stream while generating or within 10 minutes after the end: the accumulated parts

Start a reply

Writes an AI message with status streaming from the bot to the user; the user's client shows "generating" immediately. A one-to-one conversation is created if needed.

POST/{org_name}/{app_name}/ai/streams

Request body

Field	Type	Required	Description
bot	String	Yes	Bot username
to	String	Yes	Username receiving the reply
reply_to_seq	Number	No	The message being answered, usually seq of the message in ai.message_received; shown as a quote
client_stream_id	String	No	Idempotency key, 1–64 visible ASCII characters. A repeat returns the existing stream whether or not it has ended
replace	Boolean	No	If a reply is already being generated in the conversation, end it as stopped (replaced) first. Default false, which returns 409 stream_in_progress

Request example

curl -X POST "$IM_API/$ORG/$APP/ai/streams" \
  -H "Authorization: Bearer $APP_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bot": "assistant",
    "to": "zhangsan",
    "reply_to_seq": 356,
    "client_stream_id": "req-7f3a"
  }'

Response

201 Created when new, 200 OK when client_stream_id repeats; the body is { "stream": stream object }.

Errors

HTTP status	code	Description
400	invalid_argument	bot is not a bot (details.reason is not_bot), to is a bot (recipient_is_bot), or a parameter is malformed
403	permission_denied	The bot is disabled (bot_disabled)
403	user_blocked	The user blocked the bot; not_friend when the app requires friendship for one-to-one chat and they are not friends
404	not_found	The bot or user does not exist
409	stream_in_progress	A reply is already being generated in the conversation; details.stream_id is its ID
409	limit_exceeded	The bot has reached its limit of replies being generated (bot_active_streams)

Write deltas

Reply content is written as deltas. Each delta has a sequence number seq, starting at 1 and increasing without gaps, and contains one or more operations:

{ "seq": 12, "ops": [ { "op": "append", "index": 1, "type": "text", "text": "最高 23" } ] }

op	Fields	Description
append	index, type (text or reasoning), text	Appends text to part index; when index equals the current number of parts a new part is created
put	index, part	Replaces part index with part; used for images, files and tool state changes (such as running to done); when index equals the current number of parts a new part is created

• Parts are only added, never removed; index starts at 0. Skipping an index or appending to a non-text part returns 400 invalid_argument (invalid_ops).
• Deltas with sequence numbers already accepted are skipped without error, so it is safe to resend after a failure. A gap in sequence numbers returns 409 seq_gap with details.expected_seq.
• After the stream has ended (user stop, timeout, replaced, …) writes return 409 stream_closed; details.reason is the final status and details.finish_reason the reason. Stop calling the model when you get it.

There are two ways to write, and they can be mixed.

Persistent write

One long request whose body is NDJSON, one delta per line, written as the model generates. The platform processes each line without waiting for the body to end and writes an acknowledgement line to the response.

POST/{org_name}/{app_name}/ai/streams/{stream_id}/deltas

• Request header Content-Type: application/x-ndjson; send the body chunked (HTTP/1.1) or over HTTP/2.
• The platform returns 200 OK and the headers immediately. The response body is also NDJSON: after each batch it writes {"acked_seq": 21}; if a line fails it writes {"error": {"code": "…", "message": "…", "details": {…}}} and ends, and deltas already acknowledged remain valid.
• When the request body ends the platform writes a final acknowledgement and returns. The platform ends the request after idle_timeout_seconds without a new line; the request can last as long as a reply (30 minutes) and is not subject to the usual 30-second API timeout.
• A line is at most 64 KB; longer lines return payload_too_large (line).
• Reverse proxies must not buffer the request body on this path (nginx: proxy_request_buffering off), otherwise the platform receives nothing until the request ends.

printf '%s\n' \
  '{"seq":1,"ops":[{"op":"append","index":0,"type":"text","text":"明天上海"}]}' \
  '{"seq":2,"ops":[{"op":"append","index":0,"type":"text","text":"**多云**。"}]}' |
curl -N -X POST "$IM_API/$ORG/$APP/ai/streams/1841000000000000123/deltas" \
  -H "Authorization: Bearer $APP_TOKEN" \
  -H "Content-Type: application/x-ndjson" \
  --data-binary @-
# {"acked_seq":1}
# {"acked_seq":2}

Batch write

A normal request that writes a batch of deltas. Use it where streaming request bodies are not supported (some cloud functions), or to continue after a proxy breaks a persistent write. The server SDKs switch to it automatically when a persistent write fails.

POST/{org_name}/{app_name}/ai/streams/{stream_id}/deltas:batch

Field	Type	Required	Description
deltas	Array	Yes	Deltas in sequence order; at most 100 deltas and 64 KB per request

Returns 200 OK with { "acked_seq": 21 }.

curl -X POST "$IM_API/$ORG/$APP/ai/streams/1841000000000000123/deltas:batch" \
  -H "Authorization: Bearer $APP_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "deltas": [
      { "seq": 3, "ops": [ { "op": "put", "index": 1, "part": { "type": "image", "url": "https://cdn.example.com/a.png", "width": 800, "height": 600 } } ] }
    ]
  }'

Finish a reply

Finish after all deltas are acknowledged; the status becomes completed. The platform saves the final content, updates the message, sends an offline push to users not in the foreground, and sends ai.stream_finished.

POST/{org_name}/{app_name}/ai/streams/{stream_id}/finish

Field	Type	Required	Description
last_seq	Number	Yes	Sequence number of the last delta; it must equal the last delta accepted by the platform, otherwise 409 seq_gap, which guarantees all content arrived
finish_reason	String	No	Such as stop, length, tool_calls; default stop
model	String	No	Model name
usage	Object	No	{ "input_tokens": 120, "output_tokens": 380 }

Returns 200 OK with { "stream": stream object }. If moderation rejects the content or it exceeds the limit, the returned stream has status failed. If the stream has already ended, 409 stream_closed.

Report a failure

Ends the reply when, for example, the model call fails. The status becomes failed (developer); generated content is kept and the client shows "interrupted" with your message.

POST/{org_name}/{app_name}/ai/streams/{stream_id}/fail

Field	Type	Required	Description
code	String	Yes	1–64 characters of letters, digits, _, -, ., such as model_error
message	String	No	Message shown to the user, up to 256 characters

Returns 200 OK with { "stream": stream object }, or 409 stream_closed if the stream has already ended.

Get a stream

GET/{org_name}/{app_name}/ai/streams/{stream_id}

Returns 200 OK with { "stream": stream object }. While generating and within 10 minutes after the end it includes content; after that it does not, and the message body is authoritative (read it with Get a message). Returns 404 not_found if it does not exist.