Streaming and tools

Request a stream

curl --no-buffer --fail-with-body https://api.llm.netter.io/v1/chat/completions \
  -H "Authorization: Bearer $NETTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "PUBLIC_MODEL_NAME",
    "messages": [{"role": "user", "content": "Explain the result in three points."}],
    "max_tokens": 1024,
    "stream": true
  }'

Streaming uses SSE in the requested protocol. Consume events through the required completion event, including final usage. An event does not necessarily correspond to a token or a complete message.

The proxy does not collect the entire response before forwarding. Limits count started requests; refusing a new request for excessive traffic must not terminate an established stream.

Errors and interruptions

An error can arrive after streaming begins. Handle in-band errors and incomplete transport. HTTP 200 does not establish successful completion by itself.

Do not reconnect automatically as though it were the same request. Resubmission may duplicate work and charges. Missing final usage or required terminal events can leave an uncertain reservation until provider evidence is checked.

Tool calling

A model may request a tool action. The client performs an authorized action and returns its result in the native format, preserving tool identifiers. Netter does not automatically execute commands on a customer's computer.

PII restoration also covers supported text fields in tool arguments. Your software must still validate permissions, inputs and outputs before acting.

Reasoning and signed artifacts

Preserve opaque signed-thinking and encrypted artifacts and return them unchanged in the supported flow. The Netter envelope binds context and identity. Changing key, user, tenant or model may make replay incompatible.

Switchyard diagnostics

When present, these headers connect routing and cost:

Header	Meaning
x-netter-request-group	Links classification and answer records
x-netter-routing-charge-eur	Classification charge
x-netter-charge-eur	In non-streaming replies, charge including classification and answer

Answer usage is finalized at stream completion. Do not interpret initial headers as a settled final total. Use billing summaries for accounting verification.