smooth-operator · the service

A chat loop is a weekend project.
An agent you'd put near production is not.

smooth-operator remembers the whole conversation, retrieves only what the person asking is allowed to see, streams its reasoning as it works — and stops to ask you before it writes anything. One service, one wire protocol, native clients in four languages. MIT-licensed, and yours to run.

This is one layer of the Smoo AI platform, not the whole of it — the open-source agent service that Smoo AI's own AI teammates run on. The products sitting above it — CRM, support, campaigns, field service, analytics, and a dozen more — are a different and much larger story.

MIT licensedKubernetes · serverless · localBring your own modelYou approve every write

Quickstart SmooAI/smooth-operator

Engine below, protocol above

The service is the part that has to survive contact with users.

The agent orchestration itself lives in smooth-operator-core, a five-language engine. smooth-operator is what wraps it: conversations that persist, knowledge ingestion and retrieval, a tool catalog with the gates around it, and one schema-driven WebSocket protocol that native clients speak.

Everything a real deployment argues about — where state lives, how replicas talk, who the caller is — is a named seam selected by configuration. That is the whole architectural bet, and it is why one binary runs three very different ways.

What it is not is a product. Smoo AI is sixteen product areas — CRM, support, campaigns, field service, booking, e-sign, analytics, observability, security, workforce, config, testing, the LLM gateway, the content builder, website management, and the AI agents themselves. smooth-operator is the layer those agents run on, open-sourced. Everything that makes an agent useful to a business — the records it reads, the inbox it answers, the entitlements it respects — lives in the products above it.

Quickstart

Two ways in. Both end at a streaming agent.

Docker boots the whole stack — Postgres with pgvector, the operator server, and a React chat UI — with a human-in-the-loop approval you click yourself. From source gets you the same agent with no database and no Docker at all.

Docker — the fastest path

No Rust toolchain. First run builds the server image; after that it is cached.

$ git clone https://github.com/SmooAI/smooth-operator
$ cd smooth-operator/examples
$ cp .env.example .env  # set SMOOAI_GATEWAY_KEY
$ cd web-chat && docker compose up --build
# chat UI on http://localhost:8080

Any OpenAI-compatible /v1 gateway works. Prefer a terminal? examples/tui-chat drives the same stack.

From source — fully in-memory

No database, no auth, no AWS. The first compile takes minutes; after that, seconds.

$ cd smooth-operator/rust
$ export SMOOAI_GATEWAY_KEY=sk-…
$ export SMOOTH_AGENT_SEED_KB=1
$ cargo run -p smooai-smooth-operator-server
# listening on ws://127.0.0.1:8787/ws

With no env set the binary boots the local flavor: in-memory storage and backplane, loopback bind, admin off. Without a gateway key it still boots and answers protocol actions — only send_message errors cleanly.

Set SMOOTH_AGENT_STORAGE=postgres and a backplane, and the same binary graduates to the Kubernetes flavor. There is no second build.

Three flavors, one codebase

The flavor is a config value, not a build flag.

Nothing in the application code names a backend. Pick storage, backplane, and auth from the environment, and the operator becomes a k8s service, a Lambda, or a laptop process — with the same agent behavior on all three.

Kubernetes

The primary self-host target — long-running pods that scale out.

Compute: Long-running pods
Storage: Postgres + pgvector
Backplane: Redis / Valkey or NATS
SMOOTH_AGENT_STORAGE: postgres

helm install smooth-operator ./deploy/k8s

AWS serverless

API Gateway WebSocket in front of Lambda, deployed with SST.

Compute: API GW WebSocket → Lambda
Storage: DynamoDB + S3 Vectors
Backplane: API GW connections
SMOOTH_AGENT_STORAGE: dynamodb

cd deploy/sst && npx sst deploy

Local

One in-process server. No database, no auth, no AWS, no Docker.

Compute: One in-process server
Storage: In-memory
Backplane: In-memory (single process)
SMOOTH_AGENT_STORAGE: memory (default)

cargo run -p smooai-smooth-operator-server

What survives every flavor: the same Chat · RAG · Agents · Actions decomposition, connector-style ingestion, document-level ACLs over org isolation, and the batteries-included MIT self-host story.

One sharp edge worth knowing before you deploy: the Helm chart's values render bind, port, gateway URL, model, and limits — but SMOOTH_AGENT_STORAGE, SMOOTH_AGENT_BACKPLANE, and the auth mode have no chart values of their own and must go through server.extraEnv. The container image also overrides the loopback bind default to 0.0.0.0, which is what makes the pod reachable at all.

The seams

Three interfaces are doing all the work.

Storage, backplane, auth. Everything above them is written once. This is the difference between “we support Kubernetes and serverless” and two codebases quietly drifting apart.

StorageAdapter

SMOOTH_AGENT_STORAGE

Conversations, messages, and knowledge sit behind one trait. memory, postgres, or dynamodb — application code never names a backend, so the same agent code reads and writes on all three.

dynamodb is implemented in the Rust server only.

Backplane

SMOOTH_AGENT_BACKPLANE

How replicas find each other to fan a stream back to the connection that asked for it. memory (single process), redis / valkey, or nats. Set it the moment you run more than one replica.

Rust server only — the other four hosts do not read this variable.

Auth verifier

SMOOTH_AGENT_AUTH_MODE

jwt (bring your own issuer), smoo (Smoo AI identity), trusted (identity from a proxy you control), or none for local dev. The verifier is chosen at boot; nothing downstream branches on it.

AUTH_MODE is a deprecated alias. Read from the environment by the Rust and .NET hosts; the TS, Go, and Python servers take an injected verifier in code.

The protocol

One turn, from ack to answer — with a stop built in.

A client connects to /ws, opens a session, and sends one action. Everything after that is a stream of typed events defined once in spec/ as JSON Schema. The interesting part is the frame in the middle: the turn can park.

Actions you send

Client → server. Every frame carries a requestId you can correlate on.

create_conversation_session
Open a session against an agent. Returns the sessionId every later frame carries.
send_message
Start one turn. The server answers with an immediate ack, then streams.
confirm_tool_action
Approve or refuse a write the agent parked on.
get_session
Read back the current state of a session.
get_conversation_messages
Page through history. (The docs table calling this get_messages is stale — the wire string is the long one.)
cancel
Stop an in-flight turn. Implemented by all five servers.
verify_otp
Answer an OTP challenge raised mid-turn by an auth-gated tool.
ping
Liveness.

A ninth action, submit_interaction, backs richer in-chat inputs but is currently implemented by the Rust server only — the other four answer UNSUPPORTED_ACTION.

Events you receive

Server → client. Status is the whole summary: 202 in progress, 200 final, 4xx/5xx a typed error.

immediate_response202
The ack. Your turn was accepted and is running.
stream_chunk
One workflow node finished. Carries node — knowledge_search, response_gen, …
stream_token
A token delta. This is the text your UI paints.
write_confirmation_required
The turn is parked. A tool wants to write and is waiting on a human.
otp_verification_required
An auth-gated tool needs the caller verified before it will run.
eventual_response200
The authoritative final state — message, citations, cost, tokens.
error4xx/5xx
Typed code plus a message — SESSION_NOT_FOUND, RATE_LIMITED, VALIDATION_ERROR.

The Rust server additionally emits stream_reasoning, stream_preamble, and interaction_required. Build UI that ignores event types it does not recognise — that is the forward-compatible posture the spec assumes.

Human in the loop

The turn stops. It does not guess, and it does not proceed.

When a tool wants to write, the server emits write_confirmation_required and the turn parks — mid-stream, holding its state. Nothing has happened yet. You answer with confirm_tool_action, and the resumed stream flows back into the same turn handle your loop is already iterating. Refuse, and the agent is told so and carries on without it.

Which tools park is yours to declare — SMOOTH_AGENT_CONFIRM_TOOLS takes a comma-separated list matched as substrings against tool names, so delete_,send_ covers a whole family at once. Two of the ten shared conformance scenarios exist purely to pin this behavior, approved and denied.

the frame your UI reacts to — actionDescription is what you show a human

{
  "type": "write_confirmation_required",
  "requestId": "req_…",                  // echo this back on confirm_tool_action
  "data": {
    "requestId": "req_…",
    "data": {
      "toolId": "…",                     // opaque, for correlation
      "actionDescription": "Delete contact John Doe ([email protected])"
    }
  }
}

Watch it stream

Native clients, not thin REST wrappers.

Connect, open a session, send a turn, iterate the stream — and await the same handle for the authoritative final state. The shape is deliberately identical in every language, including the branch that handles a parked write.

npm i @smooai/smooth-operator

import { SmoothAgentClient } from '@smooai/smooth-operator';

const client = new SmoothAgentClient({ url: 'ws://127.0.0.1:8787/ws' });
await client.connect();

const session = await client.createConversationSession({ agentId, userName: 'Alice' });

// One turn. Iterate for the stream; await the same handle for the final state.
const turn = client.sendMessage({ sessionId: session.sessionId, message: 'How long is your return window?' });

for await (const ev of turn) {
    if (ev.type === 'stream_chunk') console.error(`  node: ${ev.node}`);
    if (ev.type === 'stream_token') process.stdout.write(ev.token ?? '');
    if (ev.type === 'write_confirmation_required') {
        // The turn is parked. Approve, and the resumed stream flows back into this same loop.
        client.confirmToolAction({ sessionId: session.sessionId, requestId: turn.requestId, approved: true });
    }
}

const final = await turn;
console.log(final.data.data.messageId, final.data.data.citations);

Where the packages live

@smooai/smooth-operator — npm, ESM-only, Node 22+
smooai-smooth-operator — PyPI, imported as smooth_operator
github.com/SmooAI/smooth-operator/go — the client is the protocol package
SmooAI.SmoothOperator — ships in-repo today, not yet on NuGet; add it as a project reference. It also carries a Microsoft.Extensions.AI IChatClient facade.

There is no Rust protocol client

Rust's role in this repo is the other end of the wire. smooai-smooth-operator is the service runtime you embed and smooai-smooth-operator-server is the reference server — both on crates.io. If you want a Rust program to talk to an operator, you are writing WebSocket frames against spec/ yourself today.

On the web

A React binding and an embeddable widget, on the same client.

The TypeScript package ships both as subpath exports — a React binding that hands you the turn state as hooks, and a custom element you can drop into any page. Neither is a re-implementation: they sit on the same SmoothAgentClient, so streaming, citations, and the parked-write branch behave identically.

import { SmoothAgentClient } from '@smooai/smooth-operator';       // core client
import { … } from '@smooai/smooth-operator/react';                 // React binding
import '@smooai/smooth-operator/widget';                           // custom element

Smoo AI's own production chat widget is a separate, standalone package — @smooai/chat-widget — that speaks this same protocol against wss://ai.smoo.ai/ws.

Grounding

Retrieval that shows its work.

On the Postgres backend, a query runs two arms in parallel — dense pgvector cosine over an HNSW index, and sparse tsvector keyword ranking — and fuses them with Reciprocal Rank Fusion. ACLs are applied in SQL before fusion, so an entitlement failure never becomes a citation.

What you actually get, per backend

postgres (Rust)

Dense + sparse + RRF fusion, ACL-filtered in SQL. The full hybrid path.

dynamodb (Rust)

S3 Vectors — dense retrieval.

memory (default)

Lexical keyword scoring only. Fine for a demo; not a search engine.

Rerank is opt-in and off by default. Set SMOOTH_AGENT_RERANK=gateway for a real cross-encoder (Cohere or Voyage through your gateway), or lexical for an offline coverage heuristic. Unset means no rerank stage at all — including in the Docker quickstart.

Your gateway key quietly picks the embedder too: with a key you get real semantic embeddings; without one, a deterministic hash embedder that keeps tests green but is not semantic.

Structured citations

Collected from the retrieval that actually happened — auto-context plus any knowledge_search tool results, deduped by document. They ride the final frame.

// eventual_response.data.data.citations[]
{
  "id":      "doc_9f3a…",   // knowledge-base document id
  "title":   "Returns policy",
  "url":     "https://…",   // when the source parses as http(s)
  "snippet": "…17 days…",   // the chunk, truncated
  "score":   0.83           // similarity
}

Durable checkpoints

The engine checkpoints each step of a turn, so a crashed process resumes rather than restarting from the user's first message. In-memory and file stores need no features; SQLite and Postgres stores are cargo features on the engine crate, and both are held to the same conformance suite against real database engines.

Full detail lives on the engine page — checkpointing is an engine concern the service inherits.

More than two people

A conversation is not a user and a bot. Every participant carries a type, and a human agent can join a thread the AI has been handling without the transcript changing shape — which is what makes escalation a seat change instead of a migration.

userai-agenthuman-agent

Kebab-case, not underscores. The values are pinned by the JSON Schema, by each language's types, and by a CHECK constraint in the Postgres schema.

Extensible — and bounded

Give it your tools. Then declare what it may never touch.

Install a tool provider and the runner merges your tools with the built-ins for every turn, scoped to that turn's org. Every tool then flows through the same gates — built-in, host-provided, or from an extension — so the guardrails hold no matter where a tool came from.

Per-agent allow-list

An agent’s tool_config.enabledTools restricts its turn to exactly those tools. Off the list, off the table — including tools the runner would otherwise merge in from a host or an extension.

Auth-level ToolHook

A tool tagged admin or end_user is blocked at call time on a public agent unless the caller is verified — the session’s OTP bit, or your own SessionAuthenticator. The hook runs before the tool, and fails closed.

Document-level ACLs

Both retrieval arms read through the storage adapter’s access-scoped view. On Postgres the ACL filter is applied in SQL before fusion — a document the caller isn’t entitled to is gone before it can reach the model or land in a citation.

Extensions are default-deny

Out-of-process SEP tool providers contribute tools only if you name them in SMOOTH_EXTENSIONS_ALLOW. Unset means the extension host is never even built — zero processes spawned. Their ui/confirm prompts bridge into the same confirmation frames.

The deeper permission model — modes, circuit-breakers, and a declarative deny-policy no prompt can talk its way past — lives in the engine. Read that story next.

Parity you can check

Five servers. One corpus. No honor system.

There are five server implementations — Rust, C#, Python, TypeScript, and Go — so a host can run the whole service in its native stack. “They behave the same” would be a claim; instead it is a test. All five run the same ten-scenario conformance corpus in spec/conformance/scenarios— language-neutral protocol flows driven by the engine's deterministic mock, so every server must produce identical frames from identical input.

Every runner globs the directory at runtime

A new scenario JSON is picked up by all five languages with no wiring.

Every language CI workflow triggers on spec/**

Touching the corpus fires all five lanes on the same commit.

No skips

No ignore attributes, no env gates, no build tags on any of the five runners.

It has caught real bugs

The corpus found and fixed error-handling divergences in the TypeScript and C# servers.

Scope note, so the claim stays honest: the corpus tests servers. Each runner drives its own in-language test client, so it is not a client×server cross-product, and the live gateway-backed suites are gated on credentials rather than run on every commit. Rust and C# carry the full surface today — ingestion, admin, ACLs, storage adapters; the TypeScript, Python, and Go servers are native protocol hosts (transport, frame dispatch, per-turn engine, sessions, auth, graceful drain).

Run it yourself — or let us run it.

smooth-operator is MIT-licensed and self-hostable end to end. The same service also runs in production underneath Smoo AI's AI teammates, behind wss://ai.smoo.ai/ws — there wired into a whole platform of CRM, inbox, knowledge, and entitlements that this repo does not contain. A standalone managed offering for the service itself is on the roadmap, not shipped.

Read the source The engine underneath Talk to your business

Built in the open, test-first. The repo's docs/ vault carries the protocol reference, storage designs, ingestion, access control, and the roadmap that separates shipped from queued.

A chat loop is a weekend project.An agent you'd put near production is not.