Skip to content

Deploy

AgentServer is a drop-in FastAPI wrapper. It deploys anywhere FastAPI runs. This guide covers the most common targets: a container, Kubernetes, serverless, and a plain VM. In every case the agent authenticates to its model provider with an API key supplied via an environment variable — no cloud-specific identity wiring required.

The shape you ship

# server.py
from tulip.agent import Agent
from tulip.server import AgentServer
from tulip.memory.backends import S3Backend

agent = Agent(
    model="anthropic:claude-sonnet-4-6",
    tools=[...],
    system_prompt="...",
    checkpointer=S3Backend(bucket="tulip-threads"),  # or RedisBackend(...)
)

server = AgentServer(
    agent=agent,
    title="Booking concierge",
    api_key="...",   # require this bearer token on every route except /health
)

if __name__ == "__main__":
    server.run(host="0.0.0.0", port=8080)

You get out of the box:

  • POST /invoke — synchronous run, returns the final message plus run metrics (success, stop_reason, iterations, tool_calls, duration_ms) as JSON.
  • POST /stream — Server-Sent Events of every typed event.
  • GET / DELETE /threads/{id} — conversation persistence.
  • GET /health — liveness probe.

Provider keys are read from the environment (OPENAI_API_KEY, ANTHROPIC_API_KEY). Inject them as secrets, never bake them into the image.

Container — the universal target

The repo ships a multi-stage Dockerfile (non-root user, HEALTHCHECK on /health). Build, push to any registry, and run anywhere that runs containers:

docker build -t registry.example.com/tulip-concierge:0.1.0 .
docker push    registry.example.com/tulip-concierge:0.1.0

docker run -p 8080:8080 \
  -e OPENAI_API_KEY=sk-... \
  registry.example.com/tulip-concierge:0.1.0

This single image drops straight into Cloud Run, ECS / Fargate, Fly.io, Azure Container Apps, or any other container host.

Serverless — scale to zero

Best for low-frequency or bursty traffic. Wrap the FastAPI app in an adapter for your platform — Mangum for AWS Lambda, or deploy the container image directly to a scale-to-zero container runtime (Cloud Run, Container Apps). Agent is constructed lazily, so cold starts stay cheap. Set the provider key as a function secret.

Kubernetes — for production

Best for multi-replica, autoscaled, multi-region production. A minimal deployment:

apiVersion: apps/v1
kind: Deployment
metadata: { name: concierge }
spec:
  replicas: 3
  selector: { matchLabels: { app: concierge } }
  template:
    metadata: { labels: { app: concierge } }
    spec:
      containers:
      - name: concierge
        image: registry.example.com/tulip-concierge:0.1.0
        ports: [{ containerPort: 8080 }]
        env:
        - name: OPENAI_API_KEY
          valueFrom: { secretKeyRef: { name: tulip-secrets, key: openai-api-key } }
        readinessProbe:
          httpGet: { path: /health, port: 8080 }
        resources:
          requests: { cpu: 500m, memory: 1Gi }
          limits:   { cpu: 2,    memory: 4Gi }
---
apiVersion: v1
kind: Service
metadata: { name: concierge }
spec:
  type: LoadBalancer
  selector: { app: concierge }
  ports: [{ port: 80, targetPort: 8080 }]

For SSE streaming, ensure your ingress / load balancer doesn't buffer the response (X-Accel-Buffering: no on nginx, or the buffering-off equivalent on your load balancer).

Plain VM — full control

Best when you need raw VM access or run the agent alongside other local services.

pip install "tulip-agents[openai,server]"
git clone https://github.com/tuliplabs-ai/sdk-python.git ~/concierge
cd ~/concierge

# Launch under systemd
sudo tee /etc/systemd/system/concierge.service <<'EOF'
[Unit]
Description=Tulip concierge agent
After=network.target

[Service]
Type=simple
User=app
Environment=OPENAI_API_KEY=sk-...
ExecStart=/home/app/.local/bin/uvicorn server:app --host 0.0.0.0 --port 8080
Restart=always

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl enable --now concierge

Sessions — thread_id for chat UIs

When the underlying agent has a checkpointer, pass a thread_id in the request body for cross-request continuity. Same browser tab → same thread_id → same context. Omit it, and each request starts fresh.

POST /invoke
Content-Type: application/json

{"prompt": "What were we discussing?", "thread_id": "user-c42-support"}

When api_key is set, the authenticated principal is prefixed onto the thread_id server-side, so threads are scoped to the caller that owns the key. The server takes a single shared api_key, so this is single-principal scoping — not per-tenant isolation.

Observability

Wire TelemetryHook to your OTLP collector for traces and metrics. Set the exporter target via the standard OpenTelemetry environment variables before the agent starts:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc

from tulip.hooks.builtin import TelemetryHook

agent = Agent(
    ...,
    hooks=[TelemetryHook(service_name="my-agent")],
)

Datadog accepts OTLP. So do Honeycomb, Tempo, Grafana Cloud, and every other backend that speaks the spec. See Observability.

See also