Container Deployment

Datafi is distributed as Docker container images for both the coordinator and edge server. You can run these images directly with Docker, orchestrate them with Kubernetes, or deploy them to any container platform that supports OCI-compliant images.

Container Images

Component	Image	Description
Coordinator	datafi/coordinator:latest	Central control plane for catalog, queries, policies, and agents
Edge Server	datafi/edge:latest	Runs close to your data sources, executes queries, returns results

info

Use specific version tags (e.g., datafi/edge:1.12.0) in production rather than latest to ensure reproducible deployments.

Port Configuration

Datafi exposes multiple ports to support its multi-protocol API surface.

Port	Protocol	Service	Description
8000	HTTP	REST API	HTTP endpoint for health checks and REST operations
50051	gRPC	gRPC API	Primary gRPC endpoint for coordinator and edge RPCs
8001	gRPC-Web	gRPC-Web API	Browser-compatible gRPC endpoint
8002	HTTP	MCP Server	Model Context Protocol server for AI assistant integration

Environment Variables

Configure Datafi containers using environment variables.

Variable	Required	Default	Description
MODE	Yes	--	Operating mode: coordinator or edge
EDGE_KEY	Edge only	--	Authentication key for the edge server to connect to the coordinator
OTEL_TOKEN	No	--	OpenTelemetry authentication token for observability
OTEL_ENDPOINT	No	--	OpenTelemetry collector endpoint URL
CACHE_ENABLED	No	true	Enable or disable query result caching
LOG_LEVEL	No	info	Logging verbosity: debug, info, warn, error
TLS_CERT_PATH	No	--	Path to TLS certificate for HTTPS
TLS_KEY_PATH	No	--	Path to TLS private key

Docker Deployment

Running the Coordinator

docker run -d \
  --name datafi-coordinator \
  -p 8000:8000 \
  -p 50051:50051 \
  -p 8001:8001 \
  -p 8002:8002 \
  -e MODE=coordinator \
  -e CACHE_ENABLED=true \
  -e LOG_LEVEL=info \
  datafi/coordinator:latest

Running the Edge Server

docker run -d \
  --name datafi-edge \
  -p 8000:8000 \
  -p 50051:50051 \
  -e MODE=edge \
  -e EDGE_KEY=your-edge-key-here \
  -e CACHE_ENABLED=true \
  datafi/edge:latest

Docker Compose

For local development or single-node deployments, you can use Docker Compose to run both components together.

# docker-compose.yml
version: "3.8"

services:
  coordinator:
    image: datafi/coordinator:latest
    ports:
      - "8000:8000"
      - "50051:50051"
      - "8001:8001"
      - "8002:8002"
    environment:
      MODE: coordinator
      CACHE_ENABLED: "true"
      LOG_LEVEL: info
    restart: unless-stopped

  edge:
    image: datafi/edge:latest
    ports:
      - "9000:8000"
      - "50052:50051"
    environment:
      MODE: edge
      EDGE_KEY: ${EDGE_KEY}
      CACHE_ENABLED: "true"
    depends_on:
      - coordinator
    restart: unless-stopped

# Start the stack
EDGE_KEY=your-edge-key docker compose up -d

Kubernetes Deployment

For production deployments, use Kubernetes to manage scaling, rolling updates, and health checks.

Coordinator Deployment

# coordinator-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: datafi-coordinator
  labels:
    app: datafi-coordinator
spec:
  replicas: 2
  selector:
    matchLabels:
      app: datafi-coordinator
  template:
    metadata:
      labels:
        app: datafi-coordinator
    spec:
      containers:
        - name: coordinator
          image: datafi/coordinator:1.12.0
          ports:
            - containerPort: 8000
              name: http
            - containerPort: 50051
              name: grpc
            - containerPort: 8001
              name: grpc-web
            - containerPort: 8002
              name: mcp
          env:
            - name: MODE
              value: coordinator
            - name: CACHE_ENABLED
              value: "true"
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: 2000m
              memory: 2Gi

Edge Server Deployment

# edge-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: datafi-edge
  labels:
    app: datafi-edge
spec:
  replicas: 2
  selector:
    matchLabels:
      app: datafi-edge
  template:
    metadata:
      labels:
        app: datafi-edge
    spec:
      containers:
        - name: edge
          image: datafi/edge:1.12.0
          ports:
            - containerPort: 8000
              name: http
            - containerPort: 50051
              name: grpc
          env:
            - name: MODE
              value: edge
            - name: EDGE_KEY
              valueFrom:
                secretKeyRef:
                  name: datafi-edge-secret
                  key: edge-key
            - name: CACHE_ENABLED
              value: "true"
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 30
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: 2000m
              memory: 2Gi

Service Configuration

# coordinator-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: datafi-coordinator
spec:
  selector:
    app: datafi-coordinator
  ports:
    - name: http
      port: 8000
      targetPort: 8000
    - name: grpc
      port: 50051
      targetPort: 50051
    - name: grpc-web
      port: 8001
      targetPort: 8001
    - name: mcp
      port: 8002
      targetPort: 8002
  type: ClusterIP

warning

Store the EDGE_KEY in a Kubernetes Secret. Never hardcode credentials in deployment manifests.

Disabling MCP

If you do not need the Model Context Protocol server (port 8002), you can disable it at startup:

docker run -d \
  --name datafi-coordinator \
  -e MODE=coordinator \
  datafi/coordinator:latest --no-mcp

Next Steps

• Azure Deployment Guide -- deploy on Azure Container Instances with Redis and Blob Storage.
• Operations Guide -- health monitoring, graceful shutdown, and rolling updates.