Skip to main content

API Overview

Datafi exposes a multi-protocol API surface that supports gRPC, gRPC-Web, HTTP, and the Model Context Protocol (MCP). All protocols share the same authentication model, authorization pipeline, and governance enforcement.

Multi-Protocol Strategy​

Datafi supports multiple protocols to serve different client types and use cases.

ProtocolPortBest ForTransport
gRPC50051Backend services, high-throughput clientsHTTP/2 + Protobuf
gRPC-Web8001Browser-based applicationsHTTP/1.1 or HTTP/2 + Protobuf
HTTP8000REST clients, health checks, simple integrationsHTTP/1.1 + JSON
MCP8002AI assistants (Claude, ChatGPT, Copilot)HTTP + JSON
info

All four protocols connect to the same coordinator and enforce identical authentication, authorization, and governance policies. The protocol choice does not affect security posture.

Authentication​

All API requests must include a valid JWT Bearer token in the request metadata or headers.

gRPC​

// Include the token in gRPC metadata
metadata: {
  "authorization": "Bearer eyJhbGciOiJSUzI1NiIs..."
}

HTTP​

curl -X POST https://api.datafi.io/v1/query \
  -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIs..." \
  -H "Content-Type: application/json" \
  -d '{"sql": "SELECT * FROM customers LIMIT 10"}'

gRPC-Web​

// gRPC-Web client with authorization metadata
const metadata = {
  'authorization': 'Bearer eyJhbGciOiJSUzI1NiIs...'
};

client.query(request, metadata, (err, response) => {
  // Handle response
});

For details on token lifecycle, identity provider configuration, and JWKS rotation, see the Authentication page.

Response Format​

Datafi returns query results in Apache Arrow columnar format for gRPC and gRPC-Web clients. HTTP clients receive JSON.

Why Apache Arrow?​

BenefitDescription
Columnar layoutEfficient for analytical queries that access a subset of columns
Zero-copy readsClients can process results without deserialization overhead
Language supportArrow libraries available for Python, Java, Go, Rust, JavaScript, and more
CompressionBuilt-in support for LZ4 and ZSTD compression

gRPC Response Example​

message QueryResponse {
  bytes arrow_record_batch = 1;  // Serialized Arrow RecordBatch
  QueryMetadata metadata = 2;
}

message QueryMetadata {
  int64 row_count = 1;
  int64 execution_time_ms = 2;
  string query_id = 3;
  bool from_cache = 4;
}

HTTP Response Example​

{
  "data": [
    {"id": 1, "name": "Alice", "region": "us-west"},
    {"id": 2, "name": "Bob", "region": "us-east"}
  ],
  "metadata": {
    "row_count": 2,
    "execution_time_ms": 142,
    "query_id": "qry_abc123",
    "from_cache": false
  }
}

API Surface​

The Datafi API is divided between the coordinator (central control plane) and the edge server (data plane).

Coordinator API​

The coordinator exposes 81 RPCs organized into four categories:

CategoryDescriptionExample RPCs
CatalogSchema management, records, assetsGetSchema, ListTables, GetRecords
QueryQuery execution, building, validationExecuteQuery, BuildQuery, ValidateQuery
AgentAI agent lifecycle managementCreateAgent, RunAgent, ListAgents
AdminTenant and user administrationGetTenant, UpdateUser, ListRoles

Edge API​

The edge server exposes 3 RPCs with a minimal surface area:

RPCDescription
GetSchemaRetrieve the schema of a connected data source
QueryExecute a query against a connected data source
PingHealth check

Rate Limiting​

API requests are subject to rate limiting based on your plan tier.

TierRequests per MinuteConcurrent Queries
Free605
Pro60025
EnterpriseCustomCustom

Rate-limited requests receive a 429 Too Many Requests response with a Retry-After header.

Error Handling​

All errors follow a consistent format across protocols.

{
  "error": {
    "code": "QUERY_UNAUTHORIZED",
    "message": "Access denied: you do not have permission to query one or more referenced resources.",
    "request_id": "req_7f8a9b2c"
  }
}
Error CodeHTTP StatusDescription
UNAUTHENTICATED401Missing or invalid JWT token
UNAUTHORIZED403Valid token but insufficient permissions
NOT_FOUND404Resource does not exist or is not visible to the tenant
QUERY_UNAUTHORIZED403Query references unauthorized tables or columns
RATE_LIMITED429Too many requests
INTERNAL500Server error (details logged internally)