Skip to main content

Coordinator API

The Datafi coordinator is the central control plane that manages catalog operations, query execution, AI agents, and tenant administration. It exposes 81 RPCs organized into four categories, accessible over gRPC (port 50051), gRPC-Web (port 8001), and HTTP (port 8000).

API Categories​

Catalog Operations​

Catalog operations manage metadata about your data sources, schemas, tables, and assets.

Schema Operations​

RPCDescriptionRequestResponse
GetSchemaRetrieve the full schema for a data sourceGetSchemaRequestSchemaResponse
ListTablesList all tables in a data sourceListTablesRequestListTablesResponse
GetTableGet metadata for a specific tableGetTableRequestTableResponse
ListColumnsList columns for a specific tableListColumnsRequestListColumnsResponse
RefreshSchemaForce a schema refresh from the data sourceRefreshSchemaRequestRefreshSchemaResponse

Example -- Get Schema:

// Request
message GetSchemaRequest {
  string connection_id = 1;
}

// Response
message SchemaResponse {
  string connection_id = 1;
  repeated TableSchema tables = 2;
}

message TableSchema {
  string name = 1;
  repeated ColumnSchema columns = 2;
  int64 row_count = 3;
}

message ColumnSchema {
  string name = 1;
  string data_type = 2;
  bool nullable = 3;
  repeated string tags = 4;
}
# HTTP equivalent
curl https://api.datafi.io/v1/catalog/schemas/conn_abc123 \
  -H "Authorization: Bearer $TOKEN"

Record Operations​

RPCDescription
GetRecordsRetrieve records from a table with pagination
GetRecordByIdRetrieve a single record by primary key
InsertRecordsInsert new records into a table
UpdateRecordsUpdate existing records
DeleteRecordsDelete records by filter criteria

Example -- Get Records:

curl https://api.datafi.io/v1/catalog/records \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_id": "conn_abc123",
    "table": "customers",
    "columns": ["id", "name", "email"],
    "limit": 100,
    "offset": 0,
    "filters": [
      {"column": "region", "operator": "eq", "value": "us-west"}
    ]
  }'

Asset Operations​

RPCDescription
ListAssetsList all registered assets (datasets, documents)
GetAssetRetrieve metadata for a specific asset
CreateAssetRegister a new asset
UpdateAssetUpdate asset metadata or configuration
DeleteAssetRemove an asset
TagAssetAdd or update tags on an asset

Query Operations​

Query operations handle SQL execution, query building, and validation.

Execute​

RPCDescription
ExecuteQueryExecute a SQL query and return results
ExecuteQueryStreamExecute a query and stream results in batches
CancelQueryCancel a running query by ID
GetQueryStatusCheck the status of a running or completed query

Example -- Execute Query:

message ExecuteQueryRequest {
  string sql = 1;
  map<string, string> parameters = 2;
  int32 timeout_seconds = 3;
  string connection_id = 4;
}

message ExecuteQueryResponse {
  bytes arrow_record_batch = 1;
  QueryMetadata metadata = 2;
}
# HTTP
curl -X POST https://api.datafi.io/v1/query/execute \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "sql": "SELECT name, region, SUM(revenue) as total_revenue FROM sales GROUP BY name, region",
    "connection_id": "conn_abc123",
    "timeout_seconds": 30
  }'

Build​

RPCDescription
BuildQueryGenerate SQL from a structured query definition
BuildQueryFromNLGenerate SQL from a natural language question

Example -- Build Query from Natural Language:

curl -X POST https://api.datafi.io/v1/query/build \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "natural_language": "Show me total revenue by region for the last quarter",
    "connection_id": "conn_abc123"
  }'

Response:

{
  "sql": "SELECT region, SUM(revenue) AS total_revenue FROM sales WHERE sale_date >= '2025-01-01' AND sale_date < '2025-04-01' GROUP BY region ORDER BY total_revenue DESC",
  "confidence": 0.94,
  "tables_referenced": ["sales"],
  "columns_referenced": ["region", "revenue", "sale_date"]
}

Validate​

RPCDescription
ValidateQueryCheck SQL syntax and permissions without executing
ExplainQueryReturn the query execution plan
tip

Use ValidateQuery before ExecuteQuery to catch permission errors without consuming query execution resources.

Agent Operations​

Agent operations manage the lifecycle and execution of AI agents.

Lifecycle​

RPCDescription
CreateAgentCreate a new AI agent with a system prompt and tool configuration
GetAgentRetrieve an agent's configuration
ListAgentsList all agents in the tenant or workspace
UpdateAgentModify an agent's configuration
DeleteAgentRemove an agent

Example -- Create Agent:

curl -X POST https://api.datafi.io/v1/agents \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Sales Analyst",
    "description": "Answers questions about sales data",
    "system_prompt": "You are a sales data analyst. Use the available tools to query sales data and provide insights.",
    "connections": ["conn_abc123"],
    "capabilities": ["query", "document_qa"]
  }'

Execution​

RPCDescription
RunAgentSend a message to an agent and receive a response
RunAgentStreamStream an agent's response in real time
GetAgentHistoryRetrieve conversation history for an agent session
CancelAgentRunCancel a running agent execution

Example -- Run Agent:

curl -X POST https://api.datafi.io/v1/agents/agent_xyz/run \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What were our top 5 customers by revenue last quarter?",
    "session_id": "session_001"
  }'

Admin Operations​

Admin operations manage tenants, users, roles, and platform configuration.

Tenant Management​

RPCDescription
GetTenantRetrieve tenant configuration
UpdateTenantUpdate tenant settings
GetTenantUsageRetrieve usage metrics for the tenant

User Management​

RPCDescription
ListUsersList all users in the tenant
GetUserRetrieve a user's profile and roles
UpdateUserUpdate a user's roles or attributes
DeactivateUserDeactivate a user account
ListRolesList available roles
AssignRoleAssign a role to a user
RevokeRoleRemove a role from a user
warning

Admin operations require the admin or super_admin role. Attempting to call admin RPCs with insufficient permissions results in a 403 Forbidden response.

Pagination​

List operations support cursor-based pagination.

{
  "page_size": 50,
  "page_token": "eyJsYXN0X2lkIjogMTAwfQ=="
}

The response includes a next_page_token if more results are available:

{
  "items": [...],
  "next_page_token": "eyJsYXN0X2lkIjogMTUwfQ==",
  "total_count": 342
}