Docs/Architecture

Architecture

Technical architecture of the Archipelag.io platform

Deep dive into the technical architecture of Archipelag.io, including system design, component responsibilities, and data flows.

Core Architecture (Stable)

These components are production-ready and power the current beta:

System Overview — Components, data flow, tech stack
Components — Detailed component breakdown
Coordinator — Control plane (Phoenix LiveView)
Island — Island software (Rust agent)
Placement & Fit Scoring — How jobs are matched to Islands
Cargos — Container execution model

Experimental Extensions

These features have code and API endpoints, but have not been verified in production. Expect breaking changes during beta.

Multi-Island Compute — Batch fan-out, pipeline parallelism, expert routing, speculative decoding
Workflows — Multi-step DAG pipelines
Inference Cache — Automatic prompt caching
A/B Testing — Traffic splitting between model versions
Federated Fine-Tuning — Distributed training without centralizing data
Confidential Inference — TEE-based encrypted compute

System Overview

High-level architecture of the Archipelag.io distributed compute network

Components

Deep dive into the Archipelag.io system components

Coordinator

Architecture of the Archipelag.io coordinator — the control plane that orchestrates Islands, jobs, and billing

Island

Architecture of the Archipelag.io Island software — the Rust service that executes Cargos on Islands

Placement & Fit Scoring

How the coordinator selects the best Island for each job — multi-dimensional scoring, GPU bandwidth estimation, and runtime-specific strategies

Multi-Island Compute

How Archipelag.io distributes work across multiple Islands — batch fan-out, pipeline parallelism (inference rings), expert routing, and speculative decoding

Workflows

Chain multiple Cargos into multi-step DAG pipelines — transcribe, summarize, translate in a single API call

Inference Cache

Automatic caching of similar prompts — cache hits are instant and free

A/B Testing

Deploy multiple model versions, split traffic between them, and measure which performs best

Cargos

Cargo architecture — runtime types, Island requirements, trust levels, pricing, and the I/O protocol

Federated Fine-Tuning

Train models across distributed Islands without centralizing data — your data never leaves your machines

Confidential Inference

Encrypted compute — Islands cannot see your data, even while processing it