Skip to main content

Status Tracking

The satellite tracks the health and availability of each MCP server installation through an 11-state status system. This enables real-time monitoring, automatic recovery, and tool availability filtering.

Overview

Status tracking serves three primary purposes:
  1. User Visibility: Users see current server state in real-time via the frontend
  2. Tool Availability: Tools from unavailable servers are filtered from discovery
  3. Automatic Recovery: System detects and recovers from failures automatically
The status system is managed by UnifiedToolDiscoveryManager and updated through:
  • Installation lifecycle events (provisioning → online)
  • Health check results (online → offline)
  • Tool execution failures (online → offline/error/requires_reauth)
  • Configuration changes (online → restarting)
  • Recovery detection (offline → connecting → online)

Status Values

StatusDescriptionTools Available?User Action Required
provisioningInitial state after installation createdNoWait
command_receivedSatellite received configuration commandNoWait
connectingConnecting to MCP serverNoWait
discovering_toolsRunning tool discoveryNoWait
syncing_toolsSyncing tools to backendNoWait
onlineServer healthy and respondingYesNone
restartingConfiguration updated, server restartingNoWait
offlineServer unreachable (auto-recovers)NoWait or check server
errorGeneral error state (auto-recovers)NoCheck logs
requires_reauthOAuth token expired/revokedNoRe-authenticate
permanently_failed3+ crashes in 5 minutes (stdio only)NoManual restart required

Status Lifecycle

Initial Installation Flow

provisioning

command_received (satellite received configure command)

connecting (spawning MCP server process or connecting to HTTP/SSE)

discovering_tools (calling tools/list)

syncing_tools (sending tools to backend)

online (ready for use)

Configuration Update Flow

online

restarting (user updated config, backend sets status immediately)

connecting (satellite receives command, restarts server)

discovering_tools

online

Failure and Recovery Flow

online

offline/error (server unreachable or error response)

[automatic recovery when server comes back]

connecting

discovering_tools

online

OAuth Failure Flow

online

requires_reauth (401/403 response or token refresh failed)

[user re-authenticates via dashboard]

connecting

discovering_tools

online

Stdio Crash Flow (Permanent Failure)

online

(stdio process crashes)

connecting (auto-restart attempt 1)

(crashes again within 5 minutes)

connecting (auto-restart attempt 2)

(crashes again within 5 minutes)

permanently_failed (manual intervention required)

Status Tracking Implementation

UnifiedToolDiscoveryManager

The status system is implemented in UnifiedToolDiscoveryManager:
// services/satellite/src/services/unified-tool-discovery-manager.ts

export type ServerAvailabilityStatus =
  | 'online'
  | 'offline'
  | 'error'
  | 'requires_reauth'
  | 'permanently_failed'
  | 'connecting'
  | 'discovering_tools';

export interface ServerStatusEntry {
  status: ServerAvailabilityStatus;
  lastUpdated: Date;
  message?: string;
}

class UnifiedToolDiscoveryManager {
  private serverStatus: Map<string, ServerStatusEntry> = new Map();

  // Set server status (called by discovery managers and MCP wrapper)
  setServerStatus(serverSlug: string, status: ServerAvailabilityStatus, message?: string): void {
    this.serverStatus.set(serverSlug, {
      status,
      lastUpdated: new Date(),
      message
    });
  }

  // Check if server is available for tool execution
  isServerAvailable(serverSlug: string): boolean {
    const statusEntry = this.serverStatus.get(serverSlug);
    if (!statusEntry) return true; // Unknown = available (safe default)
    return statusEntry.status === 'online';
  }

  // Get all tools, filtered by server status
  getAllTools(): ToolMetadata[] {
    const allTools = this.getAllToolsUnfiltered();
    return allTools.filter(tool => {
      const serverSlug = tool.tool_path.split(':')[0];
      return this.isServerAvailable(serverSlug);
    });
  }
}

Status Callbacks

Discovery managers call status callbacks when discovery succeeds or fails: HTTP/SSE Discovery:
// services/satellite/src/services/remote-tool-discovery-manager.ts

// On successful discovery
this.statusCallback?.(serverSlug, 'online');

// On connection error
const { status, message } = RemoteToolDiscoveryManager.getStatusFromError(error);
this.statusCallback?.(serverSlug, status, message);
Stdio Discovery:
// services/satellite/src/services/stdio-tool-discovery-manager.ts

// On successful discovery
this.statusCallback?.(processId, 'online');

// On discovery error
this.statusCallback?.(processId, 'error', errorMessage);

Tool Filtering by Status

Discovery Filtering

When LLMs call discover_mcp_tools, only tools from available servers are returned:
// UnifiedToolDiscoveryManager.getAllTools() filters by status
const tools = toolDiscoveryManager.getAllTools(); // Only 'online' servers

// Tools from offline/error/requires_reauth servers are hidden

Execution Blocking

When LLMs attempt to execute tools from unavailable servers:
// services/satellite/src/core/mcp-server-wrapper.ts

const serverSlug = toolPath.split(':')[0];
const statusEntry = this.toolDiscoveryManager?.getServerStatus(serverSlug);

// Block execution for non-recoverable states
if (statusEntry?.status === 'requires_reauth') {
  return {
    error: `Tool cannot be executed - server requires re-authentication.

Status: ${statusEntry.status}
The server requires re-authentication. Please re-authorize in the dashboard.

Unavailable server: ${serverSlug}`
  };
}

// Allow execution for offline/error (enables recovery detection)

Status Transition Triggers

Backend-Triggered (Database Updates)

Source: Backend API routes
TriggerNew StatusWhen
Installation createdprovisioningUser installs MCP server
Config updatedrestartingUser modifies environment vars/args/headers
OAuth callback successconnectingUser re-authenticates
Health check failsofflineServer unreachable (3-min interval)
Credential validation failsrequires_reauthOAuth token invalid

Satellite-Triggered (Event Emission)

Source: Satellite emits mcp.server.status_changed events to backend
TriggerNew StatusWhen
Configure command receivedcommand_receivedSatellite polls backend
Server connection startsconnectingSpawning process or HTTP connect
Tool discovery startsdiscovering_toolsCalling tools/list
Tool discovery succeedsonlineDiscovery completed successfully
Tool execution fails (3 retries)offline/error/requires_reauthTool call failed after retries
Server recovery detectedconnectingPreviously offline server responds
Stdio crashes 3 timespermanently_failed3 crashes within 5 minutes

Implementation Components

The status tracking system consists of several integrated components:
  • Database schema for status field
  • Backend event handler for status updates
  • Satellite status event emission
  • Tool availability filtering by status
  • Configuration update status transitions
  • Tool execution status updates with auto-recovery