Skip to main content

Status Tracking

The satellite tracks the health and availability of each MCP server instance (per-user) through a 12-state status system. This enables real-time monitoring, automatic recovery, and tool availability filtering on a per-user basis.

Overview

Per-User Instance Status

DeployStack tracks status at the instance level, not the installation level:
  • Status Location: mcpServerInstances table (per user)
  • No Installation Status: Status fields completely removed from mcpServerInstallations table
  • Independent Tracking: Each team member has independent status for each MCP server
  • User-Specific Filtering: Users see only tools from their OWN instances that are online
Example:
Team "Acme Corp" installs Filesystem MCP with 3 members:
- Alice's instance: status = 'online' (tools available to Alice)
- Bob's instance: status = 'awaiting_user_config' (tools NOT available to Bob)
- Charlie's instance: status = 'offline' (tools NOT available to Charlie)

Each member sees different tool availability based on their instance status.

Status Tracking Purposes

Status tracking serves three primary purposes:
  1. User Visibility: Users see their OWN instance status in real-time via the frontend
  2. Tool Availability: Tools from user’s unavailable instances are filtered from discovery
  3. Automatic Recovery: System detects and recovers from failures per instance
The status system is managed by UnifiedToolDiscoveryManager and updated through:
  • Instance lifecycle events (provisioning → online)
  • Health check results (online → offline)
  • Tool execution failures (online → offline/error/requires_reauth)
  • Configuration changes (online → restarting)
  • Recovery detection (offline → connecting → online)
  • User configuration completion (awaiting_user_config → provisioning)

Status Values

StatusDescriptionTools Available?User Action Required
awaiting_user_configUser hasn’t configured required user-level fieldsNoConfigure personal settings
provisioningInitial state after instance createdNoWait
command_receivedSatellite received configuration commandNoWait
connectingConnecting to MCP serverNoWait
discovering_toolsRunning tool discoveryNoWait
syncing_toolsSyncing tools to backendNoWait
onlineServer healthy and respondingYesNone
restartingConfiguration updated, server restartingNoWait
offlineServer unreachable (auto-recovers)NoWait or check server
errorGeneral error state (auto-recovers)NoCheck logs
requires_reauthOAuth token expired/revokedNoRe-authenticate
permanently_failed3+ crashes in 5 minutes (stdio only)NoManual restart required
New Status: awaiting_user_configThis status indicates that an MCP server has required user-level configuration fields (e.g., personal API keys) and the user hasn’t configured them yet. The satellite does NOT spawn processes for instances with this status. Once the user completes their configuration via the dashboard, the status automatically transitions to provisioning and the instance spawns normally.See Instance Lifecycle - Process A for details.

Status Lifecycle

Initial Installation Flow (User Has Complete Config)

provisioning

command_received (satellite received configure command)

connecting (spawning MCP server process or connecting to HTTP/SSE)

discovering_tools (calling tools/list)

syncing_tools (sending tools to backend)

online (ready for use)

Initial Installation Flow (User Missing Required Config)

When an MCP server has required user-level configuration fields and the user hasn’t configured them:
awaiting_user_config

[User configures personal settings via dashboard]

provisioning (backend updates status, sends satellite command)

command_received

connecting

discovering_tools

syncing_tools

online
Backend Filtering: The satellite does NOT receive configurations for instances with awaiting_user_config status. Backend filters these instances out in the config endpoint, preventing spawn attempts for incomplete configurations.

Configuration Update Flow

online

restarting (user updated config, backend sets status immediately)

connecting (satellite receives command, restarts server)

discovering_tools

online

Failure and Recovery Flow

online

offline/error (server unreachable or error response)

[automatic recovery when server comes back]

connecting

discovering_tools

online

OAuth Failure Flow

online

requires_reauth (401/403 response or token refresh failed)

[user re-authenticates via dashboard]

connecting

discovering_tools

online

Stdio Crash Flow (Permanent Failure)

online

(stdio process crashes)

connecting (auto-restart attempt 1)

(crashes again within 5 minutes)

connecting (auto-restart attempt 2)

(crashes again within 5 minutes)

permanently_failed (manual intervention required)

Status Tracking Implementation

UnifiedToolDiscoveryManager

The status system is implemented in UnifiedToolDiscoveryManager:
// services/satellite/src/services/unified-tool-discovery-manager.ts

export type ServerAvailabilityStatus =
  | 'online'
  | 'offline'
  | 'error'
  | 'requires_reauth'
  | 'permanently_failed'
  | 'connecting'
  | 'discovering_tools';

export interface ServerStatusEntry {
  status: ServerAvailabilityStatus;
  lastUpdated: Date;
  message?: string;
}

class UnifiedToolDiscoveryManager {
  private serverStatus: Map<string, ServerStatusEntry> = new Map();

  // Set server status (called by discovery managers and MCP wrapper)
  setServerStatus(serverSlug: string, status: ServerAvailabilityStatus, message?: string): void {
    this.serverStatus.set(serverSlug, {
      status,
      lastUpdated: new Date(),
      message
    });
  }

  // Check if server is available for tool execution
  isServerAvailable(serverSlug: string): boolean {
    const statusEntry = this.serverStatus.get(serverSlug);
    if (!statusEntry) return true; // Unknown = available (safe default)
    return statusEntry.status === 'online';
  }

  // Get all tools, filtered by server status
  getAllTools(): ToolMetadata[] {
    const allTools = this.getAllToolsUnfiltered();
    return allTools.filter(tool => {
      const serverSlug = tool.tool_path.split(':')[0];
      return this.isServerAvailable(serverSlug);
    });
  }
}

Status Callbacks

Discovery managers call status callbacks when discovery succeeds or fails: HTTP/SSE Discovery:
// services/satellite/src/services/remote-tool-discovery-manager.ts

// On successful discovery
this.statusCallback?.(serverSlug, 'online');

// On connection error
const { status, message } = RemoteToolDiscoveryManager.getStatusFromError(error);
this.statusCallback?.(serverSlug, status, message);
Stdio Discovery:
// services/satellite/src/services/stdio-tool-discovery-manager.ts

// On successful discovery
this.statusCallback?.(processId, 'online');

// On discovery error
this.statusCallback?.(processId, 'error', errorMessage);

Tool Filtering by Status

Per-User Instance Filtering

Tool availability is filtered based on the authenticated user’s OWN instance status: Key Principles:
  • Each user sees only tools from their own instances that are online
  • Other team members’ instance status does NOT affect your tool availability
  • If your instance is awaiting_user_config, you see NO tools from that server
  • If your instance is online, you see all tools (even if teammates’ instances are offline)
Example:
Team "Acme Corp" - Context7 MCP Server:
- Alice's instance: status = 'online'
  → Alice sees Context7 tools in discover_mcp_tools

- Bob's instance: status = 'awaiting_user_config'
  → Bob sees NO Context7 tools (must configure API key first)

- Charlie's instance: status = 'offline'
  → Charlie sees NO Context7 tools (server unreachable)

Discovery Filtering

When LLMs call discover_mcp_tools, only tools from the user’s available instances are returned:
// UnifiedToolDiscoveryManager.getAllTools() filters by user's instance status
const tools = toolDiscoveryManager.getAllTools(); // Only user's 'online' instances

// Tools from user's offline/error/requires_reauth/awaiting_user_config instances are hidden

Execution Blocking

When LLMs attempt to execute tools from the user’s unavailable instances:
// services/satellite/src/core/mcp-server-wrapper.ts

const serverSlug = toolPath.split(':')[0];
const statusEntry = this.toolDiscoveryManager?.getServerStatus(serverSlug);

// Block execution for non-recoverable states
if (statusEntry?.status === 'requires_reauth') {
  return {
    error: `Tool cannot be executed - your instance requires re-authentication.

Status: ${statusEntry.status}
Your instance requires re-authentication. Please re-authorize in the dashboard.

Unavailable server: ${serverSlug}`
  };
}

// Allow execution for offline/error (enables recovery detection)

Status Transition Triggers

Backend-Triggered (Database Updates)

Source: Backend API routes update mcpServerInstances table (per user)
TriggerNew StatusWhen
Instance createdprovisioning or awaiting_user_configAdmin installs MCP server or member added to team
User config updatedprovisioning (if was awaiting_user_config) or restarting (if already online)User modifies their personal config
Config updatedrestartingUser modifies environment vars/args/headers/query params
OAuth callback successconnectingUser re-authenticates
Health check failsofflineServer unreachable (3-min interval)
Credential validation failsrequires_reauthOAuth token invalid
Status Target: All backend status updates target the mcpServerInstances table. Status fields have been completely removed from mcpServerInstallations. Each user’s instance has independent status tracking.

Satellite-Triggered (Event Emission)

Source: Satellite emits mcp.server.status_changed events to backend (includes user_id field)
TriggerNew StatusWhen
Configure command receivedcommand_receivedSatellite polls backend
Server connection startsconnectingSpawning process or HTTP connect
Tool discovery startsdiscovering_toolsCalling tools/list
Tool discovery succeedsonlineDiscovery completed successfully
Tool execution fails (3 retries)offline/error/requires_reauthTool call failed after retries
Server recovery detectedconnectingPreviously offline server responds
Stdio crashes 3 timespermanently_failed3 crashes within 5 minutes
Event Payload:
{
  "event": "mcp.server.status_changed",
  "installation_id": "uuid",
  "team_id": "uuid",
  "user_id": "uuid",
  "status": "online",
  "status_message": "Server is online and ready"
}
The user_id field ensures status updates are applied to the correct user’s instance.

Implementation Components

The status tracking system consists of several integrated components:
  • Database schema for status field
  • Backend event handler for status updates
  • Satellite status event emission
  • Tool availability filtering by status
  • Configuration update status transitions
  • Tool execution status updates with auto-recovery