Status Tracking

The satellite tracks the health and availability of each MCP server instance (per-user) through a 12-state status system. This enables real-time monitoring, automatic recovery, and tool availability filtering on a per-user basis.

Overview

Per-User Instance Status

DeployStack tracks status at the instance level, not the installation level:

Status Location: mcpServerInstances table (per user)
No Installation Status: Status fields completely removed from mcpServerInstallations table
Independent Tracking: Each team member has independent status for each MCP server
User-Specific Filtering: Users see only tools from their OWN instances that are online

Example:

Team "Acme Corp" installs Filesystem MCP with 3 members:
- Alice's instance: status = 'online' (tools available to Alice)
- Bob's instance: status = 'awaiting_user_config' (tools NOT available to Bob)
- Charlie's instance: status = 'offline' (tools NOT available to Charlie)

Each member sees different tool availability based on their instance status.

Status Tracking Purposes

Status tracking serves three primary purposes:

User Visibility: Users see their OWN instance status in real-time via the frontend
Tool Availability: Tools from user’s unavailable instances are filtered from discovery
Automatic Recovery: System detects and recovers from failures per instance

The status system is managed by UnifiedToolDiscoveryManager and updated through:

Instance lifecycle events (provisioning → online)
Health check results (online → offline)
Tool execution failures (online → offline/error/requires_reauth)
Configuration changes (online → restarting)
Recovery detection (offline → connecting → online)
User configuration completion (awaiting_user_config → provisioning)

Status Values

Status	Description	Tools Available?	User Action Required
`awaiting_user_config`	User hasn’t configured required user-level fields	No	Configure personal settings
`provisioning`	Initial state after instance created	No	Wait
`command_received`	Satellite received configuration command	No	Wait
`connecting`	Connecting to MCP server	No	Wait
`discovering_tools`	Running tool discovery	No	Wait
`syncing_tools`	Syncing tools to backend	No	Wait
`online`	Server healthy and responding	Yes	None
`restarting`	Configuration updated, server restarting	No	Wait
`offline`	Server unreachable (auto-recovers)	No	Wait or check server
`error`	General error state (auto-recovers)	No	Check logs
`requires_reauth`	OAuth token expired/revoked	No	Re-authenticate
`permanently_failed`	3+ crashes in 5 minutes (stdio only)	No	Manual restart required

New Status: awaiting_user_configThis status indicates that an MCP server has required user-level configuration fields (e.g., personal API keys) and the user hasn’t configured them yet. The satellite does NOT spawn processes for instances with this status. Once the user completes their configuration via the dashboard, the status automatically transitions to provisioning and the instance spawns normally.See Instance Lifecycle - Process A for details.

Status Lifecycle

Initial Installation Flow (User Has Complete Config)

provisioning
    ↓
command_received (satellite received configure command)
    ↓
connecting (spawning MCP server process or connecting to HTTP/SSE)
    ↓
discovering_tools (calling tools/list)
    ↓
syncing_tools (sending tools to backend)
    ↓
online (ready for use)

Initial Installation Flow (User Missing Required Config)

When an MCP server has required user-level configuration fields and the user hasn’t configured them:

awaiting_user_config
    ↓
[User configures personal settings via dashboard]
    ↓
provisioning (backend updates status, sends satellite command)
    ↓
command_received
    ↓
connecting
    ↓
discovering_tools
    ↓
syncing_tools
    ↓
online

Backend Filtering: The satellite does NOT receive configurations for instances with awaiting_user_config status. Backend filters these instances out in the config endpoint, preventing spawn attempts for incomplete configurations.

Configuration Update Flow

online
    ↓
restarting (user updated config, backend sets status immediately)
    ↓
connecting (satellite receives command, restarts server)
    ↓
discovering_tools
    ↓
online

Failure and Recovery Flow

online
    ↓
offline/error (server unreachable or error response)
    ↓
[automatic recovery when server comes back]
    ↓
connecting
    ↓
discovering_tools
    ↓
online

OAuth Failure Flow

online
    ↓
requires_reauth (401/403 response or token refresh failed)
    ↓
[user re-authenticates via dashboard]
    ↓
connecting
    ↓
discovering_tools
    ↓
online

Stdio Crash Flow (Permanent Failure)

online
    ↓
(stdio process crashes)
    ↓
connecting (auto-restart attempt 1)
    ↓
(crashes again within 5 minutes)
    ↓
connecting (auto-restart attempt 2)
    ↓
(crashes again within 5 minutes)
    ↓
permanently_failed (manual intervention required)

Status Tracking Implementation

UnifiedToolDiscoveryManager

The status system is implemented in UnifiedToolDiscoveryManager:

// services/satellite/src/services/unified-tool-discovery-manager.ts

export type ServerAvailabilityStatus =
  | 'online'
  | 'offline'
  | 'error'
  | 'requires_reauth'
  | 'permanently_failed'
  | 'connecting'
  | 'discovering_tools';

export interface ServerStatusEntry {
  status: ServerAvailabilityStatus;
  lastUpdated: Date;
  message?: string;
}

class UnifiedToolDiscoveryManager {
  private serverStatus: Map<string, ServerStatusEntry> = new Map();

  // Set server status (called by discovery managers and MCP wrapper)
  setServerStatus(serverSlug: string, status: ServerAvailabilityStatus, message?: string): void {
    this.serverStatus.set(serverSlug, {
      status,
      lastUpdated: new Date(),
      message
    });
  }

  // Check if server is available for tool execution
  isServerAvailable(serverSlug: string): boolean {
    const statusEntry = this.serverStatus.get(serverSlug);
    if (!statusEntry) return true; // Unknown = available (safe default)
    return statusEntry.status === 'online';
  }

  // Get all tools, filtered by server status
  getAllTools(): ToolMetadata[] {
    const allTools = this.getAllToolsUnfiltered();
    return allTools.filter(tool => {
      const serverSlug = tool.tool_path.split(':')[0];
      return this.isServerAvailable(serverSlug);
    });
  }
}

Status Callbacks

Discovery managers call status callbacks when discovery succeeds or fails: HTTP/SSE Discovery:

// services/satellite/src/services/remote-tool-discovery-manager.ts

// On successful discovery
this.statusCallback?.(serverSlug, 'online');

// On connection error
const { status, message } = RemoteToolDiscoveryManager.getStatusFromError(error);
this.statusCallback?.(serverSlug, status, message);

Stdio Discovery:

// services/satellite/src/services/stdio-tool-discovery-manager.ts

// On successful discovery
this.statusCallback?.(processId, 'online');

// On discovery error
this.statusCallback?.(processId, 'error', errorMessage);

Tool Filtering by Status

Per-User Instance Filtering

Tool availability is filtered based on the authenticated user’s OWN instance status: Key Principles:

Each user sees only tools from their own instances that are online
Other team members’ instance status does NOT affect your tool availability
If your instance is awaiting_user_config, you see NO tools from that server
If your instance is online, you see all tools (even if teammates’ instances are offline)

Example:

Team "Acme Corp" - Context7 MCP Server:
- Alice's instance: status = 'online'
  → Alice sees Context7 tools in discover_mcp_tools

- Bob's instance: status = 'awaiting_user_config'
  → Bob sees NO Context7 tools (must configure API key first)

- Charlie's instance: status = 'offline'
  → Charlie sees NO Context7 tools (server unreachable)

Discovery Filtering

When LLMs call discover_mcp_tools, only tools from the user’s available instances are returned:

// UnifiedToolDiscoveryManager.getAllTools() filters by user's instance status
const tools = toolDiscoveryManager.getAllTools(); // Only user's 'online' instances

// Tools from user's offline/error/requires_reauth/awaiting_user_config instances are hidden

Execution Blocking

When LLMs attempt to execute tools from the user’s unavailable instances:

// services/satellite/src/core/mcp-server-wrapper.ts

const serverSlug = toolPath.split(':')[0];
const statusEntry = this.toolDiscoveryManager?.getServerStatus(serverSlug);

// Block execution for non-recoverable states
if (statusEntry?.status === 'requires_reauth') {
  return {
    error: `Tool cannot be executed - your instance requires re-authentication.

Status: ${statusEntry.status}
Your instance requires re-authentication. Please re-authorize in the dashboard.

Unavailable server: ${serverSlug}`
  };
}

// Allow execution for offline/error (enables recovery detection)

Status Transition Triggers

Backend-Triggered (Database Updates)

Source: Backend API routes update mcpServerInstances table (per user)

Trigger	New Status	When
Instance created	`provisioning` or `awaiting_user_config`	Admin installs MCP server or member added to team
User config updated	`provisioning` (if was `awaiting_user_config`) or `restarting` (if already online)	User modifies their personal config
Config updated	`restarting`	User modifies environment vars/args/headers/query params
OAuth callback success	`connecting`	User re-authenticates
Health check fails	`offline`	Server unreachable (3-min interval)
Credential validation fails	`requires_reauth`	OAuth token invalid

Status Target: All backend status updates target the mcpServerInstances table. Status fields have been completely removed from mcpServerInstallations. Each user’s instance has independent status tracking.

Satellite-Triggered (Event Emission)

Source: Satellite emits mcp.server.status_changed events to backend (includes user_id field)

Trigger	New Status	When
Configure command received	`command_received`	Satellite polls backend
Server connection starts	`connecting`	Spawning process or HTTP connect
Tool discovery starts	`discovering_tools`	Calling tools/list
Tool discovery succeeds	`online`	Discovery completed successfully
Tool execution fails (3 retries)	`offline`/`error`/`requires_reauth`	Tool call failed after retries
Server recovery detected	`connecting`	Previously offline server responds
Stdio crashes 3 times	`permanently_failed`	3 crashes within 5 minutes

Event Payload:

{
  "event": "mcp.server.status_changed",
  "installation_id": "uuid",
  "team_id": "uuid",
  "user_id": "uuid",
  "status": "online",
  "status_message": "Server is online and ready"
}

The user_id field ensures status updates are applied to the correct user’s instance.

Implementation Components

The status tracking system consists of several integrated components:

Database schema for status field
Backend event handler for status updates
Satellite status event emission
Tool availability filtering by status
Configuration update status transitions
Tool execution status updates with auto-recovery

Instance Lifecycle

Four lifecycle processes for instance creation, deletion, and team membership changes

Event Emission

Status change event details with user_id field

Recovery System

Automatic recovery logic for failed instances

Tool Discovery

How status affects per-user tool discovery

Basics

Advanced

MCP Server Management

Status & Health Tracking

Backend Communication

Status Tracking

Status Tracking

Overview

Per-User Instance Status

Status Tracking Purposes

Status Values

Status Lifecycle

Initial Installation Flow (User Has Complete Config)

Initial Installation Flow (User Missing Required Config)

Configuration Update Flow

Failure and Recovery Flow

OAuth Failure Flow

Stdio Crash Flow (Permanent Failure)

Status Tracking Implementation

UnifiedToolDiscoveryManager

Status Callbacks

Tool Filtering by Status

Per-User Instance Filtering

Discovery Filtering

Execution Blocking

Status Transition Triggers

Backend-Triggered (Database Updates)

Satellite-Triggered (Event Emission)

Implementation Components

Instance Lifecycle

Event Emission

Recovery System

Tool Discovery

Basics

Advanced

MCP Server Management

Status & Health Tracking

Backend Communication

​Status Tracking

​Overview

​Per-User Instance Status

​Status Tracking Purposes

​Status Values

​Status Lifecycle

​Initial Installation Flow (User Has Complete Config)

​Initial Installation Flow (User Missing Required Config)

​Configuration Update Flow

​Failure and Recovery Flow

​OAuth Failure Flow

​Stdio Crash Flow (Permanent Failure)

​Status Tracking Implementation

​UnifiedToolDiscoveryManager

​Status Callbacks

​Tool Filtering by Status

​Per-User Instance Filtering

​Discovery Filtering

​Execution Blocking

​Status Transition Triggers

​Backend-Triggered (Database Updates)

​Satellite-Triggered (Event Emission)

​Implementation Components

​Related Documentation

Instance Lifecycle

Event Emission

Recovery System

Tool Discovery

Status Tracking

Overview

Per-User Instance Status

Status Tracking Purposes

Status Values

Status Lifecycle

Initial Installation Flow (User Has Complete Config)

Initial Installation Flow (User Missing Required Config)

Configuration Update Flow

Failure and Recovery Flow

OAuth Failure Flow

Stdio Crash Flow (Permanent Failure)

Status Tracking Implementation

UnifiedToolDiscoveryManager

Status Callbacks

Tool Filtering by Status

Per-User Instance Filtering

Discovery Filtering

Execution Blocking

Status Transition Triggers

Backend-Triggered (Database Updates)

Satellite-Triggered (Event Emission)

Implementation Components

Related Documentation