Status Tracking
The satellite tracks the health and availability of each MCP server instance (per-user) through a 12-state status system. This enables real-time monitoring, automatic recovery, and tool availability filtering on a per-user basis.
Overview
Per-User Instance Status
DeployStack tracks status at the instance level, not the installation level:
- Status Location:
mcpServerInstances table (per user)
- No Installation Status: Status fields completely removed from
mcpServerInstallations table
- Independent Tracking: Each team member has independent status for each MCP server
- User-Specific Filtering: Users see only tools from their OWN instances that are online
Example:
Team "Acme Corp" installs Filesystem MCP with 3 members:
- Alice's instance: status = 'online' (tools available to Alice)
- Bob's instance: status = 'awaiting_user_config' (tools NOT available to Bob)
- Charlie's instance: status = 'offline' (tools NOT available to Charlie)
Each member sees different tool availability based on their instance status.
Status Tracking Purposes
Status tracking serves three primary purposes:
- User Visibility: Users see their OWN instance status in real-time via the frontend
- Tool Availability: Tools from user’s unavailable instances are filtered from discovery
- Automatic Recovery: System detects and recovers from failures per instance
The status system is managed by UnifiedToolDiscoveryManager and updated through:
- Instance lifecycle events (provisioning → online)
- Health check results (online → offline)
- Tool execution failures (online → offline/error/requires_reauth)
- Configuration changes (online → restarting)
- Recovery detection (offline → connecting → online)
- User configuration completion (awaiting_user_config → provisioning)
Status Values
| Status | Description | Tools Available? | User Action Required |
|---|
awaiting_user_config | User hasn’t configured required user-level fields | No | Configure personal settings |
provisioning | Initial state after instance created | No | Wait |
command_received | Satellite received configuration command | No | Wait |
connecting | Connecting to MCP server | No | Wait |
discovering_tools | Running tool discovery | No | Wait |
syncing_tools | Syncing tools to backend | No | Wait |
online | Server healthy and responding | Yes | None |
restarting | Configuration updated, server restarting | No | Wait |
offline | Server unreachable (auto-recovers) | No | Wait or check server |
error | General error state (auto-recovers) | No | Check logs |
requires_reauth | OAuth token expired/revoked | No | Re-authenticate |
permanently_failed | 3+ crashes in 5 minutes (stdio only) | No | Manual restart required |
New Status: awaiting_user_configThis status indicates that an MCP server has required user-level configuration fields (e.g., personal API keys) and the user hasn’t configured them yet. The satellite does NOT spawn processes for instances with this status. Once the user completes their configuration via the dashboard, the status automatically transitions to provisioning and the instance spawns normally.See Instance Lifecycle - Process A for details.
Status Lifecycle
Initial Installation Flow (User Has Complete Config)
provisioning
↓
command_received (satellite received configure command)
↓
connecting (spawning MCP server process or connecting to HTTP/SSE)
↓
discovering_tools (calling tools/list)
↓
syncing_tools (sending tools to backend)
↓
online (ready for use)
Initial Installation Flow (User Missing Required Config)
When an MCP server has required user-level configuration fields and the user hasn’t configured them:
awaiting_user_config
↓
[User configures personal settings via dashboard]
↓
provisioning (backend updates status, sends satellite command)
↓
command_received
↓
connecting
↓
discovering_tools
↓
syncing_tools
↓
online
Backend Filtering: The satellite does NOT receive configurations for instances with awaiting_user_config status. Backend filters these instances out in the config endpoint, preventing spawn attempts for incomplete configurations.
Configuration Update Flow
online
↓
restarting (user updated config, backend sets status immediately)
↓
connecting (satellite receives command, restarts server)
↓
discovering_tools
↓
online
Failure and Recovery Flow
online
↓
offline/error (server unreachable or error response)
↓
[automatic recovery when server comes back]
↓
connecting
↓
discovering_tools
↓
online
OAuth Failure Flow
online
↓
requires_reauth (401/403 response or token refresh failed)
↓
[user re-authenticates via dashboard]
↓
connecting
↓
discovering_tools
↓
online
Stdio Crash Flow (Permanent Failure)
online
↓
(stdio process crashes)
↓
connecting (auto-restart attempt 1)
↓
(crashes again within 5 minutes)
↓
connecting (auto-restart attempt 2)
↓
(crashes again within 5 minutes)
↓
permanently_failed (manual intervention required)
Status Tracking Implementation
The status system is implemented in UnifiedToolDiscoveryManager:
// services/satellite/src/services/unified-tool-discovery-manager.ts
export type ServerAvailabilityStatus =
| 'online'
| 'offline'
| 'error'
| 'requires_reauth'
| 'permanently_failed'
| 'connecting'
| 'discovering_tools';
export interface ServerStatusEntry {
status: ServerAvailabilityStatus;
lastUpdated: Date;
message?: string;
}
class UnifiedToolDiscoveryManager {
private serverStatus: Map<string, ServerStatusEntry> = new Map();
// Set server status (called by discovery managers and MCP wrapper)
setServerStatus(serverSlug: string, status: ServerAvailabilityStatus, message?: string): void {
this.serverStatus.set(serverSlug, {
status,
lastUpdated: new Date(),
message
});
}
// Check if server is available for tool execution
isServerAvailable(serverSlug: string): boolean {
const statusEntry = this.serverStatus.get(serverSlug);
if (!statusEntry) return true; // Unknown = available (safe default)
return statusEntry.status === 'online';
}
// Get all tools, filtered by server status
getAllTools(): ToolMetadata[] {
const allTools = this.getAllToolsUnfiltered();
return allTools.filter(tool => {
const serverSlug = tool.tool_path.split(':')[0];
return this.isServerAvailable(serverSlug);
});
}
}
Status Callbacks
Discovery managers call status callbacks when discovery succeeds or fails:
HTTP/SSE Discovery:
// services/satellite/src/services/remote-tool-discovery-manager.ts
// On successful discovery
this.statusCallback?.(serverSlug, 'online');
// On connection error
const { status, message } = RemoteToolDiscoveryManager.getStatusFromError(error);
this.statusCallback?.(serverSlug, status, message);
Stdio Discovery:
// services/satellite/src/services/stdio-tool-discovery-manager.ts
// On successful discovery
this.statusCallback?.(processId, 'online');
// On discovery error
this.statusCallback?.(processId, 'error', errorMessage);
Per-User Instance Filtering
Tool availability is filtered based on the authenticated user’s OWN instance status:
Key Principles:
- Each user sees only tools from their own instances that are
online
- Other team members’ instance status does NOT affect your tool availability
- If your instance is
awaiting_user_config, you see NO tools from that server
- If your instance is
online, you see all tools (even if teammates’ instances are offline)
Example:
Team "Acme Corp" - Context7 MCP Server:
- Alice's instance: status = 'online'
→ Alice sees Context7 tools in discover_mcp_tools
- Bob's instance: status = 'awaiting_user_config'
→ Bob sees NO Context7 tools (must configure API key first)
- Charlie's instance: status = 'offline'
→ Charlie sees NO Context7 tools (server unreachable)
Discovery Filtering
When LLMs call discover_mcp_tools, only tools from the user’s available instances are returned:
// UnifiedToolDiscoveryManager.getAllTools() filters by user's instance status
const tools = toolDiscoveryManager.getAllTools(); // Only user's 'online' instances
// Tools from user's offline/error/requires_reauth/awaiting_user_config instances are hidden
Execution Blocking
When LLMs attempt to execute tools from the user’s unavailable instances:
// services/satellite/src/core/mcp-server-wrapper.ts
const serverSlug = toolPath.split(':')[0];
const statusEntry = this.toolDiscoveryManager?.getServerStatus(serverSlug);
// Block execution for non-recoverable states
if (statusEntry?.status === 'requires_reauth') {
return {
error: `Tool cannot be executed - your instance requires re-authentication.
Status: ${statusEntry.status}
Your instance requires re-authentication. Please re-authorize in the dashboard.
Unavailable server: ${serverSlug}`
};
}
// Allow execution for offline/error (enables recovery detection)
Status Transition Triggers
Backend-Triggered (Database Updates)
Source: Backend API routes update mcpServerInstances table (per user)
| Trigger | New Status | When |
|---|
| Instance created | provisioning or awaiting_user_config | Admin installs MCP server or member added to team |
| User config updated | provisioning (if was awaiting_user_config) or restarting (if already online) | User modifies their personal config |
| Config updated | restarting | User modifies environment vars/args/headers/query params |
| OAuth callback success | connecting | User re-authenticates |
| Health check fails | offline | Server unreachable (3-min interval) |
| Credential validation fails | requires_reauth | OAuth token invalid |
Status Target: All backend status updates target the mcpServerInstances table. Status fields have been completely removed from mcpServerInstallations. Each user’s instance has independent status tracking.
Satellite-Triggered (Event Emission)
Source: Satellite emits mcp.server.status_changed events to backend (includes user_id field)
| Trigger | New Status | When |
|---|
| Configure command received | command_received | Satellite polls backend |
| Server connection starts | connecting | Spawning process or HTTP connect |
| Tool discovery starts | discovering_tools | Calling tools/list |
| Tool discovery succeeds | online | Discovery completed successfully |
| Tool execution fails (3 retries) | offline/error/requires_reauth | Tool call failed after retries |
| Server recovery detected | connecting | Previously offline server responds |
| Stdio crashes 3 times | permanently_failed | 3 crashes within 5 minutes |
Event Payload:
{
"event": "mcp.server.status_changed",
"installation_id": "uuid",
"team_id": "uuid",
"user_id": "uuid",
"status": "online",
"status_message": "Server is online and ready"
}
The user_id field ensures status updates are applied to the correct user’s instance.
Implementation Components
The status tracking system consists of several integrated components:
- Database schema for status field
- Backend event handler for status updates
- Satellite status event emission
- Tool availability filtering by status
- Configuration update status transitions
- Tool execution status updates with auto-recovery