Skip to main content

​
Backend Polling Implementation

The DeployStack Satellite implements a sophisticated HTTP polling system for outbound-only communication with the backend. This firewall-friendly approach enables command orchestration, configuration synchronization, and status reporting without requiring inbound connections to the satellite.

​
Polling Architecture

​
Core Components

The polling system consists of four integrated services: 
┌─────────────────────────────────────────────────────────────────────────────────┐
│                        Satellite Polling Architecture                          │
│                                                                                 │
│  ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐           │
│  │ Command Polling │    │ Dynamic Config  │    │ Command         │           │
│  │ Service         │    │ Manager         │    │ Processor       │           │
│  │                 │    │                 │    │                 │           │
│  │ • Adaptive Poll │    │ • MCP Server    │    │ • HTTP Proxy    │           │
│  │ • Command Queue │    │   Config Sync   │    │   Management    │           │
│  │ • Error Backoff │    │ • Validation    │    │ • Health Checks │           │
│  └─────────────────┘    └─────────────────┘    └─────────────────┘           │
│                                                                                 │
│  ┌─────────────────────────────────────────────────────────────────────────┐   │
│  │                    Heartbeat Service                                    │   │
│  │                                                                         │   │
│  │  • Process Status Reporting    • System Metrics Collection             │   │
│  │  • 30-second Intervals         • Error Count Tracking                  │   │
│  └─────────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────────┘

​
Service Integration Flow

Startup → Registration → Polling Start → Configuration Sync → Command Processing
   │           │             │               │                    │
Backend    API Key       Poll Timer      MCP Servers         HTTP Proxy
Connect    Received      Started         Updated             Ready

​
Command Polling Service

​
Adaptive Polling Strategy

The polling service implements priority-based polling with automatic mode transitions: Immediate Mode (2 seconds):
  • Activated when immediate priority commands are pending
  • Used for MCP installations and critical updates
  • Enables 3-second end-to-end response time goal
  • Automatically returns to normal mode when immediate commands are processed
High Priority Mode (10 seconds):
  • Activated when high priority commands are pending
  • Used for MCP deletions and configuration changes
  • Balances urgency with resource efficiency
Normal Mode (30 seconds):
  • Activated when only normal priority commands are pending
  • Used for routine maintenance and non-urgent tasks
  • Default polling interval for steady-state operation
Slow Mode (60 seconds):
  • Used when no commands are pending
  • Minimizes backend load during idle periods
  • Automatically switches to faster modes when commands arrive
Error Mode (exponential backoff):
  • Activated when polling requests fail
  • Starts at current interval, doubles on each failure
  • Maximum backoff of 300 seconds (5 minutes)
  • Resets to appropriate priority mode on successful poll

​
Polling Implementation

class CommandPollingService {
  private currentPollingMode: 'immediate' | 'normal' | 'error' = 'normal';
  private currentInterval: number = 30; // seconds
  
  private async pollForCommands(): Promise<void> {
    const queryParams = new URLSearchParams();
    queryParams.set('last_poll', this.lastPollTime.toISOString());
    queryParams.set('limit', '10');

    const response = await fetch(
      `${backendUrl}/api/satellites/${satelliteId}/commands?${queryParams}`,
      {
        headers: { 'Authorization': `Bearer ${apiKey}` },
        signal: AbortSignal.timeout(15000)
      }
    );

    const pollResponse = await response.json();
    this.updatePollingStrategy(
      pollResponse.polling_mode, 
      pollResponse.next_poll_interval
    );
  }
}

​
Command Processing Pipeline

Commands flow through a structured processing pipeline:
  1. Command Validation: Payload validation and format checking
  2. Command Routing: Route to appropriate processor based on command type
  3. Execution: Process command with error handling and timeout
  4. Result Reporting: Send execution results back to backend
Supported Command Types:
  • configure - Update MCP server configuration
  • spawn - Start HTTP MCP server proxy
  • kill - Stop HTTP MCP server proxy
  • restart - Restart HTTP MCP server proxy
  • health_check - Perform health checks on all servers

​
Dynamic Configuration Management

​
Configuration Sync Process

The satellite replaces hardcoded MCP server configurations with dynamic updates from the backend: 
interface ConfigurationUpdate {
  mcp_servers: Record<string, McpServerConfig>;
  polling_intervals?: {
    normal: number;
    immediate: number;
    error_backoff_max: number;
  };
  resource_limits?: {
    max_processes: number;
    max_memory_per_process: string;
  };
}

​
Configuration Validation

All incoming configurations undergo strict validation: Server Configuration Validation:
  • URL format validation using new URL()
  • Server type restriction to ‘http’ only
  • Timeout value validation (positive numbers)
  • Required field presence checking
Configuration Change Detection:
  • Deep comparison of server configurations
  • Identification of added, removed, and modified servers
  • Structured logging of all configuration changes

​
Integration with Existing Services

Configuration updates trigger cascading updates across satellite services: 
Config Update → Dynamic Config Manager → HTTP Proxy Manager → Tool Discovery Manager
      │                    │                      │                      │
   Validate            Apply Changes         Re-initialize         Rediscover Tools
   Changes             Update Cache          Proxy Routes          Update Cache

​
HTTP Proxy Management Integration

​
Dynamic Server Registration

The HTTP Proxy Manager integrates with the dynamic configuration system: 
class HttpProxyManager {
  private configManager?: DynamicConfigManager;
  
  async handleConfigurationUpdate(config: DynamicMcpServersConfig): Promise<void> {
    // Re-initialize proxy routes with new server configurations
    await this.initialize();
  }
}

​
Server Health Monitoring

The command processor implements health checking for HTTP MCP servers: 
private async checkServerHealth(processInfo: ProcessInfo): Promise<HealthResult> {
  const response = await fetch(serverConfig.url, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', ...serverConfig.headers },
    body: JSON.stringify({
      jsonrpc: '2.0',
      id: 'health-check',
      method: 'tools/list',
      params: {}
    }),
    signal: AbortSignal.timeout(serverConfig.timeout || 10000)
  });
  
  return {
    health_status: response.ok ? 'healthy' : 'unhealthy',
    response_time_ms: Date.now() - startTime
  };
}

​
Tool Discovery Integration

​
Dynamic Tool Rediscovery

The Remote Tool Discovery Manager integrates with configuration updates: 
class RemoteToolDiscoveryManager {
  async handleConfigurationUpdate(config: DynamicMcpServersConfig): Promise<void> {
    // Reset and rediscover tools from updated server configurations
    this.isInitialized = false;
    this.cachedTools = [];
    await this.initialize();
  }
}

​
Tool Cache Management

Tool discovery maintains an in-memory cache that updates when server configurations change:
  • Cache Invalidation: Complete cache reset on configuration changes
  • Namespace Preservation: Tools maintain server-prefixed naming
  • Error Resilience: Failed discoveries don’t block other servers
  • Performance Optimization: Memory-only storage for fast access

​
Error Handling and Recovery

​
Polling Error Recovery

The polling service implements comprehensive error handling: Network Errors:
  • Automatic retry with exponential backoff
  • Maximum backoff limit of 300 seconds
  • Connection timeout handling (15 seconds)
  • Graceful degradation on persistent failures
Authentication Errors:
  • 401 Unauthorized handling
  • API key validation logging
  • Structured error reporting to logs
Configuration Errors:
  • Invalid server configuration rejection
  • Partial configuration application
  • Rollback to previous working configuration

​
Command Execution Error Handling

Command processing includes robust error handling: 
async processCommand(command: SatelliteCommand): Promise<CommandResult> {
  try {
    // Execute command with timeout and error handling
    const result = await this.executeCommand(command);
    return { command_id: command.id, status: 'completed', result };
    
  } catch (error) {
    return { 
      command_id: command.id, 
      status: 'failed', 
      error: error.message 
    };
  }
}

​
Heartbeat Integration

​
Process Status Reporting

The heartbeat service integrates with the command processor to report process status: 
class HeartbeatService {
  setCommandProcessor(commandProcessor: CommandProcessor): void {
    this.commandProcessor = commandProcessor;
  }
  
  private async sendHeartbeat(): Promise<void> {
    const processes = this.commandProcessor ? 
      this.commandProcessor.getAllProcesses() : [];
      
    const payload = {
      status: 'active',
      system_metrics: await this.collectSystemMetrics(),
      processes: processes,
      error_count: 0,
      version: '0.1.0'
    };
  }
}

​
System Metrics Collection

Current system metrics include:
  • Memory Usage: Node.js heap usage in MB
  • Process Uptime: Satellite process uptime in seconds
  • Process Count: Number of managed HTTP proxy processes
  • Error Count: Recent error count for health assessment

​
Development Integration

​
Service Initialization Order

The polling system requires specific initialization order: 
// 1. Backend connection and registration
const backendClient = new BackendClient(backendUrl, logger);
await backendClient.testConnection();
const registration = await backendClient.registerSatellite(data);

// 2. Configuration and processing services
const dynamicConfigManager = new DynamicConfigManager(logger);
const commandProcessor = new CommandProcessor(logger, dynamicConfigManager);

// 3. HTTP proxy and tool discovery with config integration
const httpProxyManager = new HttpProxyManager(server, logger);
httpProxyManager.setConfigManager(dynamicConfigManager);

const toolDiscoveryManager = new RemoteToolDiscoveryManager(logger);
toolDiscoveryManager.setConfigManager(dynamicConfigManager);

// 4. Polling service with handlers
const commandPollingService = new CommandPollingService(satelliteId, backendClient, logger);
commandPollingService.setConfigurationUpdateHandler(handleConfigUpdate);
commandPollingService.setCommandHandler(handleCommand);
commandPollingService.start();

​
Environment Configuration

Polling behavior is controlled by environment variables: 
# Backend connection
DEPLOYSTACK_BACKEND_URL=http://localhost:3000

# Satellite identification
DEPLOYSTACK_SATELLITE_NAME=dev-satellite-001

# Logging level affects polling debug output
LOG_LEVEL=debug

​
Performance Characteristics

​
Polling Efficiency

The adaptive polling strategy optimizes resource usage:
  • Normal Operations: 30-second intervals minimize backend load
  • Immediate Response: 2-second intervals for urgent commands
  • Error Backoff: Exponential backoff prevents cascade failures
  • Network Optimization: Query parameters reduce response size

​
Memory Usage

The polling system maintains minimal memory footprint:
  • Configuration Cache: ~1KB per MCP server configuration
  • Command Queue: Temporary storage for pending commands
  • Tool Cache: ~1KB per discovered tool
  • Process Tracking: Minimal metadata per HTTP proxy process

​
Network Traffic

Polling generates predictable network patterns:
  • Command Polling: Small JSON requests every 30 seconds (normal mode)
  • Configuration Sync: Infrequent larger payloads on configuration changes
  • Heartbeats: Regular status reports every 30 seconds
  • Command Results: Small JSON responses after command execution
Implementation Status: The polling system is fully implemented and operational. It successfully handles command orchestration, configuration synchronization, and status reporting through outbound-only HTTP communication with the backend.

​
Troubleshooting

​
Common Issues

401 Unauthorized Errors:
  • Indicates missing backend endpoints for satellite management
  • Expected during development when backend endpoints are not implemented
  • Satellite continues normal operation, polling will succeed once endpoints exist
Configuration Validation Failures:
  • Check server URL format and accessibility
  • Verify server type is set to ‘http’
  • Ensure timeout values are positive numbers
Polling Failures:
  • Check backend connectivity and availability
  • Verify satellite API key is valid
  • Monitor exponential backoff behavior in logs

​
Debug Logging

Enable debug logging to monitor polling behavior: 
LOG_LEVEL=debug npm run dev
Debug logs include:
  • Polling attempt details and timing
  • Configuration update processing
  • Command execution results
  • Error handling and recovery actions
⌘I