HTTP Debugging & Troubleshooting

This guide covers HTTP-specific debugging, API troubleshooting, and network connectivity issues with the WhoisXML API MCP Server. It includes comprehensive request/response logging capabilities for diagnosing communication problems.

# Set environment variable
export WHOISXMLAPI_HTTP_DEBUG="/tmp/whoisxmlapi-debug.log"

# Run server and check logs
tail -f /tmp/whoisxmlapi-debug.log

HTTP Request Logging

The WhoisXML API MCP Server includes comprehensive HTTP request logging capabilities for debugging and monitoring API communications.

Enabling HTTP Logging

Environment Variable Method:

# Set debug log file location
export WHOISXMLAPI_HTTP_DEBUG="/tmp/whoisxmlapi-debug.log"

# Run server
mcp-whoisxmlapi --help

Docker Configuration:

{
  "mcpServers": {
    "whoisxmlapi": {
      "command": "docker",
      "args": [
        "run", "--rm", "-i",
        "--env", "WHOISXMLAPI_TOKEN",
        "--env", "WHOISXMLAPI_HTTP_DEBUG",
        "-v", "/tmp:/tmp",
        "whoisxmlapidotcom/mcp-whoisxmlapi:v1"
      ],
      "env": {
        "WHOISXMLAPI_TOKEN": "your-token",
        "WHOISXMLAPI_HTTP_DEBUG": "/tmp/whoisxmlapi-debug.log"
      }
    }
  }
}

Binary Configuration:

{
  "mcpServers": {
    "whoisxmlapi": {
      "command": "/usr/local/bin/mcp-whoisxmlapi",
      "env": {
        "WHOISXMLAPI_TOKEN": "your-token",
        "WHOISXMLAPI_HTTP_DEBUG": "/tmp/whoisxmlapi-debug.log"
      }
    }
  }
}

npx Configuration:

{
  "mcpServers": {
    "whoisxmlapi": {
      "command": "npx",
      "args": ["-y", "@whoisxmlapidotcom/mcp-whoisxmlapi"],
      "env": {
        "WHOISXMLAPI_TOKEN": "your-token",
        "WHOISXMLAPI_HTTP_DEBUG": "/tmp/whoisxmlapi-debug.log"
      }
    }
  }
}

Log Format and Content

The debug log captures comprehensive information for each HTTP request:

Request/Response Format:

• Unique Request ID: Each request gets a UUID for correlation
• Outgoing Requests: Prefixed with [REQUEST_ID] >>>
• Incoming Responses: Prefixed with [REQUEST_ID] <<<
• Timing Metrics: Detailed performance data for each request

Example Log Output:

[abc123ef-...] >>>  POST /v1/whois HTTP/1.1
[abc123ef-...] >>>  Host: api.whoisxmlapi.com
[abc123ef-...] >>>  Authorization: Bearer <REDACTED>
[abc123ef-...] >>>  Content-Type: application/json
[abc123ef-...] >>>  User-Agent: mcp-whoisxmlapi/1.0.0
[abc123ef-...] >>>
[abc123ef-...] >>>  {"domainName":"example.com","apiKey":"<REDACTED>"}
[abc123ef-...] Timing: dns=2ms conn=15ms tls=45ms ttfb=125ms total=150ms status=ok
[abc123ef-...] <<<  HTTP/1.1 200 OK
[abc123ef-...] <<<  Content-Type: application/json
[abc123ef-...] <<<  Content-Length: 1234
[abc123ef-...] <<<  Date: Mon, 01 Jan 2024 12:00:00 GMT
[abc123ef-...] <<<
[abc123ef-...] <<<  {"domainName":"example.com","registryData":{"..."},"..."}

Security Features

Automatic Redaction: The logging system automatically redacts sensitive information:

• Authorization Headers: Authorization: Bearer <REDACTED>
• API Keys in URLs: ?apiKey=<REDACTED>
• API Keys in JSON: "apiKey": "<REDACTED>"
• Custom tokens: Any field containing "token", "key", or "secret"

What Gets Redacted:

# Original request
Authorization: Bearer wxt-abc123def456
POST /api?apiKey=wxt-abc123def456
{"apiKey": "wxt-abc123def456", "domain": "example.com"}

# Logged output
Authorization: Bearer <REDACTED>
POST /api?apiKey=<REDACTED>
{"apiKey": "<REDACTED>", "domain": "example.com"}

Timing Metrics

Each request includes detailed timing information:

• dns - DNS lookup time

  • Typical Range: 1-50ms

• conn - Connection establishment

  • Typical Range: 10-100ms

• tls - TLS handshake

  • Typical Range: 50-200ms

• ttfb - Time to first byte

  • Typical Range: 100-2000ms

• total - Total request time

  • Typical Range: 200-3000ms

• status - Request outcome

  • Values: ok or error: description

Network Connectivity Issues

DNS Resolution Problems

Symptoms:

• "Host not found" or "DNS resolution failed"
• Long delays before connection attempts
• Intermittent connectivity issues

Diagnosis:

# Test DNS resolution
nslookup whoisxmlapi.com
dig whoisxmlapi.com

# Test with different DNS servers
nslookup whoisxmlapi.com 8.8.8.8
dig @1.1.1.1 whoisxmlapi.com

# Check system DNS configuration
cat /etc/resolv.conf  # Linux
scutil --dns  # macOS

Solutions:

1. Configure custom DNS:

   # Docker with custom DNS
   docker run --rm --dns 8.8.8.8 \
     -e WHOISXMLAPI_TOKEN=your-token \
     whoisxmlapidotcom/mcp-whoisxmlapi:v1 --help
   

2. System DNS configuration:

   # Linux: Edit /etc/resolv.conf
   echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
   
   # macOS: System Preferences → Network → Advanced → DNS
   

3. Use IP addresses (temporary):

   # Find IP address
   nslookup whoisxmlapi.com
   
   # Use custom base URL with IP
   mcp-whoisxmlapi --api-base-url https://IP_ADDRESS
   

SSL/TLS Certificate Issues

Symptoms:

• "Certificate verification failed"
• "SSL handshake timeout"
• "Invalid certificate" errors

Diagnosis:

# Test SSL connection
openssl s_client -connect whoisxmlapi.com:443

# Check certificate details
openssl s_client -connect whoisxmlapi.com:443 -servername whoisxmlapi.com

# Test with curl
curl -I https://whoisxmlapi.com
curl -v https://whoisxmlapi.com

Solutions:

1. Update system certificates:

   # Ubuntu/Debian
   sudo apt-get update
   sudo apt-get install ca-certificates
   
   # CentOS/RHEL
   sudo yum update ca-certificates
   
   # macOS
   brew install ca-certificates
   

2. Check system time:

   # Ensure system time is correct
   date
   sudo ntpdate -s time.nist.gov  # Linux
   sudo sntp -sS time.apple.com   # macOS
   

3. Corporate certificate issues:

   # Add corporate certificates
   sudo cp corporate-cert.crt /usr/local/share/ca-certificates/
   sudo update-ca-certificates
   

Proxy and Firewall Issues

Symptoms:

• Connection timeouts
• "Connection refused" errors
• Requests hang indefinitely

Diagnosis:

# Check for proxy settings
echo $HTTP_PROXY
echo $HTTPS_PROXY
echo $NO_PROXY

# Test direct connection
curl -I https://whoisxmlapi.com

# Test with proxy bypass
curl --noproxy "*" -I https://whoisxmlapi.com

Solutions:

1. Configure proxy settings:

   # Set proxy environment variables
   export HTTP_PROXY=http://proxy.company.com:8080
   export HTTPS_PROXY=http://proxy.company.com:8080
   export NO_PROXY=localhost,127.0.0.1
   
   # For Docker
   docker run --rm \
     -e HTTP_PROXY=http://proxy.company.com:8080 \
     -e HTTPS_PROXY=http://proxy.company.com:8080 \
     -e WHOISXMLAPI_TOKEN=your-token \
     whoisxmlapidotcom/mcp-whoisxmlapi:v1 --help
   

2. Configure firewall rules:

   # Allow outbound HTTPS (port 443)
   sudo ufw allow out 443
   
   # For specific IP ranges
   sudo iptables -A OUTPUT -p tcp --dport 443 -j ACCEPT
   

3. Test network connectivity:

   # Test basic connectivity
   ping whoisxmlapi.com
   telnet whoisxmlapi.com 443
   nc -zv whoisxmlapi.com 443
   

API Response Issues

Authentication Errors

Symptoms:

• "401 Unauthorized" responses
• "Invalid API key" messages
• "Authentication failed" errors

Diagnosis using HTTP logs:

# Check authentication headers in logs
grep "Authorization" /tmp/whoisxmlapi-debug.log

# Look for 401 responses
grep "401" /tmp/whoisxmlapi-debug.log

# Check token format in requests
grep "apiKey" /tmp/whoisxmlapi-debug.log

Solutions:

1. Verify token format:

   # Token should start with specific prefix
   echo $WHOISXMLAPI_TOKEN | head -c 10
   
   # Check for extra characters
   echo "$WHOISXMLAPI_TOKEN" | hexdump -C
   

2. Check token configuration: Using your WhoisXMLAPI account, you can verify that the token is valid by checking it at the top of the account page.

Rate Limiting Issues

Symptoms:

• "429 Too Many Requests" responses
• Increasing response times
• Intermittent failures under load

Diagnosis using HTTP logs:

# Check for rate limit responses
grep "429" /tmp/whoisxmlapi-debug.log

# Check timing patterns
grep "Timing:" /tmp/whoisxmlapi-debug.log | tail -20

# Look for rate limit headers
grep -A 5 "429" /tmp/whoisxmlapi-debug.log

Solutions:

1. Implement request spacing:

   # Add delays between requests
   mcp-whoisxmlapi --timeout 30s
   

2. Check rate limits:

   # Review API documentation for rate limits
   # Monitor response headers for rate limit info
   

3. Reduce concurrent requests:

   # Disable resource-intensive tools
   export WHOISXMLAPI_DISABLED_TOOLS="threat_intelligence,reverse_whois"
   

API Response Timeouts

Symptoms:

• Requests hang indefinitely
• "Request timeout" errors
• Partial responses

Diagnosis using HTTP logs:

# Check timing metrics
grep "Timing:" /tmp/whoisxmlapi-debug.log | grep "total=[0-9]\{4,\}"

# Look for timeout patterns
grep "timeout\|Timeout" /tmp/whoisxmlapi-debug.log

# Check for incomplete responses
grep -B 2 -A 2 "status=error" /tmp/whoisxmlapi-debug.log

Solutions:

1. Increase timeout:

   # Use longer timeout
   mcp-whoisxmlapi --timeout 60s
   
   # Or via environment variable
   export WHOISXMLAPI_TIMEOUT=60s
   

2. Check network performance:

   # Test network latency
   ping whoisxmlapi.com
   
   # Test bandwidth
   curl -w "@curl-format.txt" -o /dev/null -s https://whoisxmlapi.com
   

Performance Analysis

Request Timing Analysis

Using HTTP logs for performance analysis:

# Extract timing data
grep "Timing:" /tmp/whoisxmlapi-debug.log | tail -50

# Average total response time
grep "Timing:" /tmp/whoisxmlapi-debug.log | \
  sed 's/.*total=\([0-9]*\)ms.*/\1/' | \
  awk '{sum+=$1; count++} END {print "Average:", sum/count "ms"}'

# Find slowest requests
grep "Timing:" /tmp/whoisxmlapi-debug.log | \
  sort -k2 -t'=' -n | tail -10

# DNS resolution performance
grep "Timing:" /tmp/whoisxmlapi-debug.log | \
  sed 's/.*dns=\([0-9]*\)ms.*/\1/' | \
  sort -n | tail -10

Performance Bottleneck Identification

High DNS times (>50ms):

# Solutions:
# 1. Use faster DNS servers
# 2. Configure DNS caching
# 3. Use IP addresses directly (testing only)

High connection times (>200ms):

# Solutions:
# 1. Check network connectivity
# 2. Verify no proxy issues
# 3. Test from different location

High TLS times (>500ms):

# Solutions:
# 1. Update system certificates
# 2. Check for certificate validation issues
# 3. Verify system time is correct

High TTFB times (>3000ms):

# Solutions:
# 1. Increase request timeout
# 2. Check API rate limiting
# 3. Simplify requests
# 4. Contact API support

Debugging Tools and Techniques

Real-time Log Monitoring

# Monitor logs in real-time
tail -f /tmp/whoisxmlapi-debug.log

# Filter for specific patterns
tail -f /tmp/whoisxmlapi-debug.log | grep "error\|Error\|401\|429\|500"

# Monitor timing issues
tail -f /tmp/whoisxmlapi-debug.log | grep "Timing:" | grep "total=[0-9]\{4,\}"

# Watch for specific request IDs
tail -f /tmp/whoisxmlapi-debug.log | grep "abc123ef"

Request Correlation

# Find all logs for a specific request
REQUEST_ID="abc123ef-1234-5678-9abc-def123456789"
grep "$REQUEST_ID" /tmp/whoisxmlapi-debug.log

# Extract request/response pairs
grep ">>>\|<<<" /tmp/whoisxmlapi-debug.log | head -20

# Find failed requests
grep -B 5 -A 5 "status=error" /tmp/whoisxmlapi-debug.log

Log Analysis Scripts

Simple timing analysis:

#!/bin/bash
# analyze-timing.sh

LOG_FILE="/tmp/whoisxmlapi-debug.log"

echo "=== Timing Analysis ==="
echo "Total requests: $(grep -c "Timing:" $LOG_FILE)"
echo

echo "Average timings:"
grep "Timing:" $LOG_FILE | sed 's/.*dns=\([0-9]*\)ms.*/\1/' | awk '{sum+=$1; count++} END {printf "DNS: %.1fms\n", sum/count}'
grep "Timing:" $LOG_FILE | sed 's/.*conn=\([0-9]*\)ms.*/\1/' | awk '{sum+=$1; count++} END {printf "Connection: %.1fms\n", sum/count}'
grep "Timing:" $LOG_FILE | sed 's/.*tls=\([0-9]*\)ms.*/\1/' | awk '{sum+=$1; count++} END {printf "TLS: %.1fms\n", sum/count}'
grep "Timing:" $LOG_FILE | sed 's/.*ttfb=\([0-9]*\)ms.*/\1/' | awk '{sum+=$1; count++} END {printf "TTFB: %.1fms\n", sum/count}'
grep "Timing:" $LOG_FILE | sed 's/.*total=\([0-9]*\)ms.*/\1/' | awk '{sum+=$1; count++} END {printf "Total: %.1fms\n", sum/count}'

echo
echo "Error count: $(grep -c "status=error" $LOG_FILE)"

Advanced Debugging

Network packet capture:

# Capture HTTP traffic (Linux)
sudo tcpdump -i any -s 0 -w whoisxmlapi.pcap host whoisxmlapi.com

# Analyze with Wireshark
wireshark whoisxmlapi.pcap

System-level debugging:

# Monitor system calls (Linux)
strace -e trace=network -p $(pgrep mcp-whoisxmlapi)

# Monitor file descriptors
lsof -p $(pgrep mcp-whoisxmlapi)

# Check network connections
netstat -tp | grep mcp-whoisxmlapi

Log Management

Log Rotation and Cleanup

Prevent large log files:

# Rotate logs daily
logrotate -f /etc/logrotate.d/whoisxmlapi

# Manual rotation
mv /tmp/whoisxmlapi-debug.log /tmp/whoisxmlapi-debug.log.old
touch /tmp/whoisxmlapi-debug.log

Cleanup old logs:

# Remove logs older than 7 days
find /tmp -name "whoisxmlapi-debug.log*" -mtime +7 -delete

# Compress old logs
gzip /tmp/whoisxmlapi-debug.log.old

Selective Logging

Log only errors:

# Filter logs in real-time
tail -f /tmp/whoisxmlapi-debug.log | grep -E "(error|Error|40[0-9]|50[0-9])"

Log specific tools:

# Enable debugging only for specific scenarios
# Use temporary debug file for focused debugging
export WHOISXMLAPI_HTTP_DEBUG="/tmp/debug-$(date +%Y%m%d-%H%M%S).log"

Security Considerations

Log Security

Sensitive data protection:

• Logs automatically redact API tokens and keys
• Review logs before sharing or storing
• Set appropriate file permissions: chmod 600 debug.log
• Avoid logging to shared directories

Log file permissions:

# Secure log file
touch /tmp/whoisxmlapi-debug.log
chmod 600 /tmp/whoisxmlapi-debug.log

# Verify permissions
ls -la /tmp/whoisxmlapi-debug.log

Network Security

Monitor for security issues:

# Check for suspicious patterns
grep -E "(injection|script|malware)" /tmp/whoisxmlapi-debug.log

# Monitor authentication failures
grep "401\|403" /tmp/whoisxmlapi-debug.log

# Check for unusual request patterns
grep "User-Agent" /tmp/whoisxmlapi-debug.log | sort | uniq -c

Getting Help

Information to Collect

When reporting HTTP/network issues, include:

1. HTTP debug logs:

   # Last 100 lines of debug log
   tail -100 /tmp/whoisxmlapi-debug.log
   
   # Specific error patterns
   grep -B 5 -A 5 "error\|Error\|40[0-9]\|50[0-9]" /tmp/whoisxmlapi-debug.log
   

2. Network connectivity tests:

   # Basic connectivity
   ping whoisxmlapi.com
   curl -I https://whoisxmlapi.com
   
   # DNS resolution
   nslookup whoisxmlapi.com
   

3. System information:

   # System details
   uname -a
   date
   
   # Network configuration
   ip route  # Linux
   route -n get default  # macOS
   

Related Documentation

• General Troubleshooting - Other common issues
• Docker Troubleshooting - Container-specific problems
• npm Troubleshooting - Node.js and npm issues
• AI Client Troubleshooting - Client integration issues
• Server Configuration - Advanced configuration options