RelayMiner RPS Testing

Light load test

make test_baseline_static_server_load R=10000 C=1000

Quick test

make test_baseline_static_server_load R=5000 C=500 D=10s

Maximum load test

make test_baseline_static_server_load R=200000 T=32 C=15000 D=45s

⚡ RelayMiner Only Load Test

Session Validity Critical

This test generates real RelayRequest data with proper cryptographic signatures. Sessions expire periodically (typically every few blocks), which will cause ALL requests to fail validation once the session changes during your test.

This test covers:

• RelayMiner request processing performance

• Signature verification overhead

• Backend service communication latency

• Request validation and response serialization

• Resource contention under high load

Default Configuration:

• Rate: 512 requests per second

• Threads: 16 worker threads

• Connections: 256 concurrent connections

• Duration: 300 seconds (5 minutes)

Command:

make test_relayminer_only_load

make test_relayminer_only_load

Custom parameters:

{% endcode %}

## 🔧 Infrastructure Components

### 📊 Instruction-Level Timing Metrics

The RelayMiner now includes granular timing instrumentation that measures the duration between each step of relay processing.

Key Instructions Tracked:

- `init_request_logger` - Initial request setup
- `get_start_block` - Blockchain state retrieval
- `new_relay_request` - Request parsing and validation
- `relay_request_basic_validation` - Basic request validation
- `pre_request_verification` - Cryptographic verification
- `build_service_backend_request` - Backend request preparation
- `http_client_do` - Backend service call
- `serialize_http_response` - Response serialization
- `response_sent` - Final response transmission

Metrics Available:

- `RelayMiner_instruction_time_seconds` - Histogram of instruction durations
- Average duration per instruction
- 99th percentile latency per instruction

### 🌐 High-Performance Nginx Server

A highly optimized nginx server (`nginx-chainid`) provides static JSON-RPC responses for baseline testing.

Optimizations:

- Worker processes: Auto-scaled to available CPU cores
- Connection limits: 65,536 worker connections with Linux epoll
- Keep-alive: 10,000 requests per connection, 300s timeout
- Logging disabled: Maximum performance with access_log off
- Buffer optimization: Tuned client buffers and output buffers
- HTTP/2 support: Enabled for connection multiplexing

Response: Returns static {"jsonrpc":"2.0","id":1,"result":"0x1"} for all requests

### 🔨 wrk2 Load Testing Tool

Modern HTTP benchmarking tool with constant rate limiting and accurate latency measurement.

Key Features:

- Constant rate limiting: Maintains precise RPS regardless of latency
- Latency accuracy: True latency measurement
- Lua scripting: Custom request generation with proper headers
- Thread scaling: Configurable worker threads for high concurrency

Access: Available in `Tilt` k8s as `wrk2` deployment

### 📈 RelayMiner Custom HTTP Client

Enhanced HTTP client with performance optimizations and detailed debugging capabilities.

Performance Features:

- Connection pooling: Scaled with concurrency limits
- Buffer management: Reusable byte buffers to reduce GC pressure
- Concurrency limiting: Semaphore-based admission control
- Timeout optimization: Granular timeout control per request

Debug Capabilities:

- Phase timing: DNS, connection, TLS, request, response phases
- Connection reuse tracking: Monitor pool effectiveness
- Error categorization: Timeout vs. connection vs. other errors
- Resource monitoring: Active requests, goroutine counts

## 🏃 Running Tests

### ⚠️ Prerequisites

Before running RPS tests, ensure your environment is configured:

{% hint style="info" %}
Critical Setup Steps
{% endhint %}

- Configure consensus timeouts to 30s in `config.yml` for stable sessions
- [LocalNet](https://dev.poktroll.com/develop/networks/localnet) must be running with proper configuration
- Run `make acc_initialize_pub_keys` to initialize blockchain accounts
- Verify sufficient system resources (CPU, memory, file descriptors)

### 🔧 LocalNet Setup

Proper LocalNet configuration is essential for reliable RPS testing.

Required Configuration:

{% stepper %}
{% step %}
## Consensus Timeouts

Set these values in `config.yml`:

{% code %}
consensus:
     timeout_commit: "30s"
     timeout_propose: "30s"

Service Configuration

The "static" service must be configured:

• Service ID: static

• Compute units per relay: 1 (minimal)

• Backend: nginx-chainid server

📋 Test Parameters

All RPS tests support flexible parameter configuration:

Parameter	Description	Default
R	Requests per second	512-100k
t	Worker threads	16
c	Concurrent connections	256-5000
d	Test duration	30s-300s

Parameter Selection Guidelines:

• Light load: R=100-1000, c=50-500, d=30s-60s

• Medium load: R=1000-5000, c=500-2000, d=60s-300s

• Heavy load: R=5000+, c=2000+, d=300s+

📊 Monitoring Results

🖥️ Test Output

RPS tests provide detailed performance metrics and diagnostics.

Key Metrics to Monitor:

• Requests/sec: Actual vs. target RPS

• Latency percentiles: 50th, 90th, 99th, 99.9th percentiles

• Error rates: Connection errors, timeouts, validation failures

• Resource usage: CPU, memory, connection pool utilization

Sample Output:

</div>