3_e2e_tests_deep_dive

## Supported Services in E2E Tests

To see the list of supported services for the tests, see the `test_cases` array in the [E2E Test Config](https://github.com/buildwithgrove/path/blob/main/e2e/config/e2e_load_test.config.default.yaml) file.

## Environment Variables

These environment variables are set by the test make targets, but if you wish to set them manually, see the table below:

<details>
<summary>Env Vars Table</summary>
| Variable         | Description                                                                                       | Values                              | Required |
| ---------------- | ------------------------------------------------------------------------------------------------- | ----------------------------------- | -------- |
| TEST_MODE        | Determines the test execution mode                                                                | `e2e`                               | Yes      |
| TEST_PROTOCOL    | Specifies which protocol to test                                                                  | `shannon`                           | Yes      |
| TEST_SERVICE_IDS | Specifies which service IDs to test. If not set, all service IDs for the protocol will be tested. | Comma-separated list of service IDs | No       |
| TEST_WEBSOCKETS  | Run only Websocket tests, skipping HTTP tests entirely                                            | `true` or `false`                   | No       |
</details>

## Extending/Updating/Adding EVM E2E Tests

To add new services or methods to the E2E tests, you will need to open a new PR to PATH's `main` branch.

{% stepper %}
{% step %}
## Add service definitions

Add new service definitions to the `services` array in the `e2e/config/services_shannon.yaml` configuration file.
{% endstep %}

{% step %}
## Configure service parameters

Configure service parameters including contract addresses, start blocks, and transaction hashes for archival tests.
{% endstep %}
{% endstepper %}

**Example new service configuration:**

```yaml
services:
  - name: "New Chain E2E Test"
    protocol: "shannon"
    service_id: "newchain"
    archival: true
    service_params:
      contract_address: "0x..."
      contract_start_block: 1000000
      transaction_hash: "0x..."
      call_data: "0x18160ddd"

Test Metrics and Validation

{% hint style="warning" %} Threshold Validation

Tests will fail if any configured thresholds are exceeded, ensuring consistent service quality and performance. {% endhint %}

The E2E tests collect and validate comprehensive metrics across multiple dimensions:

Category	Metrics Collected
HTTP Metrics	- Success rates (HTTP 200) - Status code distribution - HTTP error categorization
Latency Metrics	- P50, P95, P99 latency percentiles - Average latency - Per-method latency analysis
JSON-RPC Validation	- Response unmarshaling success - JSON-RPC error field validation - Result field validation - Protocol-specific validation
Service-Level Metrics	- Per-service success aggregation - Cross-method performance comparison - Service reliability scoring - Error categorization and reporting

Websocket Testing

PATH E2E tests support Websocket testing for compatible services. Currently, XRPLEVM services are configured with Websocket support.

Websocket Test Features

• Transport-Agnostic Validation: Uses the same JSON-RPC validation logic as HTTP tests

• Real-time Connection: Establishes persistent Websocket connections to test real-time communication

• EVM JSON-RPC Support: Tests all standard EVM JSON-RPC methods over Websocket

• Separate from HTTP: Websocket tests run independently from HTTP tests

Websocket Test Modes

Mode	Command	Description
HTTP Only	make e2e_test xrplevm	Runs only HTTP tests (default behavior)
Websocket Only	make e2e_test_websocket xrplevm	Runs only Websocket tests, skipping HTTP tests entirely
All Websocket Services	make e2e_test_websocket_all	Runs Websocket tests for all Websocket-compatible services

Service Configuration

To enable Websocket testing for a service, add websockets: true to the service configuration in services_shannon.yaml:

yaml
- name: "Shannon - xrplevm (XRPL EVM MainNet) Test"
  service_id: "xrplevm" 
  service_type: "cosmos_sdk"
  websockets: true  # Enable Websocket testing
  supported_apis: ["json_rpc", "rest", "comet_bft", "websocket"]
  # ... rest of configuration

Reviewing PATH Logs

In E2E test mode, logs may be written to ./path_log_e2e_test_{timestamp}.txt.

In order to enable this, set the log_to_file field:

yq eval '.e2e_load_test_config.e2e_config.docker_config.log_to_file = true' -i ./e2e/config/.e2e_load_test.config.yaml

You should see the following log line at the bottom of the test summary:

===== 👀 LOGS 👀 =====

 ✍️ PATH container output logged to /tmp/path_log_e2e_test_1745527319.txt ✍️

===== 👀 LOGS 👀 =====

🌿 Grove Employees Only 🌿

Review the Anvil Shannon Beta TestNet Debugging Playbook if you believe the Anvil Supplier is broken.