Monitoring

Every Pocket Network service exposes Prometheus metrics. This guide covers what to monitor and where to find it.

Metrics Endpoints by Service

Service	Metrics Port	Key Metrics
poktroll (full node)	26660 (CometBFT)	Block height, peers, consensus state
RelayMiner (relayer)	9090	`ha_relayer_relays_served_total`, `ha_relayer_relays_rejected_total`, `ha_relayer_relay_latency_seconds`
RelayMiner (miner)	9092	`ha_miner_claims_created_total`, `ha_miner_claim_inclusion_outcome_total`, `ha_miner_proof_inclusion_outcome_total`, `ha_miner_leader_status`
PATH	9090	requests_total, relay_duration, reputation_score

What to Monitor

Critical Alerts (Set These First)

Condition	Severity	Action
A required proof not found on-chain	Critical	`ha_miner_proof_inclusion_outcome_total{outcome="on_chain_found"}` falls short of the settled claims that required a proof — lost rewards. Check miner logs and wallet balance.
Node wallet balance < 1 POKT	Warning	May not have enough for claim/proof tx fees. Top up immediately.
Block height stale > 5 min	Critical	Node out of sync — not earning. Check full node and peers.
Claims built but not on-chain	Warning	`ha_miner_claim_inclusion_outcome_total{outcome="on_chain_found"}` below `ha_miner_claims_created_total`. Check `pocket-relay-miner redis submissions`.
Relays refused	Warning	`ha_relayer_relays_rejected_total` rising; the `reason` label says why (for example `storage_saturated` means Redis is nearly full).

Operational Metrics

Relay throughput: ha_relayer_relays_served_total — track over time to understand volume trends and capacity utilization.

Latency: ha_relayer_relay_latency_seconds (end to end) and ha_relayer_backend_latency_seconds (your backend alone) — monitor p50, p95, p99. Degraded latency lowers your reputation with gateways.

Error rate: 5xx responses, timeout counts. High error rates trigger reputation penalties in PATH.

Session participation: Track how many sessions your node is selected for vs. total available sessions.

Owner Staking Dashboard

The easiest reward-tracking option for most operators. analytics.pocket.network/owner-staking — add up to 10 owner addresses (saved to your browser, no account needed) and track:

Rewards over time — per address or aggregated, over 24h/7d/30d/60d
Settlement events — every reward issuance with block height, service, relays, settled and minted amounts
Rewards by service — which chains earn most, from the service column
Mint ratio — the ratio applied at the latest settlement

Info

Owner Staking covers rewards and settlements, not claim/proof health. To catch missing or expired proofs, use the RelayMiner metrics below — an expired proof never produces a settlement, so it shows up as absence rather than an error.

Prometheus + Grafana Setup

Quick Start

The RelayMiner repository ships Prometheus and Grafana with seven ready dashboards in examples/observability, and the poktroll repository includes full node dashboards in localnet/grafana-dashboards/. For production:

Run Prometheus pointing at your service metrics endpoints
Import the RelayMiner and poktroll dashboards
Add alerts for the critical conditions listed above

Example Prometheus Config

yaml

scrape_configs:
  - job_name: 'poktroll'
    static_configs:
      - targets: ['localhost:26660']
  - job_name: 'relayer'
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'miner'
    static_configs:
      - targets: ['localhost:9092']

RelayMiner Monitoring

Beyond the alerts above, the RelayMiner exposes leader election status, Redis memory, claim tree, and submission metrics. Read them as identities in the order the repository’s metrics triage guide gives; a counter named after a loss, such as ha_miner_upokt_lost_total, counts failed attempts and is not evidence of a loss by itself.

Inspect Redis state with pocket-relay-miner redis, which decodes sessions, claim trees, and every claim and proof submission. See RelayMiner Operations.

Key Performance Baselines

Metric	Healthy Range
Relay latency (p95)	< 500ms for most EVM chains
Required proofs found on-chain	All of them (any missing = lost rewards)
Claims found on-chain / claims created	1.0
Block height lag	< 2 blocks behind network
Wallet balance	> 10 POKT (comfortable buffer for tx fees)