Validator Operations Guide

This comprehensive guide explains how to operate an ENI validator node. We will cover the full lifecycle of a validator, from initial setup to ongoing operations and maintenance. Understanding these concepts is critical for maintaining reliable and secure validator operations.

Understanding Validator Responsibilities

Validators in the ENI network perform several key functions. As a validator, you are responsible for:

• Participating in consensus by proposing and validating blocks

• Maintaining high uptime and performance to avoid penalties

• Providing accurate asset pricing oracle data

• Managing delegator relationships and maintaining transparent operations

• Participating in governance and network upgrades

Initial Setup

Key Management

Validator security begins with proper key management. Your validator requires several distinct keys:

# Validator Consensus Key - Used for signing blocks
enid tendermint show-validator

# Operator Key - Used for managing validator operations
enid keys add operator

# Oracle Key - Used for price submissions
enid keys add oracle

These keys serve different purposes and should be managed with appropriate security measures. The consensus key, stored in priv_validator_key.json, is particularly critical as it is used to sign blocks.

Hardware Security Module (HSM) Integration

For production validators, using an HSM is strongly recommended. Here’s how to configure an HSM with your validator:

HSM Configuration Steps

```bash
# Install required libraries
sudo apt-get install opensc pkcs11-utils

# Configure YubiHSM2
yubihsm-connector -d

# Generate a key in the HSM
yubihsm-shell

# Configure enid to use the HSM
tee "$HOME/.eni/config/priv_validator_config.json" << EOF
{
    "chain_id": "eni-chain",
    "key_type": "yubihsm",
    "state_file": "$HOME/.eni/data/priv_validator_state.json",
    "hsm_serial": "YOUR_HSM_SERIAL",
    "hsm_key_id": "YOUR_KEY_ID"
}
EOF
```

Oracle Price Feeder Setup

As an ENI validator, you must run a price feeder to provide oracle data. This is critical for network operations.

First, install the price feeder:

cd eni-chain
make install-price-feeder

Create a configuration file for your price feeder:

Price Feeder Configuration

```toml
gas_adjustment = 1.5
gas_prices = "0.01ueni"
enable_server = true
enable_voter = true
provider_timeout = "500ms"

[server]
listen_addr = "0.0.0.0:7171"
read_timeout = "20s"
verbose_cors = true
write_timeout = "20s"

[keyring]
backend = "file"
dir = "/home/eni/.eni"

[rpc]
grpc_endpoint = "localhost:9090"
rpc_timeout = "500ms"
tmrpc_endpoint = "http://localhost:26657"

[telemetry]
enable_hostname = true
enable_hostname_label = true
enable_service_label = true
prometheus_retention = 120
service_name = "price-feeder"

[[provider_endpoints]]
name = "binance"
rest = "https://api1.binance.com"
websocket = "stream.binance.com:9443"
```

Start the price feeder as a service:

sudo tee /etc/systemd/system/price-feeder.service << EOF
[Unit]
Description=ENI Price Feeder
After=network-online.target

[Service]
User=$USER
ExecStart=$(which price-feeder) /path/to/config.toml
Restart=always
RestartSec=3
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable price-feeder
sudo systemctl start price-feeder

Monitoring and Alerts

Validator-Specific Metrics

In addition to basic node monitoring, validators should track additional metrics:

# Check validator status
enid query staking validator $(enid keys show -a $VALIDATOR_KEY)

# Monitor signing status
enid query slashing signing-info $(enid tendermint show-validator)

# Check current delegations
enid query staking delegations-to $(enid keys show -a $VALIDATOR_KEY)

Alert Configuration

Set up critical alerts for validator operations. Below are basic Prometheus alert rules:

Validator Alert Rules

```yaml
groups:
- name: validator_alerts
  rules:
  - alert: ValidatorMissedBlocks
    expr: increase(tendermint_consensus_validator_missed_blocks[1h]) > 0
    labels:
      severity: critical
    annotations:
      summary: 'Validator missed blocks'

  - alert: ValidatorJailed
    expr: tendermint_consensus_validator_status == 0
    labels:
      severity: critical
    annotations:
      summary: 'Validator has been jailed'

  - alert: OraclePriceFeedDelay
    expr: time() - eni_oracle_price_timestamp > 300
    labels:
      severity: critical
    annotations:
      summary: 'Oracle price feed delayed'
```

Security Practices

Network Security

Implement a sentry node architecture to protect your validator:

# Validator node config.toml
[p2p]
pex = false
persistent_peers = "sentry_node_id@sentry_node_ip:26656"
private_peer_ids = ""
addr_book_strict = false

# Sentry node config.toml
[p2p]
pex = true
private_peer_ids = "validator_node_id"
addr_book_strict = true

Key Management Practices

Implement a secure key backup procedure:

Key Backup Script

```bash
#!/bin/bash
# Create an encrypted backup of validator keys
BACKUP_DIR="/secure/validator/backup"
DATE=$(date +%Y%m%d)

# Backup validator key
tar czf - $HOME/.eni/config/priv_validator_key.json | \
gpg --symmetric --cipher-algo AES256 \
-o $BACKUP_DIR/validator_key_$DATE.tar.gz.gpg

# Backup keyring
tar czf - $HOME/.eni/keyring-file | \
gpg --symmetric --cipher-algo AES256 \
-o $BACKUP_DIR/keyring_$DATE.tar.gz.gpg

# Create SHA256 checksums
sha256sum $BACKUP_DIR/*.gpg > $BACKUP_DIR/checksums_$DATE.txt
```

Maintenance Procedures

Scheduled Maintenance

When performing scheduled maintenance:

# Notify delegators (recommended at least 24 hours in advance)
# Consider posting to:
# - On-chain governance forum
# - Social media channels
# - Validator website

# Gracefully stop the validator
sudo systemctl stop enid

# Perform maintenance tasks

# Restart services
sudo systemctl start enid
sudo systemctl start price-feeder

Emergency Procedures

Create an emergency response plan:

Emergency Response Procedures

```bash
# 1. If double-signing is detected:
sudo systemctl stop enid
# Check priv_validator_state.json
# Contact team and delegators

# 2. If node is stuck:
enid status
# Check for consensus failure
journalctl -u enid -n 100
# Attempt a safe restart
sudo systemctl restart enid

# 3. If oracle feed fails:
systemctl status price-feeder
# Check price-feeder logs
journalctl -u price-feeder -n 100
# Restart if necessary
sudo systemctl restart price-feeder
```

Governance Participation

As a validator, you have a responsibility to participate in governance. Monitor and vote on proposals:

# List active proposals
enid query gov proposals --status voting_period

# Vote on a proposal
enid tx gov vote 1 yes \
    --from operator \
    --chain-id eni-chain \
    --gas auto \
    --gas-prices 0.01ueni

Validator Economics

Understanding validator economics is critical for long-term success:

• Commission Rate Strategy: Set competitive rates while ensuring operational sustainability

• Delegation Management: Maintain good delegator relationships through transparent communication

• Reward Distribution: Rewards are distributed in real-time as blocks are produced

• Penalty Risks: Understand and mitigate penalty risks through proper operations

Recovery Procedures

Validator Recovery

If you need to recover your validator on a new machine:

# 1. Set up a new machine with Eni node
# 2. Copy secure backup files
# 3. Restore validator key
gpg -d validator_key_backup.tar.gz.gpg | tar xzf -
# 4. Restore keyring
gpg -d keyring_backup.tar.gz.gpg | tar xzf -
# 5. Start services
sudo systemctl start enid
sudo systemctl start price-feeder

This guide provides the foundation for operating an ENI validator. Keep in mind that validator operations require ongoing attention to security, performance, and network participation. Stay engaged with the ENI community and keep informed about network developments.