# Node Operators Guide

This guide covers the detailed operational aspects of running an ENI node, including configuration management, maintenance procedures, and best practices for stable and efficient operation.

### Configuration Management

#### Directory Structure

The configuration for an ENI node is stored in the `~/.eni/config/` directory:

```bash
~/.eni/config/
├── app.toml          # Application configuration (gas fees, API settings, pruning, etc.)
├── client.toml       # CLI and client-related settings
├── config.toml       # Core Tendermint settings (network, consensus, and RPC)
├── genesis.json      # Genesis file for the chain, defining the initial state
├── node_key.json     # Unique node identity key for peer-to-peer (P2P) networking
└── priv_validator_key.json  # Validator private signing key (if running as a validator)
```

#### Key Configuration Parameters

**Network Settings (config.toml)**

```toml
[p2p]
# Public IP for other nodes to reach you
external_address = "your-public-ip:26656"
# Local address to listen for incoming P2P connections
laddr = "tcp://0.0.0.0:26656"
# Allowed number of peers
max_num_inbound_peers = 40
max_num_outbound_peers = 20
# Network bandwidth limits to prevent congestion
send_rate = 20480000  # 20MB/s
recv_rate = 20480000  # 20MB/s

[rpc]
# RPC listening address
laddr = "tcp://0.0.0.0:26657"
# Maximum concurrent connections
max_open_connections = 900
# Transaction confirmation timeout
timeout_broadcast_tx_commit = "10s"
```

**Application Settings (app.toml)**

```toml
# Minimum gas prices [to prevent spam transactions]
minimum-gas-prices = "0.01ueni"

[api]
# Enable API server
enable = true
max-open-connections = 1000

[state-commit]
# Use ENI-DB for improved performance
sc-enable = true

[state-store]
# Enable state store for historical queries
ss-enable = true
# Keep 100,000 blocks for querying
# 0 = "keep all"
ss-keep-recent = 100000
```

### Database Management

#### Database Types

ENI supports two database backends:

1. **ENI-DB (Recommended)**

* Optimized for performance and sync time
* Reduced resource usage
* Suitable for all nodes

2. **Traditional IAVL DB**

* Standard Cosmos SDK database
* More extensively tested

#### ENI-DB Configuration

```toml
[state-commit]
sc-enable = true
sc-async-commit-buffer = 100
sc-keep-recent = 1  # Keep only the latest state for performance
sc-snapshot-interval = 10000  # State snapshot every 10,000 blocks

[state-store]
ss-enable = true
ss-backend = "pebbledb"  # Default, required
ss-async-write-buffer = 100
ss-keep-recent = 100000  # Keep the most recent 100,000 blocks
ss-prune-interval = 600  # Pruning cleanup interval
```

Setting a very small \[more frequent] pruning interval may cause conflicts with automatic snapshots. A very large \[less frequent] pruning interval means longer total pruning time, which could lead to missed blocks and extended resync times.

#### Database Maintenance

The database is generally stable and can run without intervention, but occasional attention may be required:

```bash
# Check database size
du -sh ~/.eni/data/

# Confirm the correct file paths based on your setup before proceeding
# Before performing database cleanup, ensure you back up critical files:
# - `node_key.json`: Preserves node identity for maintaining peer connections.
# - `priv_validator_key.json`: Validator critical; losing this risks double-signing.
# - `config.toml` and `app.toml`: Retain node-specific configuration.
# - `genesis.json`: Required for proper chain initialization.

# Compact the database to optimize storage (only if needed)
find ~/.eni/data/ -mindepth 1 ! -name 'priv_validator_state.json' -delete && rm -rf ~/.eni/wasm

# Back up the database
cp -r ~/.eni/data/ ~/eni-backup-$(date +%Y%m%d)/
```

### Service Management

#### Systemd Commands

```bash
# Check service status
systemctl status enid

# Start service
systemctl start enid

# Stop service
systemctl stop enid

# Restart service
systemctl restart enid

# View logs in real-time
journalctl -fu enid -o cat
```

#### Log Management

Enable log rotation to prevent logs from consuming excessive disk space:

```bash
sudo tee /etc/logrotate.d/eni > /dev/null << EOF
/var/log/eni/*.log {
    daily
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 eni eni
    sharedscripts
    postrotate
        systemctl reload enid
    endscript
}
EOF
```

### Update Procedures

#### Minor Updates

For minor updates that do not break consensus:

```bash
# Stop the node
sudo systemctl stop enid

# Update the binary
cd eni-chain
git fetch --all
git checkout [new-version]
make install

# Restart the node
sudo systemctl restart enid
```

#### Major Updates

For major upgrades introducing state-breaking changes:

1. Wait for the specified upgrade block height \[visible in the upgrade proposal’s 'plan']
2. The node will stop automatically.
3. Update/replace the binary
4. Restart the node.

```bash
# After the node stops
cd eni-chain
git pull
git checkout [new-version]
make install
sudo systemctl restart enid
```

*Tip - Build the upgrade before the halt height so it can be quickly swapped in, minimizing downtime.*

### Performance Optimization

The effectiveness of performance optimizations may vary depending on system hardware, workload, and network conditions. Research and test any changes in a controlled environment before implementation to ensure compatibility with your specific configuration and requirements. Always back up critical data before making modifications.

#### Memory Management (sysctl Tuning)

Optimizing memory management settings can improve performance and stability, especially for high-load nodes. These settings control swap usage and the handling of dirty pages in RAM.

```bash
vm.swappiness = 1  # Reduce swapping to disk, ensuring RAM is used effectively before relying on slower swap space
vm.dirty_background_ratio = 3  # Percentage of system memory that can be filled with "dirty" pages before background writing starts
vm.dirty_ratio = 10  # Maximum percentage of system memory that can be filled with "dirty" pages before triggering a full flush
vm.dirty_expire_centisecs = 300  # Time before dirty data is written to disk (in hundredths of a second)
vm.dirty_writeback_centisecs = 100  # Frequency at which "dirty" pages are written to disk (in hundredths of a second)
```

#### Network Stack Optimization

Tuning the network stack can improve packet handling efficiency and throughput, particularly for nodes handling many peers and high transaction volumes.

```bash
net.core.somaxconn = 32768  # Maximum number of queued connections
net.core.netdev_max_backlog = 32768  # Number of queued packets allowed before dropping incoming packets
net.ipv4.tcp_max_syn_backlog = 16384  # Number of outstanding SYN requests (half-open connections) allowed before dropping new ones
net.core.rmem_max = 16777216  # Receive buffer size for network sockets
net.core.wmem_max = 16777216  # Send buffer size for network sockets
```

#### Storage Optimization

Optimizing storage settings can significantly reduce write latency and improve database performance, especially for nodes using NVMe SSDs.

```bash
echo "none" > /sys/block/nvme0n1/queue/scheduler  # Disable I/O scheduler to reduce latency
blockdev --setra 4096 /dev/nvme0n1  # Read-ahead value to optimize sequential reads
```

### Backup and Recovery

#### Regular Backups

Automate backups to prevent data loss:

```bash
#!/bin/bash
BACKUP_DIR="/backup/eni"
DATE=$(date +%Y%m%d)

# Stop the service
systemctl stop enid

# Create the backup
tar czf $BACKUP_DIR/eni-backup-$DATE.tar.gz ~/.eni/

# Restart the service
systemctl start enid
```

#### Recovery Procedure

Restore from a backup in case of corruption or accidental deletion:

```bash
# Stop the node
systemctl stop enid

# Remove corrupted data
rm -rf ~/.eni/data/

# Restore from backup
tar xzf eni-backup-[date].tar.gz -C /

# Restart the node
systemctl start enid
```

### Troubleshooting

#### Common Issues and Solutions

1. **Synchronization Issues**

* Check available disk space (`df -h`)
* Ensure proper peer connections (`enid net info`)
* Verify firewall settings (port 26656 open)

2. **Performance Issues**

* Monitor system resources (`htop` or `iotop`)
* Check disk I/O performance (`iostat`)
* Analyze network traffic (`iftop`)

3. **Database Issues**

* Run a database integrity check with:

```bash
enid debug dump-db | grep -i error
```

If errors are detected, consider restoring from a recent backup.

* Consider pruning excessive historical data by adjusting `ss-keep-recent` in `app.toml`, or run:

```bash
enid unsafe-reset-all --home=$HOME/.eni --keep-addr-book
```

Alternatively, manually delete old state snapshots to free up space:

```bash
rm -rf ~/.eni/data/snapshots/*
```

#### Diagnostic Commands

```bash
# Check sync status
enid status | jq .SyncInfo

# View peer count
enid net info | jq '.n_peers'

# Check validator status
enid query staking validator $(enid keys show -a $VALIDATOR_KEY)

# View recent blocks
enid query blocks recent
```

### Security Considerations

* Use a firewall and rate limiting to prevent attacks
* Keep the system and node software updated
* Secure SSH access with key-based authentication
* Protect validator keys with offline storage or a hardware security module (HSM)

For more in-depth system and configuration guidance, refer to the *\[Advanced Configuration and Monitoring Guide]*(./advanced-config-monitoring).