Validator Jumpstart

A guide for setting up validators, blazingly fast. I have been playing around with different systems, actively switching validator machines, so this acts as a guide for me personally. This is an opinionated guide that covers how to setup a validator, the hardware requirements and settings I like to tune.

Basic Overview

System recommendation refer to Solanahcl list by ferric / StakeWare

Three or more disks are required with the following configuration:

SSD primary OS (~500 GB)
NVMe Ledger (≥2TB)
NVMe Accounts and snapshot (≥2TB)

Base OS: Ubuntu 22.04

Disk Setup

Directory structure:

Ledger Disk → /mnt/ledger
Account & Snapshot Disk → /mnt/extras
- /mnt/extras/snapshot (For Snapshots)
- /mnt/extras/accounts (For Accounts)

Setup Steps

Format the block

sudo mkfs -t ext4 /dev/nvme0n1

Spin up directory + give sol user permission

sudo chown -R sol:sol <PATH TO DIR>

Mount to the directory

sudo mount /dev/nvme0n1 <PATH TO DIR>

Ports Opening

Note: RPC port remains closed, only SSH and gossip ports are opened.

For new machines with UFW disabled:

Add OpenSSH first to prevent lockout if you don't have password access
Open required ports:

sudo ufw allow 8000:8020/tcp

sudo ufw allow 8000:8020/udp

System Tuning and Validator Setup

System Performance Optimization

Kernel and Network Tuning

Create and run the following script to optimize system performance:

#!/bin/bash

# Set sysctl performance variables
cat >> /etc/sysctl.conf <<- EOM
# TCP Buffer Sizes (10k min, 87.38k default, 12M max)
net.ipv4.tcp_rmem=10240 87380 12582912
net.ipv4.tcp_wmem=10240 87380 12582912

# TCP Optimization
net.ipv4.tcp_congestion_control=westwood
net.ipv4.tcp_fastopen=3
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_sack=1
net.ipv4.tcp_low_latency=1
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_no_metrics_save=1
net.ipv4.tcp_moderate_rcvbuf=1

# Kernel Optimization
kernel.timer_migration=0
kernel.hung_task_timeout_secs=30
kernel.pid_max=49152

# Virtual Memory Tuning
vm.swappiness=30
vm.max_map_count=2000000
vm.stat_interval=10
vm.dirty_ratio=40
vm.dirty_background_ratio=10
vm.min_free_kbytes=3000000
vm.dirty_expire_centisecs=36000
vm.dirty_writeback_centisecs=3000
vm.dirtytime_expire_seconds=43200

# Solana Specific Tuning
net.core.rmem_max=134217728
net.core.rmem_default=134217728
net.core.wmem_max=134217728
net.core.wmem_default=134217728
EOM

# Reload sysctl settings
sysctl -p

# Set CPU governor to performance mode
echo 'GOVERNOR="performance"' | tee /etc/default/cpufrequtils
echo "performance" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# Set performance governor for bare metal (ignore errors)
echo "performance" | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor || true

Session File Limits

Choose one of the following configurations:

Service-specific limits in /etc/systemd/system.conf:

[Service]
LimitNOFILE=1000000

System-wide limits in /etc/systemd/system.conf:

[Manager]
DefaultLimitNOFILE=1000000

Validator Setup

Installing Agave/Jito Client

Grant execution permissions to the install script:

chmod +x bin/ice-staking/start/init.sh

Run the installation with specific version tag:

bin/ice-staking/start/init.sh -t v1.18.23-jito

Post-Installation Setup

Create symlink for Jito client (if used):

ln -sf /home/sol/.local/share/solana/install/releases/v1.18.15-jito/bin /home/sol/.local/share/solana/install/active_release/

Add the following to your .bashrc or .bash_profile:

# Environment Setup
. "$HOME/.cargo/env"
export PATH="/home/sol/.local/share/solana/install/active_release/bin:$PATH"

# Helpful Aliases
alias catchup='solana catchup --our-localhost'
alias monitor='solana-validator --ledger /mnt/ledger monitor'
alias logtail='tail -f /home/sol/solana-validator.log'

Start script Use the start script here, specifically configured for a voting validator node. Note that the configuration includes modifications to support RPC functionality. additional flag for RPC node here

Additional Resources

Installation script source: ice-staking repository

Hot-Swap Validator Setup Guide

Overview

This guide describes how to set up two servers for hot-swapping to maintain 100% uptime during system changes. The process follows the Identity Transition methodology by Pumpkin.

Identity Keypair Configuration

Required Keypairs

Unstaked Keypair (unstaked.json)
- Functions as a burner keypair
- Maintains zero SOL balance to prevent voting capabilities
Staked Keypair (staked.json)
- Serves as the primary staked keypair
- Used for validator transitions when needed

Transferring Keypairs

Transfer the keypairs to your validator server using SCP:

scp <source_files> ice-ams:

Note: Customize the SSH configuration according to your setup. Ensure proper permissions are set for the sol user after transfer.

Log Rotation Configuration

Create and implement log rotation for validator logs:

cat > logrotate.sol <<EOF
/home/sol/solana-validator.log {
    rotate 7
    daily
    missingok
    postrotate
        systemctl kill -s USR1 sol.service
    endscript
}
EOF

sudo cp logrotate.sol /etc/logrotate.d/sol
systemctl restart logrotate.service

Systemd Service Configuration

Create a systemd service file for the Solana validator:

[Unit]
Description=Solana Validator
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=sol
LimitNOFILE=1000000
LogRateLimitIntervalSec=0
Environment="SOLANA_METRICS_CONFIG=host=https://metrics.solana.com:8086,db=mainnet-beta,u=mainnet-beta_write,p=password"
Environment="PATH=/home/sol/bin:/home/sol/.local/share/solana/install/active_release/bin:/home/sol/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
ExecStart=/home/sol/bin/ice-staking/start/start.sh mainnet-beta

[Install]
WantedBy=multi-user.target

Service Management Commands

Start Service

sudo systemctl enable --now sol

Stop Service

sudo systemctl stop sol

Restart Service

sudo systemctl restart sol

After this check the log file snapshot download should have started

tail -f solana-validator.log

Solana Validator Operations Guide

Metrics & Monitoring Solutions

1. Built-in Dashboard Options

Solana Metrics Dashboard
- Official solution from Solana Labs
- Access via URL specified in service file
- Provides real-time validator performance metrics

2. Third-Party Solutions

Stakeconomy's SolanaMonitoring
- Repository: github.com/stakeconomy/solanamonitoring
- Community-maintained monitoring solution
- Features:
  - Performance tracking
  - Health checks

3. Custom Monitoring Stack

Grafana + InfluxDB Setup
- Fully customizable metrics visualization
- Time-series data storage
- Benefits:
  - Custom dashboards
  - Historical data analysis

Active Monitoring Tools

1. Solana Watcher

Official monitoring tool by Solana Labs
Documentation: docs.solanalabs.com/operations/best-practices/monitoring
Features:
- Automated health checks
- System alerts

2. Stakewiz Update bot Integration

Telegram notification system
Real-time alerts and updates

Security Best Practices

1. Firewall Configuration

Only open required ports
Implement port-specific rules
Regular audit of open ports
Use UFW (Uncomplicated Firewall) for simple management

2. User Management

✅ Run validator with non-root user
❌ Avoid running as root
❌ Validator user should not have sudo privileges
Create a dedicated service account for validator operations

3. SSH Security

Disable password authentication
Use SSH keys exclusively
Consider:
- Custom SSH port
- Key-based authentication only
- Rate limiting for failed attempts

4. Keypair Security

Secure storage of validator keypairs
Best practices:
- Encrypted backups
- Access control logs

Credits

Solana Labs / docs
Tim Garcia / youtube
Overclock / setup guide
Ferric / StakeWare
Pumpkin's Pool / Pumpkin's pool

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
.github/workflows		.github/workflows
LICENSE		LICENSE
README.md		README.md
_config.yml		_config.yml

License

aldrin-labs/validator-jumpstart

Folders and files

Latest commit

History

Repository files navigation