Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.celesto.ai/llms.txt

Use this file to discover all available pages before exploring further.

This guide covers common issues you might encounter when using SmolVM and how to resolve them.

​
Diagnostics

SmolVM includes a built-in diagnostic tool to check your system configuration: 
# Auto-detect backend and check prerequisites
smolvm doctor

# Check specific backend
smolvm doctor --backend firecracker
smolvm doctor --backend qemu

# CI-friendly JSON output
smolvm doctor --json --strict
The smolvm doctor command validates:
  • KVM availability (Linux/Firecracker)
  • Firecracker binary installation
  • QEMU installation and HVF support (macOS)
  • Network configuration (nftables, iproute2)
  • System permissions

​
Common issues

Problem: Image builds fail because Docker is not installed, the daemon is not running, or your user lacks permission to access it.SmolVM automatically diagnoses the specific Docker issue and returns a targeted error message. The three most common scenarios are:Docker not installed:
Docker is required to build images. Install Docker Desktop (macOS) or docker.io (Linux).
Daemon not running:
Docker is installed, but SmolVM could not reach the Docker daemon.
Start Docker Desktop or the Docker service and try again.
Permission denied:
Docker is installed, but this user cannot access the Docker daemon socket.
Make sure Docker Desktop is running or grant access to /var/run/docker.sock.
Solution:
  1. Install Docker if missing:
# macOS
brew install --cask docker

# Debian/Ubuntu
sudo apt-get install docker.io
  1. Start the Docker daemon:
# macOS
open -a Docker

# Linux
sudo systemctl start docker
  1. Fix socket permissions (Linux):
sudo usermod -aG docker $USER
newgrp docker  # Or log out and back in
You can also check Docker status programmatically before building:
from smolvm.build import ImageBuilder

builder = ImageBuilder()
if not builder.check_docker():
    error = builder.docker_requirement_error()
    print(error)  # Specific diagnosis and fix suggestion
Problem: The Firecracker backend requires KVM hardware virtualization.Solution:
  1. Verify KVM is available:
ls -l /dev/kvm
  1. If missing, enable virtualization in your BIOS/UEFI settings
  2. Add your user to the kvm group: 
sudo usermod -aG kvm $USER
newgrp kvm  # Or log out and back in
  1. Verify permissions:
# Should show rw-rw---- with kvm group
ls -l /dev/kvm
Alternative: Use the QEMU backend if KVM is unavailable:
from smolvm import SmolVM
vm = SmolVM(backend="qemu")
Problem: The firecracker executable is not in PATH.Solution:
  1. Run the system setup script:
sudo ./scripts/system-setup.sh --configure-runtime
  1. Or install manually using the HostManager:
from smolvm.host import HostManager

host = HostManager()
host.install_firecracker()
  1. Verify installation:
which firecracker
# Should show /usr/local/bin/firecracker or ~/.smolvm/bin/firecracker
Problem: User lacks permissions to create TAP networking devices.Solution:
  1. Run the network setup script:
sudo ./scripts/system-setup.sh --configure-runtime
  1. Or configure manually:
# Allow user to create TAP devices
sudo setcap cap_net_admin+ep $(which ip)

# Enable IP forwarding
sudo sysctl -w net.ipv4.ip_forward=1
echo 'net.ipv4.ip_forward=1' | sudo tee -a /etc/sysctl.conf
  1. Verify nftables is installed:
sudo nft list ruleset
Problem: VM starts successfully but SSH connection fails.Solution:
  1. Check VM status:
from smolvm import SmolVM

vm = SmolVM.from_id("vm-xxxxx")
info = vm.info
print(f"Status: {info.status}")
if info.network:
    print(f"IP: {info.network.guest_ip}")
    print(f"SSH Port: {info.network.ssh_host_port}")
vm.close()
  1. Test network connectivity:
# Ping guest IP (Firecracker backend)
ping 172.16.0.2

# Test SSH port forwarding
nc -zv localhost 2200
  1. Check firewall rules:
sudo nft list ruleset | grep 2200
  1. Examine VM logs:
cat ~/.local/state/smolvm/vm-xxxxx.log
  1. Increase SSH timeout:
from smolvm import SmolVM, VMConfig

config = VMConfig(ssh_timeout=60.0)  # Increase from default 30s
vm = SmolVM(config)
Problem: Multiple SmolVM processes trying to access the state database simultaneously.Solution:
  1. The state database uses exclusive locks for writes. Ensure only one process modifies VMs at a time.
  2. Check for stale processes: 
ps aux | grep firecracker
ps aux | grep qemu-system
  1. Clean up stale resources using the CLI:
smolvm cleanup --all
  1. If persistent, manually remove the lock:
# WARNING: Only do this if no SmolVM processes are running
rm ~/.local/state/smolvm/smolvm.db-wal
rm ~/.local/state/smolvm/smolvm.db-shm
Problem: All IPs in the 172.16.0.2-254 range are allocated.Solution:
  1. List all VMs:
smolvm list
  1. Clean up stopped or stale VMs:
smolvm cleanup --all
  1. The IP pool supports 253 concurrent VMs (172.16.0.2 through 172.16.0.254). If you need more, consider implementing a custom IP allocator.
Problem: QEMU process terminates immediately after start.Solution:
  1. Verify QEMU installation:
qemu-system-aarch64 --version
qemu-system-x86_64 --version
  1. Check HVF acceleration support:
qemu-system-aarch64 -accel help
# Should list 'hvf' for Hypervisor.framework
  1. Reinstall QEMU via Homebrew:
brew uninstall qemu
brew install qemu
  1. Check VM logs for kernel panic:
cat ~/.local/state/smolvm/vm-xxxxx.log
  1. Use auto-config mode to let SmolVM pick a compatible kernel and rootfs:
from smolvm import SmolVM

# Auto-config handles kernel/rootfs selection
with SmolVM() as vm:
    result = vm.run("echo works")
    print(result.stdout)
Problem: VM is marked as ERROR and cannot be restarted.Solution:
  1. Check VM details:
from smolvm import SmolVM

vm = SmolVM.from_id("vm-xxxxx")
info = vm.info
print(f"Status: {info.status}")
print(f"PID: {info.pid}")
vm.close()
  1. Examine logs:
cat ~/.local/state/smolvm/vm-xxxxx.log
  1. Delete the failed VM and create a fresh one:
# Delete the failed VM
vm = SmolVM.from_id("vm-xxxxx")
vm.delete()
vm.close()

# Create a fresh VM
with SmolVM() as new_vm:
    result = new_vm.run("echo 'back in business'")
    print(result.stdout)
  1. Run cleanup to remove all stale resources:
smolvm cleanup --all
Problem: SmolVM cannot create or write to any data directory.Solution:
  1. Check directory permissions:
ls -ld ~/.local/state/smolvm
ls -ld /var/lib/smolvm
  1. Create directory manually:
mkdir -p ~/.local/state/smolvm
chmod 755 ~/.local/state/smolvm
  1. Set explicit data directory:
from pathlib import Path
from smolvm import SmolVM

data_dir = Path("/tmp/smolvm-data")
manager = SmolVM(data_dir=data_dir)
  1. Use environment variable:
export SMOLVM_DATA_DIR=/tmp/smolvm-data

​
Debugging tips

​
Enable debug logging

import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

from smolvm import SmolVM

# Now you'll see detailed debug output
with SmolVM() as vm:
    vm.run("echo hello")

​
Inspect Firecracker socket

Manually query the Firecracker API: 
from smolvm.api import FirecrackerClient
from pathlib import Path

socket_path = Path("/tmp/fc-vm-xxxxx.sock")
client = FirecrackerClient(socket_path)
info = client.get_instance_info()
print(info)
client.close()

​
Check network rules

# List all nftables rules
sudo nft list ruleset

# Check NAT rules for specific VM
sudo nft list table nat | grep smolvm

# List TAP devices
ip link show | grep tap

# Show routes to guest VMs
ip route show | grep 172.16.0

​
Monitor VM processes

# List all Firecracker processes
ps aux | grep firecracker

# List all QEMU processes  
ps aux | grep qemu-system

# Check resource usage
top -p $(pgrep -d',' firecracker)

​
Getting help

If you’re still experiencing issues:
  1. Check the GitHub Issues for similar problems
  2. Run smolvm doctor --json and include the output in your bug report
  3. Include relevant logs from ~/.local/state/smolvm/*.log
  4. Join the Celesto AI Slack community

​
Known limitations

  • IP Pool: Maximum 253 concurrent VMs (172.16.0.2-254)
  • SSH Port Pool: Maximum 800 concurrent VMs (ports 2200-2999)
  • Firecracker: Linux only (requires KVM)
  • QEMU: Slower boot times compared to Firecracker
  • SSH Trust Model: Host keys not strictly verified by default (see SECURITY.md)
Last modified on May 5, 2026