qvm/README.md

# QVM - Lightweight QEMU Development VM Wrapper

A standalone CLI tool for running commands in an isolated NixOS VM with persistent state and shared caches.

## Motivation

Running AI coding agents in isolation presents a security challenge. Containers provide some isolation, but kernel exploits remain a real attack surface. QVM uses full VM isolation for stronger security guarantees while maintaining developer ergonomics.

**Why QVM?**

- **VM isolation over container isolation** - Hypervisor boundary is fundamentally stronger than kernel namespaces
- **One master image, shared caches** - Single ~7GB base image instead of per-project images
- **Transparent workspace mounting** - Current directory automatically available in VM
- **Persistent state** - VM overlay preserves installed tools and configuration
- **Shared build caches** - Cargo, pnpm, and sccache caches shared across all projects

**Primary use case:** Running AI coding agents (opencode, aider, cursor) in isolation to prevent unintended host filesystem access while maintaining build cache performance.

## Installation

### System-wide installation (NixOS flake)

Add QVM to your NixOS configuration:

```nix
{
  inputs.qvm.url = "github:yourusername/qvm";
  
  environment.systemPackages = [
    inputs.qvm.packages.${system}.default
  ];
}
```

### Direct execution

Run without installation:

```bash
nix run github:yourusername/qvm -- start
```

### Development shell

For local development:

```bash
git clone https://github.com/yourusername/qvm
cd qvm
nix develop
```

## Quick Start

```bash
# Start the VM (auto-builds base image on first run)
qvm start

# Run commands in VM with current directory mounted
qvm run opencode
qvm run cargo build --release
qvm run npm install

# Interactive SSH session
qvm ssh

# Check VM status
qvm status

# Stop VM
qvm stop
```

The first `qvm start` will build the base image, which takes several minutes. Subsequent starts are fast (3-5 seconds).

## Commands

### qvm start

Start the VM daemon. Creates base image and overlay if they don't exist.

```bash
qvm start
```

**Behavior:**
- Checks if VM already running (no-op if yes)
- Builds base image if missing (via `qvm rebuild`)
- Creates overlay.qcow2 if missing (copy-on-write backed by base image)
- Launches QEMU with KVM acceleration
- Mounts all registered workspaces and cache directories via 9p
- Waits for SSH to become available
- Exits when SSH is ready (VM runs as background daemon)

### qvm stop

Gracefully stop the running VM.

```bash
qvm stop
```

**Behavior:**
- Sends ACPI shutdown signal to VM
- Waits up to 30 seconds for graceful shutdown
- Force kills QEMU if timeout exceeded
- Cleans up PID file
- No-op if VM not running

### qvm run

Execute a command in the VM with current directory mounted as workspace.

```bash
qvm run <command> [args...]
```

**Examples:**
```bash
qvm run cargo build
qvm run npm install
qvm run "ls -la"
qvm run bash              # Interactive shell in VM
```

**Behavior:**
1. Generates hash from current directory absolute path
2. Registers workspace in `~/.local/state/qvm/workspaces.json`
3. Auto-starts VM if not running
4. Checks if workspace is mounted (warns to restart VM if newly registered)
5. SSHs into VM and executes: `cd /workspace/{hash} && <command>`
6. Streams stdout/stderr to terminal
7. Exits with command's exit code
8. VM stays running for next command

**Note:** Workspaces are mounted at VM startup. If you run from a new directory, you'll need to restart the VM to mount it.

### qvm ssh

Open interactive SSH session or run command in VM.

```bash
qvm ssh                   # Interactive shell
qvm ssh -c "command"      # Run single command
```

**Examples:**
```bash
qvm ssh                           # Drop into root shell
qvm ssh -c "systemctl status"     # Check systemd status
qvm ssh -c "df -h"                # Check disk usage
```

Connects as root user. Password: `root` (if needed, though SSH key auth is configured).

### qvm status

Show VM state, SSH port, and mounted workspaces.

```bash
qvm status
```

**Example output:**
```
VM Status: Running
PID: 12345
SSH Port: 2222

Mounted Workspaces:
  abc12345 -> /home/josh/projects/qvm
  def67890 -> /home/josh/projects/myapp

Cache Directories:
  /cache/cargo     -> ~/.cache/qvm/cargo-home
  /cache/target    -> ~/.cache/qvm/cargo-target
  /cache/pnpm      -> ~/.cache/qvm/pnpm-store
  /cache/sccache   -> ~/.cache/qvm/sccache
```

### qvm rebuild

Rebuild the base VM image from NixOS flake.

```bash
qvm rebuild
```

**Behavior:**
- Runs `nix build` on `~/.config/qvm/flake`
- Copies result to `~/.local/share/qvm/base.qcow2`
- Warns if VM is running (restart required to use new image)

**Use when:**
- Customizing VM configuration (edited `~/.config/qvm/flake/flake.nix`)
- Updating base image with new packages or settings
- Pulling latest changes from upstream NixOS

### qvm reset

Delete overlay and start fresh. Keeps base image intact.

```bash
qvm reset
```

**Behavior:**
- Stops VM if running
- Deletes `overlay.qcow2` (all VM state changes)
- Deletes `workspaces.json` (registered workspaces)
- Next `qvm start` creates fresh overlay from base image

**Use when:**
- VM state is corrupted
- Want to return to clean base image state
- Testing fresh install scenarios

## Directory Layout

QVM uses XDG-compliant directories:

```
~/.config/qvm/
  └── flake/              # User's customizable NixOS flake
      ├── flake.nix       # VM system configuration
      └── flake.lock      # Pinned dependencies

~/.local/share/qvm/
  └── base.qcow2          # Base VM image (~7GB, read-only)

~/.local/state/qvm/
  ├── overlay.qcow2       # Persistent VM state (copy-on-write)
  ├── vm.pid              # QEMU process ID
  ├── ssh.port            # SSH forwarded port
  ├── serial.log          # VM console output
  └── workspaces.json     # Registered workspace mounts

~/.cache/qvm/
  ├── cargo-home/         # Shared cargo registry/cache
  ├── cargo-target/       # Shared cargo build artifacts
  ├── pnpm-store/         # Shared pnpm content-addressable store
  └── sccache/            # Shared compilation cache
```

### Inside the VM

```
/workspace/
  ├── abc12345/          # Mounted from /home/josh/projects/qvm
  └── def67890/          # Mounted from /home/josh/projects/myapp

/cache/
  ├── cargo/             # Mounted from ~/.cache/qvm/cargo-home
  ├── target/            # Mounted from ~/.cache/qvm/cargo-target
  ├── pnpm/              # Mounted from ~/.cache/qvm/pnpm-store
  └── sccache/           # Mounted from ~/.cache/qvm/sccache
```

Environment variables in VM:
- `CARGO_HOME=/cache/cargo`
- `CARGO_TARGET_DIR=/cache/target`
- `PNPM_HOME=/cache/pnpm`
- `SCCACHE_DIR=/cache/sccache`

## Workspace Management

When you run `qvm run` from different directories, each gets registered and mounted:

```bash
cd ~/projects/myapp
qvm run cargo build       # Workspace abc12345

cd ~/projects/other
qvm run npm install       # Workspace def67890
```

Both workspaces are mounted simultaneously in the VM. The hash is derived from the absolute path, so the same directory always maps to the same workspace ID.

**Important:** Workspaces are mounted at VM start time. If you run from a new directory:
1. Workspace gets registered in `workspaces.json`
2. `qvm run` will detect it's not mounted and warn you
3. Restart VM to mount: `qvm stop && qvm start`

## Cache Sharing

Build caches are shared between host and VM via 9p mounts:

**Cargo:**
- Registry and crate cache shared across all projects
- Each project still uses its own `Cargo.lock`
- Different versions coexist peacefully

**pnpm:**
- Content-addressable store shared
- Each project links to shared store
- Massive disk space savings

**sccache:**
- Compilation cache shared
- Speeds up repeated builds across projects

All caches persist across VM restarts and resets (they live in `~/.cache/qvm`, not in the overlay).

## Customization

### Editing VM Configuration

The VM is defined by a NixOS flake at `~/.config/qvm/flake/flake.nix`. Edit this file to customize the VM.

**Default packages included:**
- git, vim, tmux, htop
- curl, wget, jq, ripgrep, fd
- opencode (AI coding agent)
- Language toolchains can be added

**Example: Add Rust and Node.js:**

```nix
environment.systemPackages = with pkgs; [
  # ... existing packages ...
  
  # Add language toolchains
  rustc
  cargo
  nodejs_22
  python3
];
```

**Example: Add custom shell aliases:**

```nix
environment.shellAliases = {
  "ll" = "ls -lah";
  "gst" = "git status";
};
```

**Example: Include your dotfiles:**

```nix
environment.etc."vimrc".source = /path/to/your/vimrc;
```

**Apply changes:**

```bash
# Edit the flake
vim ~/.config/qvm/flake/flake.nix

# Rebuild base image
qvm rebuild

# Restart VM to use new image
qvm stop
qvm start
```

### Resource Allocation

Default resources:
- **Memory:** 8GB
- **CPUs:** 4 cores
- **Disk:** 20GB

To customize, set environment variables before `qvm start`:

```bash
export QVM_MEMORY="16G"
export QVM_CPUS="8"
qvm start
```

These will be configurable in a config file in future versions.

## Security Model

### VM Isolation Benefits

**Why VM over containers?**
- Container escapes via kernel exploits are well-documented
- VM escapes require hypervisor exploits (far rarer)
- For long-running unattended AI sessions, VM isolation provides stronger guarantees

### 9p Mount Security

Filesystem sharing uses `security_model=mapped-xattr`:
- No direct passthrough to host filesystem
- Only explicitly mounted directories are visible
- Host filesystem outside mounts is completely invisible to VM
- File ownership and permissions mapped via extended attributes

**What the VM can access:**
- Registered workspaces (directories you ran `qvm run` from)
- Shared cache directories
- Nothing else on the host

**What the VM cannot access:**
- Your home directory (except mounted workspaces)
- System directories
- Other users' files
- Any unmounted host paths

### Network Isolation

VM uses QEMU user-mode networking:
- VM can make outbound connections
- No inbound connections to VM except forwarded SSH port
- SSH port forwarded to random high port on localhost only

## Troubleshooting

### VM won't start

Check if QEMU/KVM is available:

```bash
qemu-system-x86_64 --version
lsmod | grep kvm
```

Check the serial log for errors:

```bash
cat ~/.local/state/qvm/serial.log
```

### SSH timeout

If SSH doesn't become ready within 60 seconds:

1. Check if VM process is running: `ps aux | grep qemu`
2. Check serial log: `cat ~/.local/state/qvm/serial.log`
3. Try increasing timeout (future feature)

### Workspace not mounted

If you see "Workspace not mounted in VM":

```bash
qvm stop
qvm start
qvm run <command>
```

Workspaces must be registered before VM start.

### Build cache not working

Verify cache directories exist and are mounted:

```bash
qvm ssh -c "ls -la /cache"
qvm ssh -c "echo \$CARGO_HOME"
```

Check that cache directories on host exist:

```bash
ls -la ~/.cache/qvm/
```

### VM is slow

Ensure KVM is enabled (not using emulation):

```bash
lsmod | grep kvm_intel  # or kvm_amd
```

Check resource allocation:

```bash
qvm status  # Shows allocated CPUs/memory (future feature)
```

## Limitations

**Explicit exclusions:**
- **Multi-VM:** Only one VM at a time
- **Per-project configs:** Single global VM (use other tools for project-specific VMs)
- **Cross-platform:** Linux + KVM only (no macOS/Windows)
- **GUI:** Headless only, no desktop environment
- **Snapshots:** Only overlay reset, no checkpoint/restore

## Dependencies

Required on host system:
- `qemu` (with KVM support)
- `nix` (for building VM images)
- `openssh` (SSH client)
- `jq` (JSON processing)
- `nc` (netcat, for port checking)

All dependencies are included automatically when installing via Nix flake.

## Architecture

```
HOST                                VM
────────────────────────────────    ──────────────────────────

~/.local/share/qvm/base.qcow2   →   (read-only base image)
         ↓
~/.local/state/qvm/overlay.qcow2 →  (persistent changes)


~/.cache/qvm/cargo-home/    ──9p──→  /cache/cargo/
~/.cache/qvm/cargo-target/  ──9p──→  /cache/target/
~/.cache/qvm/pnpm-store/    ──9p──→  /cache/pnpm/
~/.cache/qvm/sccache/       ──9p──→  /cache/sccache/

$(pwd)                      ──9p──→  /workspace/{hash}/
```

**Image layering:**
- Base image contains NixOS system from flake
- Overlay is copy-on-write layer for runtime changes
- `qvm reset` deletes overlay, preserves base
- `qvm rebuild` updates base, keeps overlay

**9p virtfs mounts:**
- Used for workspace and cache sharing
- `security_model=mapped-xattr` for security
- `msize=104857600` for performance (100MB transfer size)
- Mounts configured at VM start, no hotplug

## Contributing

Contributions welcome! This is a simple Bash-based tool designed to be readable and hackable.

**Key files:**
- `bin/qvm` - Main dispatcher
- `bin/qvm-*` - Subcommand implementations
- `lib/common.sh` - Shared utilities and paths
- `flake/default-vm/flake.nix` - Default VM template

**Development:**

```bash
git clone https://github.com/yourusername/qvm
cd qvm
nix develop
./bin/qvm start
```

## License

MIT