pyhdwallet/playbook.md

# pyhdwallet v1.1.0 (hdwalletpy)

**Deterministic BIP32/BIP39/BIP44 HD wallet generator** for Bitcoin, Ethereum, and other cryptocurrencies. Designed for **offline, air-gapped use** with complete multi-platform support.

## Features

- ✅ **BIP39** 24-word mnemonic generation with configurable entropy
- ✅ **BIP32** hierarchical deterministic wallet derivation  
- ✅ **BIP44** multi-account structure (Bitcoin, Ethereum, Litecoin, etc.)
- ✅ **Air-gapped operation** with vendored dependencies for 3 platforms
- ✅ **AES-256 encrypted ZIP** artifacts with password protection
- ✅ **PGP encryption** support for at-rest storage
- ✅ **Deterministic output** for reproducible wallet generation
- ✅ **Multi-platform offline support:**
  - macOS ARM64 (Apple Silicon)
  - Linux x86_64 (Intel/AMD servers)
  - Linux aarch64 (ARM64 servers, Raspberry Pi, Mac containers)
- ✅ **Regression test suite** with BIP test vectors
- ✅ **Checksum verification** for all vendored wheels

## Installation

### Prerequisites

- **Python 3.12** (required for all platforms)
- **macOS:** `brew install python@3.12`
- **Debian/Ubuntu:** `apt-get install python3.12 python3.12-venv`
- **Docker** (optional, for building Linux wheels on macOS)

### Quick Installation (macOS/Linux with Internet)

```bash
# Clone repository
git clone https://github.com/yourusername/hdwalletpy.git
cd hdwalletpy

# Run automated installer (auto-detects your platform)
./install_offline.sh
```

The installer automatically detects:

- **macOS ARM64** → uses `vendor/macos-arm64/`
- **Linux x86_64** → uses `vendor/linux-x86_64/`
- **Linux aarch64/arm64** → uses `vendor/linux-aarch64/`

### Air-Gapped Installation (No Internet)

The project includes **pre-built vendored wheels** for offline installation on all supported platforms.

#### On Internet-Connected Machine

```bash
# Clone and verify
git clone https://github.com/yourusername/hdwalletpy.git
cd hdwalletpy

# Verify checksums for your target platform
cd vendor/macos-arm64 && shasum -a 256 -c SHA256SUMS       # macOS ARM64
cd vendor/linux-x86_64 && sha256sum -c SHA256SUMS          # Linux x86_64
cd vendor/linux-aarch64 && sha256sum -c SHA256SUMS         # Linux ARM64

# Transfer entire repo to USB/CD
cp -r hdwalletpy /Volumes/USB/
```

#### On Air-Gapped Machine

```bash
# Ensure Python 3.12 is installed
python3.12 --version

# Navigate to repository
cd /path/to/hdwalletpy

# Run installer (auto-detects platform, creates venv, installs offline)
./install_offline.sh

# Activate environment
source .venv/bin/activate

# Generate wallet
python src/pyhdwallet.py gen --help
```

### Development Installation (with pytest)

For development with full test suite on air-gapped machines:

```bash
# Install with dev dependencies (includes pytest)
./install_offline.sh --dev
```

Platform-specific dev wheels are located in:

- `vendor/macos-arm64-dev/`
- `vendor/linux-x86_64-dev/`
- `vendor/linux-aarch64-dev/`

### Manual Installation (Without Script)

If you prefer manual control:

```bash
# Create venv
python3.12 -m venv .venv
source .venv/bin/activate

# Install from vendored wheels
# For macOS ARM64:
pip install --no-index --find-links=vendor/macos-arm64 -r requirements.txt

# For Linux x86_64:
pip install --no-index --find-links=vendor/linux-x86_64 -r requirements.txt

# For Linux aarch64:
pip install --no-index --find-links=vendor/linux-aarch64 -r requirements.txt

# Verify installation
python src/pyhdwallet.py test
```

### Dependencies (Top-Level Intent)

- **bip-utils** – BIP32/39/44 cryptographic operations
- **base58** – Bitcoin address encoding
- **cbor2** – CBOR serialization for Cardano
- **ecdsa** – Elliptic curve cryptography
- **pynacl** – NaCl cryptography library
- **pyzipper** – AES-256 encrypted ZIP archives
- **pytest** (dev only) – Test framework

## Quick Start

### 1) Generate (debug/test; prints mnemonic)

```bash
python src/pyhdwallet.py gen
```

**Output:** Mnemonic, master keys, and derived addresses to stdout.

### 2) Generate and save AES ZIP artifact to `.wallet/`

```bash
python src/pyhdwallet.py gen --file --zip
# Enter password when prompted
```

**Output:** Encrypted ZIP in `.wallet/wallet-YYYYMMDD-HHMMSS.zip`

### 3) Generate with PGP encryption and ZIP (recommended for "at-rest" storage)

```bash
python src/pyhdwallet.py gen --file --zip --pgp recipient@example.com
```

**Output:**  

- `.wallet/wallet-YYYYMMDD-HHMMSS.zip` (AES encrypted)
- `.wallet/wallet-YYYYMMDD-HHMMSS.zip.gpg` (PGP encrypted)

### 4) Recover (addresses) from mnemonic

```bash
python src/pyhdwallet.py recover
# (then paste mnemonic and press Ctrl-D)
```

### 5) Recover with interactive input + ZIP artifact

```bash
python src/pyhdwallet.py recover --file --zip
```

### 6) Run built-in smoke test

```bash
python src/pyhdwallet.py test
```

### 7) Run full regression test suite

```bash
# Requires dev installation
./install_offline.sh --dev
source .venv/bin/activate
pytest -v tests/test_vectors.py
```

## Testing

### Regression Test Suite

#### Test Coverage

- **BIP39 Mnemonic Generation:** Entropy, word list, checksum validation
- **BIP32 Key Derivation:** Master key, child key derivation paths
- **BIP44 Multi-Currency:** Bitcoin, Ethereum, Litecoin address generation
- **Known Test Vectors:** Official BIP39/BIP32 test cases

#### Running Tests

```bash
# Run all tests
pytest -v tests/test_vectors.py

# Run specific test
pytest -v tests/test_vectors.py::test_bip39_generation

# Run with detailed output
pytest -vv tests/test_vectors.py
```

#### Test Artifacts (Committed)

- `tests/vectors.json` – Official BIP test vectors
- `tests/test_vectors.py` – Regression test suite

#### When to Regenerate Test Vectors

Only if:

- Upgrading `bip-utils` library
- Adding new BIP implementations
- Changing derivation paths

## Vendored Dependencies

### What is Vendoring?

Pre-compiled Python wheels stored in `vendor/` for **offline installation** without PyPI access.

### Vendor Directory Structure

```Word
vendor/
├── macos-arm64/          # macOS Apple Silicon (runtime)
├── macos-arm64-dev/      # macOS dev wheels (pytest)
├── linux-x86_64/         # Linux Intel/AMD (runtime)
├── linux-x86_64-dev/     # Linux x86_64 dev wheels
├── linux-aarch64/        # Linux ARM64 (runtime)
├── linux-aarch64-dev/    # Linux ARM64 dev wheels
└── PROVENANCE.md         # Build information
```

### Building Vendor Wheels (Maintainers)

**Using Makefile (recommended):**

```bash
# Build runtime wheels for all platforms
make vendor-all

# Build dev wheels for all platforms (includes pytest)
make vendor-all-dev

# Build specific platform
make vendor-macos          # macOS ARM64
make vendor-macos-dev      # macOS dev
make vendor-linux          # Linux x86_64 via Docker
make vendor-linux-dev      # Linux x86_64 dev
make vendor-linux-arm      # Linux ARM64 via Docker
make vendor-linux-arm-dev  # Linux ARM64 dev

# Commit updated wheels
git add vendor/
git commit -m "vendor: update wheels to latest versions"
```

**Platform-specific notes:**

- **macOS wheels:** Must build on native macOS (uses `.venv312`)
- **Linux wheels:** Built via Docker (works on any platform)
- **ARM64 wheels:** Built with `--platform linux/arm64` in Docker

### Verifying Vendor Integrity

```bash
# Check checksums before using on air-gapped machine
cd vendor/macos-arm64 && shasum -a 256 -c SHA256SUMS
cd vendor/linux-x86_64 && sha256sum -c SHA256SUMS
cd vendor/linux-aarch64 && sha256sum -c SHA256SUMS
```

## Outputs and File Structure

### Stdout Behavior

When running `gen` without `--file`, output goes to **stdout** (terminal).  
**Security warning:** Use `--file` to avoid exposing mnemonic in shell history or logs.

### `--file` Behavior (Deterministic, Secured Output)

- Creates `.wallet/` directory
- Generates timestamped file: `wallet-YYYYMMDD-HHMMSS.txt`
- With `--zip`: Creates AES-256 encrypted ZIP
- With `--pgp`: Adds GPG encryption layer

### Password Handling for ZIP

- Interactive prompt (hidden input)
- Minimum 8 characters enforced
- Used for AES-256 encryption of ZIP archive

## Commands

### fetchkey (online)

```bash
python src/pyhdwallet.py fetchkey keyserver.ubuntu.com recipient@example.com
```

Downloads PGP public key for encryption. **Requires internet.**

### gen (offline)

```bash
python src/pyhdwallet.py gen [--file] [--zip] [--pgp EMAIL] [--force]
```

Generates new HD wallet. Fully offline after dependencies installed.

### recover (offline)

```bash
python src/pyhdwallet.py recover [--file] [--zip]
```

Recovers wallet from existing mnemonic. Reads from stdin.

### test (offline)

```bash
python src/pyhdwallet.py test
```

Runs built-in smoke test (quick validation).

## When to Use `--force`

```bash
# This redirects stdout to a file (non-TTY). The tool will refuse unless forced.
python src/pyhdwallet.py gen > output.txt  # ❌ Blocked by default

# Explicitly override the safety guard (dangerous).
python src/pyhdwallet.py gen --force > output.txt  # ✅ Allowed but discouraged
```

**Recommendation:** Use `--file` instead of shell redirection.

## Security Notes (Practical)

## Offline Security Best Practices

### Air-Gapped Setup (Highest Security)

An **air-gapped machine** has:

- ❌ No network interfaces (WiFi, Ethernet disabled/removed)
- ❌ No Bluetooth
- ❌ No USB devices except for initial setup
- ✅ Physically isolated environment

**Setup steps:**

1. **Build/verify on trusted internet machine:**

   ```bash
   git clone https://github.com/yourusername/hdwalletpy.git
   cd hdwalletpy
   
   # Verify checksums for target platform
   cd vendor/linux-x86_64 && sha256sum -c SHA256SUMS
   ```

2. **Transfer via USB/CD:**

   ```bash
   # Copy entire repo to USB
   cp -r hdwalletpy /media/USB/
   
   # On air-gapped machine, verify checksums again
   cd /media/USB/hdwalletpy/vendor/linux-x86_64
   sha256sum -c SHA256SUMS
   ```

3. **Install offline:**

   ```bash
   ./install_offline.sh
   source .venv/bin/activate
   ```

4. **Generate wallet:**

   ```bash
   python src/pyhdwallet.py gen --file --zip --pgp recipient@example.com
   ```

5. **Store encrypted artifacts** on separate USB (not the air-gapped machine)

### Threats Air-Gapping Mitigates

✅ **Network-based attacks:** Malware, keyloggers, remote exploitation  
✅ **Supply chain attacks:** Compromised PyPI packages (using vendored wheels)  
✅ **Clipboard hijacking:** If using `--file` mode

### Threats Air-Gapping Does NOT Fully Mitigate

⚠️ **Hardware keyloggers** (physical security required)  
⚠️ **Side-channel attacks** (electromagnetic, acoustic)  
⚠️ **Physical access** (disk imaging, cold boot attacks)  
⚠️ **Compromised build environment** (verify checksums, inspect code)

### Physical Security

- 🔒 **Boot from read-only media** (Live USB/CD)
- 🔒 **Full disk encryption** on air-gapped machine
- 🔒 **Store encrypted wallet artifacts separately** from air-gapped machine
- 🔒 **Destroy USB media** after wallet generation (or keep offline)

### Verification Through Testing

```bash
# On the air-gapped machine, before generating your wallet:
python src/pyhdwallet.py test
pytest -v tests/test_vectors.py  # If dev mode installed
```

### Entropy Sources

The tool uses Python's `secrets.token_bytes()` which relies on:

- **Linux:** `/dev/urandom` (kernel CSPRNG)
- **macOS:** `Security.framework` randomness
- **Windows:** `CryptGenRandom()`

**Additional entropy sources:**

- System uptime
- Process IDs
- Hardware interrupts

### PGP Key Fingerprint Pinning

```bash
# First, get and verify the fingerprint of your recipient key
gpg --fingerprint recipient@example.com

# Manually verify fingerprint matches what you expect (check via another channel)

# Then use it with pinning:
python src/pyhdwallet.py gen --file --zip --pgp recipient@example.com
```

### Operational Checklist

Before generating a production wallet:

- [ ] Verified Python 3.12 installation
- [ ] Verified vendored wheel checksums
- [ ] Tested smoke test (`python src/pyhdwallet.py test`)
- [ ] Tested recovery process with test mnemonic
- [ ] Disabled all network interfaces on air-gapped machine
- [ ] Prepared encrypted storage media for wallet artifacts
- [ ] Have PGP recipient key ready (if using `--pgp`)
- [ ] Have strong password ready for ZIP encryption

### Lower-Risk Scenarios

If full air-gapping is impractical:

- ✅ Use a **dedicated laptop** (factory reset, minimal software)
- ✅ Boot from **Live USB** (Tails, Ubuntu Live)
- ✅ Disconnect network **before** wallet generation
- ✅ Use **hardware wallets** for signing (this tool generates seeds only)

### Trust But Verify

- **Inspect the source code** before trusting
- **Build vendor wheels yourself** instead of using pre-built
- **Run test suite** to ensure cryptographic correctness
- **Compare generated addresses** with other tools (Ian Coleman's BIP39 tool)

## Troubleshooting

### "Permission denied" running `./src/pyhdwallet.py`

```bash
chmod +x src/pyhdwallet.py
# or
python3.12 src/pyhdwallet.py gen
```

### Python 3.12 not found

```bash
# macOS
brew install python@3.12

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install python3.12 python3.12-venv

# Verify
python3.12 --version
```

### "Missing dependency" errors

```bash
# Check your platform
uname -sm

# Install for macOS ARM64
pip install --no-index --find-links=vendor/macos-arm64 -r requirements.txt

# Install for Linux x86_64
pip install --no-index --find-links=vendor/linux-x86_64 -r requirements.txt

# Install for Linux aarch64
pip install --no-index --find-links=vendor/linux-aarch64 -r requirements.txt
```

### Checksum verification failures

```bash
# If failures occur, re-download from trusted source
git clone --depth=1 https://github.com/yourusername/hdwalletpy.git

# DO NOT use corrupted wheels for production wallets
```

### Unzipping AES ZIP issues

```bash
# Use pyzipper (installed with vendored deps)
python -c "import pyzipper; pyzipper.AESZipFile('wallet.zip').extractall(pwd=b'password')"
```

### Test failures after code changes

```bash
# Regenerate test vectors if you changed derivation logic
python src/pyhdwallet.py gen > tests/expected_output.txt

# Re-run tests
pytest -v tests/test_vectors.py
```

## Development

### Building Vendor Wheels

```bash
# Clean old wheels
make clean-vendor

# Build for all platforms
make vendor-all          # Runtime only
make vendor-all-dev      # Runtime + dev (pytest)

# Build specific platform
make vendor-macos
make vendor-linux
make vendor-linux-arm

# Verify and commit
git add vendor/
git commit -m "vendor: update wheels for v1.1.0"
```

### Running Tests (Dev)

```bash
# Quick test
python src/pyhdwallet.py test

# Full regression suite
pytest -v tests/test_vectors.py

# Specific test
pytest -v tests/test_vectors.py::test_bip39_generation

# With coverage
pytest --cov=src tests/
```

### Docker Development Environment

```bash
# Build image
make build-image

# Start warm container
make up

# Inside container
make shell
python src/pyhdwallet.py test
```

## Changelog

### v1.1.0 (2026-01-12)

- ✨ Added multi-platform vendor support (macOS ARM64, Linux x86_64, Linux aarch64)
- ✨ Added dev wheel packages for all platforms (includes pytest)
- ✨ Added automatic platform detection in `install_offline.sh`
- ✨ Added checksum verification for all vendor wheels
- ✨ Updated Makefile with `vendor-all` and `vendor-all-dev` targets
- 📚 Updated documentation for air-gapped multi-platform setup

### v1.0.5 (Previous)

- Initial vendoring support for macOS and Linux x86_64
- BIP32/BIP39/BIP44 implementation
- PGP encryption support
- AES-256 ZIP encryption