pyhdwallet/playbook.md

# pyhdwallet v1.0.5 (hdwalletpy)

A command-line tool for generating and recovering HD wallets (BIP39) with support for Ethereum, Solana, and Bitcoin. It is designed for offline operation, optional PGP encryption, and writing deterministic AES-encrypted ZIP "wallet artifacts" into a local `.wallet/` folder.

Repository structure (current):

- `src/pyhdwallet.py` — main CLI script
- `tests/` — offline regression test suite
  - `test_vectors.py` — pytest suite for derivation integrity
  - `bootstrap_vectors.py` — one-time script to generate test vectors
  - `vectors.json` — frozen expected values (committed)
  - `data/recipient.asc` — test PGP key (committed)
- `.wallet/` — generated wallet artifacts (should be gitignored)
- `requirements.in`, `requirements.txt`
- `README.md`, `playbook.md`, `LICENSE`

**Note**: This tool requires a subcommand (e.g., `gen`, `recover`). Running without one displays help. Use `<subcommand> -h` for detailed options.

---

## Features

- **Offline-first**: `gen`, `recover`, and `test` block network access (best-effort in-process guard).
- **Multi-chain support**: Derives addresses for Ethereum, Solana, and Bitcoin.
- **PGP encryption**: Encrypts a JSON payload to a PGP public key and outputs an ASCII-armored PGP message (typically saved as `.asc`).
- **AES-encrypted ZIP artifacts**: When `--file` is used, output is written as an AES-encrypted ZIP via `pyzipper`.
- **TTY safety guard + --force**: If stdout is piped/redirected (non-TTY), the tool refuses to print sensitive data unless `--force` is explicitly set (to avoid accidental leaks into logs/files). `isatty()` is the classic way to detect whether stdout is connected to a terminal.
- **Off-screen mode**: `--off-screen` suppresses printing sensitive data to stdout.
- **Regression test suite**: Offline tests with frozen vectors ensure derivation logic, PGP fingerprinting, and seed generation remain stable across code changes.

---

## Installation

### Prerequisites

- Python 3.11+
- Recommended: a virtual environment

### Setup

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
```

### Dependencies (top-level intent)

- `bip-utils` — BIP39 + BIP derivation logic
- `PGPy` — encryption to OpenPGP public keys
- `pynacl` + `base58` — Solana seed/key handling
- `pyzipper` — AES-encrypted ZIP writing (only needed when using `--file`)
- `pytest` — test framework (development only)

---

## Quick Start

### 1) Generate (debug/test; prints mnemonic)

```bash
python ./src/pyhdwallet.py gen
```

### 2) Generate and save AES ZIP artifact to `.wallet/`

```bash
python ./src/pyhdwallet.py gen --file
```

### 3) Generate with PGP encryption and ZIP (recommended for "at-rest" storage)

```bash
python ./src/pyhdwallet.py gen --pgp-pubkey-file pubkeys/mykey.asc --file
```

### 4) Recover (addresses) from mnemonic

```bash
python ./src/pyhdwallet.py recover --mnemonic-stdin
# (then paste mnemonic and press Ctrl-D)
```

### 5) Recover with interactive input + ZIP artifact

```bash
python ./src/pyhdwallet.py recover --interactive --file
```

### 6) Run built-in smoke test

```bash
python ./src/pyhdwallet.py test
```

### 7) Run full regression test suite

```bash
pytest -v tests/test_vectors.py
```

---

## Testing

### Regression Test Suite

The project includes a comprehensive offline test suite that validates critical functionality without requiring network access. All tests use committed test vectors to ensure deterministic, repeatable results.

#### Test Coverage

- **PGP fingerprint calculation**: Verifies that PGP key fingerprinting logic produces expected results and enforces fingerprint matching
- **BIP39 seed derivation**: Ensures mnemonics (with and without passphrases) always produce the same seed hex
- **Multi-chain address derivation**: Validates Ethereum, Bitcoin (3 address types), and Solana (3 profiles) derivation paths remain stable
- **CLI integration**: Smoke tests for `recover` command with correct and incorrect fingerprints

#### Running Tests

```bash
# Run all tests
pytest -v tests/test_vectors.py

# Run specific test
pytest -v tests/test_vectors.py::test_address_derivation_integrity

# Run with detailed output
pytest -vv tests/test_vectors.py
```

#### Test Artifacts (Committed)

- `tests/vectors.json` — Frozen expected values for all test cases
- `tests/data/recipient.asc` — Test PGP public key (safe to commit; public key only)
- `tests/bootstrap_vectors.py` — One-time script to regenerate vectors (run only when intentionally updating expected behavior)

#### When to Regenerate Test Vectors

Only regenerate `tests/vectors.json` when you **intentionally** change:

- Derivation paths (e.g., switching from `m/44'/60'/0'/0/i` to a different path)
- Solana profile logic
- BIP39 seed generation

To regenerate:

```bash
rm tests/vectors.json
python3 tests/bootstrap_vectors.py
pytest -v tests/test_vectors.py  # Verify all pass
git add tests/vectors.json
git commit -m "test: update vectors after intentional derivation change"
```

**Warning**: If tests fail after a code change and you didn't intend to change behavior, **do not regenerate vectors**. Fix the code regression instead.

---

## Outputs and file structure

### Stdout behavior

- `gen` prints the mnemonic + derived addresses by default (intended for debug/test comparisons).
- `--off-screen` suppresses printing sensitive data to stdout.

### `--file` behavior (deterministic, secured output)

If `--file` is present, the tool writes **only** an AES-encrypted ZIP file (no raw `.json`/`.asc` file is left on disk). AES ZIP is implemented using `pyzipper`.

Default output folder:

- `./.wallet/`

Override folder:

- `--wallet-location /path/to/folder`

Naming uses UTC timestamps (e.g. `20260106_161830Z`):

- No PGP: zip contains `test_wallet_<UTC>.json`, zip name `test_wallet_<UTC>.zip`
- With PGP: zip contains `encrypted_wallet_<UTC>.asc`, zip name `encrypted_wallet_<UTC>.zip`

### Password handling for ZIP

- Default: `--zip-password-mode prompt` prompts for the ZIP password (attempts hidden entry).
- If hidden entry is not supported in the environment, it falls back to visible `input()` with loud warnings.
- Optional: `--zip-password-mode auto` generates a Base58 password (length controlled by `--zip-password-len`).
- If auto mode is used, password is shown **only if** `--show-generated-password` is set, and it prints to **stderr** (not stdout) to reduce accidental capture when stdout is redirected.

`pyzipper` supports AES encryption via `AESZipFile` and password-setting APIs.

---

## Commands

### fetchkey (online)

Download and verify a PGP public key from a URL.

```bash
python ./src/pyhdwallet.py fetchkey <url> [--out FILE] [--timeout SECONDS] [--off-screen] [--expected-fingerprint FINGERPRINT]
```

Options:

- `url`: URL to the ASCII-armored PGP public key
- `--out FILE`: save key to file
- `--timeout SECONDS`: request timeout (default 15)
- `--off-screen`: reduced output / temp-file behavior
- `--expected-fingerprint`: refuse if downloaded key fingerprint doesn't match (40 hex chars)

---

### gen (offline)

Generate a BIP39 mnemonic and derive addresses.

```bash
python ./src/pyhdwallet.py gen [options]
```

Core options:

- `--words {12,15,18,21,24}`
- `--dice-rolls "1 2 3 ..."` (space-separated; adds user entropy)
- `--passphrase` (prompts for BIP39 passphrase)
- `--passphrase-hint HINT`
- `--chains ethereum solana bitcoin`
- `--addresses N`
- `--sol-profile {phantom_bip44change,phantom_bip44,phantom_deprecated,solana_bip39_first32}`

PGP options:

- `--pgp-pubkey-file FILE` (encrypt payload to pubkey; `.asc` content)
- `--pgp-ignore-usage-flags`
- `--expected-fingerprint FINGERPRINT` (refuse if PGP key fingerprint doesn't match)

Artifact options:

- `--file` (write AES ZIP only)
- `--wallet-location PATH` (default: `./.wallet`)
- `--zip-password-mode {prompt,auto}`
- `--zip-password-len N` (default: 12)
- `--show-generated-password` (auto mode only; prints password to stderr)

Safety options:

- `--off-screen` (don't print sensitive output)
- `--force` (only for non-TTY stdout; see below)

---

### recover (offline)

Recover addresses from mnemonic and optionally write a ZIP artifact.

```bash
python ./src/pyhdwallet.py recover [options]
```

Input options (choose one):

- `--mnemonic-stdin` (recommended; read from stdin to avoid shell history)
- `--interactive` (word-by-word guided entry with validation)

Additional options:

- same chain/PGP/ZIP/off-screen/force options as `gen`
- `--export-private` (requires `--pgp-pubkey-file`; includes private keys in encrypted payload)
- `--include-source` (includes mnemonic in payload; requires `--pgp-pubkey-file`)

Notes:

- `--export-private` requires `--pgp-pubkey-file` (private keys only travel inside encrypted payload).
- `--include-source` controls whether mnemonic is included inside the encrypted payload (only meaningful when PGP is used).

---

### test (offline)

Run minimal self-tests.

```bash
python ./src/pyhdwallet.py test
```

For comprehensive regression testing, use the pytest suite:

```bash
pytest -v tests/test_vectors.py
```

---

## When to use `--force`

Use `--force` only if you **intentionally** want to print sensitive output while stdout is being piped/redirected.

Why it exists:

- When you pipe/redirect output (e.g., `>` or `|`), stdout is typically **not a TTY**.
- The program checks `sys.stdout.isatty()` and refuses to print secrets by default to avoid accidental leakage into files/logs.
- `isatty()` is commonly used to detect terminal vs pipe/redirect.

Examples:

```bash
# This redirects stdout to a file (non-TTY). The tool will refuse unless forced.
python ./src/pyhdwallet.py gen > out.txt

# Explicitly override the safety guard (dangerous).
python ./src/pyhdwallet.py gen --force > out.txt
```

If running normally in your interactive terminal, stdout is a TTY and `--force` does nothing (output looks the same).

---

## Security notes (practical)

- `gen` printing the mnemonic is intentionally "debug/test" behavior. Assume stdout can be recorded (scrollback, logging, screen recording, CI logs).
- Prefer `--off-screen` for reduced exposure.
- Prefer `--file` so artifacts go into `.wallet/` and are AES-encrypted via `pyzipper`.
- For stronger at-rest security: combine `--pgp-pubkey-file` + `--file` so the ZIP contains only an encrypted `.asc` payload. PGPy encryption style follows `PGPMessage.new(...)` then `pubkey.encrypt(...)`.
- Use `--expected-fingerprint` to enforce PGP key pinning and prevent key substitution attacks.

---

## Troubleshooting

### "Permission denied" running `./src/pyhdwallet.py`

Run via python, or set executable bit:

```bash
python ./src/pyhdwallet.py gen
# or
chmod +x ./src/pyhdwallet.py
./src/pyhdwallet.py gen
```

### "Missing dependency: pyzipper" but pip says installed

You likely installed into a different venv. Use:

```bash
python -m pip install pyzipper
python -c "import pyzipper; print(pyzipper.__version__)"
```

### "Missing dependency: pytest" when running tests

Install pytest in your virtualenv:

```bash
python -m pip install pytest
```

### Unzipping AES ZIP issues

Some `unzip` tools may not support AES-encrypted ZIPs. Use Python + pyzipper to extract if needed:

```bash
python - <<'PY'
import pyzipper
zip_path = ".wallet/your_wallet.zip"
out_dir = "unzip_out"
pw = input("ZIP password: ").encode()
with pyzipper.AESZipFile(zip_path) as zf:
    zf.pwd = pw
    zf.extractall(out_dir)
print("Extracted to", out_dir)
PY
```

AES ZIP support is the reason `pyzipper` is used.

### Test failures after code changes

If `pytest -v tests/test_vectors.py` fails after you modified code:

1. **Did you intentionally change derivation logic?**
   - Yes → Regenerate vectors: `rm tests/vectors.json && python3 tests/bootstrap_vectors.py`
   - No → You have a regression bug; revert your changes and fix the code

2. **Is only one test failing?**
   - Run with `-vv` for detailed diff: `pytest -vv tests/test_vectors.py::test_name`

---

## Changelog

- **v1.0.5**
  - `--unsafe-print` removed; `gen` prints mnemonic by default (debug/test behavior).
  - `--output` removed; file payload is always JSON (unencrypted) or `.asc` (PGP encrypted).
  - `--file` is now a boolean flag that writes **only** AES-encrypted ZIP artifacts (no raw output files).
  - Added: `--wallet-location`, `--zip-password-mode`, `--zip-password-len`, `--show-generated-password`.
  - Added: `--force` to override the non-TTY printing safety guard (use only when redirecting/piping).
  - Added: `--expected-fingerprint` for PGP key pinning.
  - Added: Offline regression test suite with frozen vectors (`tests/test_vectors.py`).
  - Changed: `recover` now uses `--mnemonic-stdin` and `--interactive` instead of `--mnemonic` CLI arg.
- **v1.0.3** Documentation and CLI behavior updates.
- **v1.0.2** Added off-screen behaviors and security improvements.
- **v1.0.1** Renamed to pyhdwallet and added version flag.
- **v1.0.0** Initial release.

Key additions:

1. **Testing section** with full coverage explanation
2. **Repository structure** updated to include `tests/` folder
3. **Test artifacts** documentation
4. **When to regenerate vectors** guidance
5. **Troubleshooting** section for test failures
6. **Changelog** updated with v1.0.5 test suite addition
7. **Quick Start** updated with correct `recover` command syntax (`--mnemonic-stdin`)
8. **Dependencies** updated to include pytest

[1](https://ppl-ai-file-upload.s3.amazonaws.com/web/direct-files/attachments/50321846/e5caab46-cb40-4f76-8ea9-60ed65bf4927/pyhdwallet.py)
[2](https://ppl-ai-file-upload.s3.amazonaws.com/web/direct-files/attachments/images/50321846/085d5710-8637-45d5-8686-77c98df9dcdf/image.jpg)