---
name: zfs-maintenance
description: Use this skill for ZFS/OpenZFS administration, design, troubleshooting, recovery, backups, replication, monitoring, performance, encryption, disk replacement, and pool imports. Use ZFSBEST.md as the full local reference.
metadata: {"version": "1.0", "skill-author": "Grzegorz Surmann", "author-email": "zfs@funil.de"}
---

# ZFS Maintenance

This skill gives compact operational guidance for ZFS/OpenZFS work. For full
details, examples, and caveats, consult [ZFSBEST.md](ZFSBEST.md).

## Safety Rules

- Treat ZFS commands as potentially destructive.
- Do not invent pool names, dataset names, disk names, GUIDs, keys, host IDs,
  or device paths.
- Prefer inspection commands before commands that change pool or dataset state.
- Use stable device paths such as `/dev/disk/by-id/...` where available.
- Do not clear errors just to hide them. Capture status, events, history, and
  SMART data first.
- Do not run filesystem repair tools such as `fsck` on ZFS pools.
- If disks may be physically failing, clone suspect disks before heavy reads,
  scrubs, resilvers, repeated imports, or recovery attempts.
- If a pool imports read-only, copy or replicate important data out before
  deeper repair work.
- Treat platform details as variable. Check operating system or appliance
  documentation for bootloader support, ACL behavior, service names, mount
  behavior, compatibility profiles, and encryption key loading.

## First Checks

Start with read-only facts:

```bash
zpool status -x
zpool status -gLPv pool0
zpool list -v pool0
zfs list -r pool0
zfs list -t snapshot -r pool0
zpool events -v
zpool history -l pool0
lsblk -o NAME,SIZE,MODEL,SERIAL,TYPE
smartctl -a /dev/disk/by-id/disk0
```

Read `zpool status` from the top down:

- `state`: current pool health.
- `status`: what ZFS believes is wrong.
- `action`: the safest next action ZFS suggests.
- `errors`: permanent data errors or named damaged files.
- `READ`, `WRITE`, `CKSUM`: per-device error counters.

## Planning Defaults

Use conservative defaults unless the workload clearly needs something else:

- Direct disk access; avoid hardware RAID under ZFS.
- Burn in and inventory new or reused disks before creating important pools.
- `ashift=12` for modern HDDs and SSDs unless proven otherwise.
- Mirrors for VM, database, container, and random-I/O pools.
- RAIDZ2 or RAIDZ3 for large archive and backup pools.
- Avoid RAIDZ1 for important new pools with large disks.
- `compression=zstd` by default on current OpenZFS.
- `atime=off` for most server datasets.
- Linux ACL-heavy datasets: `acltype=posixacl` and `xattr=sa`, after checking
  compatibility needs.
- Separate datasets for separate snapshot, quota, recordsize, mountpoint,
  sharing, and backup policies.
- Set `recordsize` or `volblocksize` before writing data.
- Keep pools below about 80% used.
- Avoid dedup unless the dedup ratio, DDT size, RAM, and recovery plan are all
  acceptable.
- Document pool layout, disk serials, feature compatibility, encryption
  recovery, rescue media, and backup paths after pool creation.

## High-Risk Commands

Require explicit intent, backups or recovery options, and a short pre-flight
check before recommending:

- `zpool create`
- `zpool destroy`
- `zfs destroy`
- `zpool add`
- `zpool remove`
- `zpool labelclear`
- `zpool upgrade`
- `zfs receive -F`
- rollback
- checkpoint rewind
- `zpool import -F`
- `zpool import -X`
- `zdb -F`
- `zdb -X`

Specific checks:

- `zpool create`: verify device identity, existing pools, mount state, backups,
  and intended vdev layout.
- `zpool add`: verify the command adds a complete intended top-level vdev. Use
  `zpool attach` when adding redundancy to an existing disk.
- `zfs destroy`: confirm dataset or snapshot names, backups, and service state.
- rollback: inspect with `zfs diff` or a clone when possible and confirm newer
  changes can be lost.
- `zfs receive -F`: confirm the destination can be rolled back or overwritten.
- `zpool upgrade`: confirm rescue media, bootloader, older hosts, appliances,
  and compatibility requirements.
- `zpool import -F`, `zpool import -X`, `zdb -F`, and `zdb -X`: prefer dry
  runs, cloned media, read-only no-mount imports, and backup restore paths
  first.

## Recovery Triage

During incidents:

1. Stop writes if corruption, ransomware, accidental deletion, or failing
   hardware is suspected.
2. Capture the minimum recovery bundle:

   ```bash
   zpool status -gLPv pool0
   zpool list -v pool0
   zpool import -d /dev/disk/by-id
   zpool events -v
   zpool history -l pool0
   zfs list -r pool0
   zfs list -t snapshot -r pool0
   lsblk -o NAME,SIZE,MODEL,SERIAL,TYPE
   smartctl -a /dev/disk/by-id/disk0
   ```

3. Prefer read-only and no-mount import during investigation:

   ```bash
   zpool import -d /dev/disk/by-id -N -o readonly=on pool0
   ```

4. Clone failing disks with `ddrescue` before aggressive recovery attempts.
5. Restore files from snapshots or backups before rolling back whole datasets.
6. Use `zpool import -F -n` as a dry run before any rewind import.
7. Treat `-X`, `zdb -F`, and `zdb -X` as last-resort expert recovery paths.

Relevant `ZFSBEST.md` sections:

- `Disaster Recovery`
- `Recovery Decision Tree`
- `Minimum Safe Recovery Bundle`
- `Failed Pool Recovery Triage`
- `Clone Failing Disks Before Recovery Attempts`
- `Physical Pool Inspection With zdb`
- `Pool Rewind Recovery With -F`
- `Handling Permanent Data Errors`
- `Ransomware Or Mass Deletion Recovery`

## Disk Replacement

Replace disks based on evidence, not guesswork.

Strong replacement signals:

- SMART shows current pending sectors, offline uncorrectable sectors, or new
  reallocated sectors.
- `zpool status` shows repeated or increasing `READ`, `WRITE`, or `CKSUM`
  errors for the same device.
- The disk disappears, reappears, faults, or becomes `UNAVAIL`.
- Long SMART tests fail.
- Resilver or scrub repeatedly stalls on the same device.
- Cabling, power, slots, and controller paths have been checked or swapped.

Replace one disk at a time when possible, let resilver finish, then scrub.

## Replication Safety

Treat replication as incomplete until the destination is inspected and restore
testing has succeeded.

- Receive with `-u` so datasets do not mount immediately with unexpected
  preserved mountpoints.
- Keep the source snapshot until the destination has received and verified the
  matching snapshot.
- Save source and destination snapshot names in backup logs.
- Monitor `receive_resume_token` for interrupted receives.
- Use raw sends for encrypted datasets when the receiver should not see
  plaintext.
- Avoid `zfs receive -F` unless destination rollback is intentional.
- Do not prune the only common snapshot needed for the next incremental send.

## Property Reminders

Use the property glossary in `ZFSBEST.md` for details. Common rules:

- `ashift` is set at vdev creation and cannot be changed for that vdev.
- `recordsize` and `volblocksize` should be set before writing data.
- `compression=zstd` is usually safe and useful.
- `atime=off` reduces metadata writes.
- `quota`, `reservation`, `refquota`, and `refreservation` affect visible free
  space.
- `sync=disabled` can lose acknowledged writes; avoid it unless loss is
  acceptable.
- `special_small_blocks` needs redundant special vdevs and conservative
  thresholds.
- Encryption keys or passphrases must be recoverable offline.
- `copies` is not a replacement for pool redundancy or backups.