ZFSのHDD交換日記

自宅サーバーで組んでるNASのHDDの交換日記

2022-02-28

  • 状況確認

.-12:44 asonas@archlinux:~
`-% zpool status
pool: bigdata
state: ONLINE
scan: scrub repaired 0B in 05:33:51 with 0 errors on Sat Feb 26 01:03:53 2022
config:

NAME STATE READ WRITE CKSUM
bigdata ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

.-12:45 asonas@archlinux:~
`-% sudo smartctl -a /dev/sda
smartctl 7.2 2020-12-30 r5155 x86_64-linux-5.15.4-arch1-1 (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Western Digital Green
Device Model: WDC WD30EZRX-00D8PB0
Serial Number: WD-WMC4N1136837
LU WWN Device Id: 5 0014ee 2097ce288
Firmware Version: 80.00A80
User Capacity: 3,000,592,982,016 bytes 3.00 TB
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Device is: In smartctl database for details use: -P show
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Feb 28 12:45:32 2022 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (41040) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 412) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x7035) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 70
3 Spin_Up_Time 0x0027 188 177 021 Pre-fail Always - 5600
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 96
5 Reallocated_Sector_Ct 0x0033 186 186 140 Pre-fail Always - 430
7 Seek_Error_Rate 0x002e 200 187 000 Old_age Always - 0
9 Power_On_Hours 0x0032 030 030 000 Old_age Always - 51621
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 95
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 71
193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 4506328
194 Temperature_Celsius 0x0022 116 101 000 Old_age Always - 34
196 Reallocated_Event_Count 0x0032 166 001 000 Old_age Always - 34
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 2
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 2
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 17
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 5

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error

1 Short offline Completed without error 00% 48018 -

2 Extended offline Aborted by host 90% 48018 -

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

.-12:45 asonas@archlinux:~
`-% sudo smartctl -a /dev/sdb
sudo asonas のパスワード:
smartctl 7.2 2020-12-30 r5155 x86_64-linux-5.15.4-arch1-1 (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Western Digital Green
Device Model: WDC WD30EZRX-00D8PB0
Serial Number: WD-WMC4N1136787
LU WWN Device Id: 5 0014ee 2b427d907
Firmware Version: 80.00A80
User Capacity: 3,000,592,982,016 bytes 3.00 TB
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Device is: In smartctl database for details use: -P show
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Feb 28 12:50:44 2022 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (39540) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 397) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x7035) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 190 180 021 Pre-fail Always - 5466
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 95
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 028 028 000 Old_age Always - 52813
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 94
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 70
193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 4378590
194 Temperature_Celsius 0x0022 109 097 000 Old_age Always - 41
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged. To run self-tests, use: smartctl -t

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

smartctl 7.2 2020-12-30 r5155 x86_64-linux-5.15.4-arch1-1 (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Western Digital Red
Device Model: WDC WD30EFRX-68EUZN0
Serial Number: WD-WCC4N5RPHLYU
LU WWN Device Id: 5 0014ee 26695e61e
Firmware Version: 82.00A82
User Capacity: 3,000,592,982,016 bytes 3.00 TB
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Device is: In smartctl database for details use: -P show
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Feb 28 12:51:41 2022 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (39540) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 397) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x703d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 189 181 021 Pre-fail Always - 5550
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 46
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 095 095 000 Old_age Always - 4024
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 46
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 29
193 Load_Cycle_Count 0x0032 199 199 000 Old_age Always - 3664
194 Temperature_Celsius 0x0022 109 097 000 Old_age Always - 41
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged. To run self-tests, use: smartctl -t

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

sdaとsdbが5万時間超え、sdbは3000時間
RAIDの用途としては録画サーバーのストレージなので、交換前に録画の予約をストップさせて置くのがよさそう。
物理世界側の手順として

  • 最低限の接続にして仕事机の足元にサーバーを持ってくる
    • 今は仕事机の右奥に鎮座させている
    • replaceの時間を読めないので最悪子供が保育園からかえってくる、朝起きてくるタイミングでもパッと見つけられないようにする
    • 電源スイッチの挙動を変えられたら変える
      • 多分、今は起動中に電源を押すとリセットされてしまう
        • 子はスイッチの類を何でも押したがる
      • 足元に置く予定ので注意が必要
        ソフトウェア側は
  • epgstationの録画のルールをすべてオフにする
    • replace中に読み書き発生するとだるいかなぁとか思った
  • ぐらい..?
    • zfs scrubを毎週やってるけどまぁ被ることはなかろう
    • たぶんいい感じにスキップしてくれるやろ

2022-03-15

`-% zpool status
pool: bigdata
state: ONLINE
scan: scrub repaired 0B in 05:47:25 with 0 errors on Sat Mar 12 01:17:27 2022
config:

NAME STATE READ WRITE CKSUM
bigdata ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

`-% sudo dmesg | grep WDC
4.099915 ata3.00: ATA-9: WDC WD30EZRX-00D8PB0, 80.00A80, max UDMA/133
4.102262 ata4.00: ATA-9: WDC WD30EZRX-00D8PB0, 80.00A80, max UDMA/133
4.102297 ata6.00: ATA-9: WDC WD30EFRX-68EUZN0, 82.00A82, max UDMA/133
4.102422 ata5.00: ATA-11: WDC WDS500G2B0A-00SM50, 415020WD, max UDMA/133
4.109010 scsi 2:0:0:0: Direct-Access ATA WDC WD30EZRX-00D 0A80 PQ: 0 ANSI: 5
4.110380 scsi 3:0:0:0: Direct-Access ATA WDC WD30EZRX-00D 0A80 PQ: 0 ANSI: 5
4.111027 scsi 4:0:0:0: Direct-Access ATA WDC WDS500G2B0A 20WD PQ: 0 ANSI: 5
4.111722 scsi 5:0:0:0: Direct-Access ATA WDC WD30EFRX-68E 0A82 PQ: 0 ANSI: 5

`-% ll /dev/disk/by-id/
合計 0
lrwxrwxrwx 1 root root 9 3月 15 16:23 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU -> ../../sdd
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU-part1 -> ../../sdd1
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU-part9 -> ../../sdd9
lrwxrwxrwx 1 root root 9 3月 15 16:23 ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 -> ../../sdb
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787-part9 -> ../../sdb9
lrwxrwxrwx 1 root root 9 3月 15 16:23 ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 -> ../../sda
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837-part9 -> ../../sda9
lrwxrwxrwx 1 root root 9 3月 15 16:23 ata-WDC_WDS500G2B0A-00SM50_21183T800611 -> ../../sdc
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WDS500G2B0A-00SM50_21183T800611-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 3月 15 16:23 ata-WDC_WDS500G2B0A-00SM50_21183T800611-part2 -> ../../sdc2
lrwxrwxrwx 1 root root 13 3月 15 16:23 nvme-WDC_WDS250G2B0C-00PXH0_2025C8801175 -> ../../nvme0n1
lrwxrwxrwx 1 root root 15 3月 15 16:23 nvme-WDC_WDS250G2B0C-00PXH0_2025C8801175-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 15 3月 15 16:23 nvme-WDC_WDS250G2B0C-00PXH0_2025C8801175-part2 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 13 3月 15 16:23 nvme-eui.e8238fa6bf530001001b448b49662749 -> ../../nvme0n1
lrwxrwxrwx 1 root root 15 3月 15 16:23 nvme-eui.e8238fa6bf530001001b448b49662749-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 15 3月 15 16:23 nvme-eui.e8238fa6bf530001001b448b49662749-part2 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 9 3月 15 16:23 wwn-0x50014ee2097ce288 -> ../../sda
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x50014ee2097ce288-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x50014ee2097ce288-part9 -> ../../sda9
lrwxrwxrwx 1 root root 9 3月 15 16:23 wwn-0x50014ee26695e61e -> ../../sdd
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x50014ee26695e61e-part1 -> ../../sdd1
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x50014ee26695e61e-part9 -> ../../sdd9
lrwxrwxrwx 1 root root 9 3月 15 16:23 wwn-0x50014ee2b427d907 -> ../../sdb
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x50014ee2b427d907-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x50014ee2b427d907-part9 -> ../../sdb9
lrwxrwxrwx 1 root root 9 3月 15 16:23 wwn-0x5001b448ba554b42 -> ../../sdc
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x5001b448ba554b42-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 3月 15 16:23 wwn-0x5001b448ba554b42-part2 -> ../../sdc2

  • WMC4N1136837
    • sda
  • WMC4N1136787
    • sdb
  • WCC4N5RPHLYU
    • sdd
  • sda -> sdb -> sddの順で交換していく
    • コマンド
    • sda
      • zpool offline bigdata ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837
    • sdb
      • zpool offline bigdata ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787
    • sdd
      • zpool offline bigdata ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU
  • 全然関係ないけどraidつくったときのコマンド
    • zpool create -f -o ashift=12 -m /mnt/data bigdata raidz ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU

-% zpool offline bigdata ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 cannot offline ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837: permission denied .-[16:41](weblog-authoring-wiki://24) asonas@archlinux:~/ghq/github.com/l3tnun/EPGStation (master) -% sudo zpool offline bigdata ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837
sudo asonas のパスワード:
.-16:41 asonas@archlinux:~/ghq/github.com/l3tnun/EPGStation (master)
`-% zpool status
pool: bigdata
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: scrub repaired 0B in 05:47:25 with 0 errors on Sat Mar 12 01:17:27 2022
config:

NAME STATE READ WRITE CKSUM
bigdata DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 OFFLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errorsy

あとで写真をはる

.-17:04 asonas@archlinux:~
`-% ll /dev/sd*
brw-rw—- 1 root disk 8, 0 3月 15 16:57 /dev/sda
brw-rw—- 1 root disk 8, 16 3月 15 16:57 /dev/sdb
brw-rw—- 1 root disk 8, 17 3月 15 16:57 /dev/sdb1
brw-rw—- 1 root disk 8, 25 3月 15 16:57 /dev/sdb9
brw-rw—- 1 root disk 8, 32 3月 15 16:57 /dev/sdc
brw-rw—- 1 root disk 8, 33 3月 15 16:57 /dev/sdc1
brw-rw—- 1 root disk 8, 34 3月 15 16:57 /dev/sdc2
brw-rw—- 1 root disk 8, 48 3月 15 16:57 /dev/sdd
brw-rw—- 1 root disk 8, 49 3月 15 16:57 /dev/sdd1
brw-rw—- 1 root disk 8, 57 3月 15 16:57 /dev/sdd9

`-% zpool status
pool: bigdata
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: scrub repaired 0B in 05:47:25 with 0 errors on Sat Mar 12 01:17:27 2022
config:

NAME STATE READ WRITE CKSUM
bigdata DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 OFFLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

Command (? for help): p
Disk /dev/sda: 15628053168 sectors, 7.3 TiB
Model: WDC WD80EAZZ-00B
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 217ED5B8-9D92-409B-A7A8-183AC2C2C18A
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 15628053134
Partitions will be aligned on 2048-sector boundaries
Total free space is 15628053101 sectors (7.3 TiB)

Number Start (sector) End (sector) Size Code Name

Command (? for help): o
This option deletes all partitions and creates a new protective MBR.
Proceed? (Y/N): Y

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-15628053134, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-15628053134, default = 15628053134) or {+-}size{KMGTP}:
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.

.-17:10 asonas@archlinux:~
`-% ll /dev/sd*
brw-rw—- 1 root disk 8, 0 3月 15 17:09 /dev/sda
brw-rw—- 1 root disk 8, 1 3月 15 17:09 /dev/sda1
brw-rw—- 1 root disk 8, 16 3月 15 16:57 /dev/sdb
brw-rw—- 1 root disk 8, 17 3月 15 16:57 /dev/sdb1
brw-rw—- 1 root disk 8, 25 3月 15 16:57 /dev/sdb9
brw-rw—- 1 root disk 8, 32 3月 15 16:57 /dev/sdc
brw-rw—- 1 root disk 8, 33 3月 15 16:57 /dev/sdc1
brw-rw—- 1 root disk 8, 34 3月 15 16:57 /dev/sdc2
brw-rw—- 1 root disk 8, 48 3月 15 16:57 /dev/sdd
brw-rw—- 1 root disk 8, 49 3月 15 16:57 /dev/sdd1
brw-rw—- 1 root disk 8, 57 3月 15 16:57 /dev/sdd9

`-% sudo zpool replace bigdata ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK

`-% zpool status
pool: bigdata
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Tue Mar 15 17:13:18 2022
1.73T scanned at 77.2G/s, 1.77M issued at 79.0K/s, 6.00T total
0B resilvered, 0.00% done, no estimated completion time
config:

NAME STATE READ WRITE CKSUM
bigdata DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
replacing-0 DEGRADED 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 OFFLINE 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

しばらくまつ

Every 1.0s: zpool status archlinux: Tue Mar 15 22:13:57 2022

pool: bigdata
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Tue Mar 15 17:13:18 2022
6.00T scanned at 349M/s, 5.31T issued at 309M/s, 6.00T total
1.77T resilvered, 88.44% done, 00:39:17 to go
config:

NAME STATE READ WRITE CKSUM
bigdata DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
replacing-0 DEGRADED 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136837 OFFLINE 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ONLINE 0 0 0 (resilvering)
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

もうちょっと

.-23:13 asonas@archlinux:~
`-% zpool status
pool: bigdata
state: ONLINE
scan: resilvered 2.00T in 05:52:18 with 0 errors on Tue Mar 15 23:05:36 2022
config:

NAME STATE READ WRITE CKSUM
bigdata ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

終わった。当たり前だけどまだ容量は増えてない（3TBから8TBへのジャンプ）
sdbをやる

.-23:15 asonas@archlinux:~
-% sudo zpool offline bigdata ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 [sudo](weblog-authoring-wiki://31) asonas のパスワード: .-[23:16](weblog-authoring-wiki://32) asonas@archlinux:~ -% zpool status
pool: bigdata
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: resilvered 2.00T in 05:52:18 with 0 errors on Tue Mar 15 23:05:36 2022
config:

NAME STATE READ WRITE CKSUM
bigdata DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 OFFLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

HDD交換済ませた
パーティションもきった（ログ略）

.-23:36 asonas@archlinux:~
`-% zpool status
pool: bigdata
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: resilvered 2.00T in 05:52:18 with 0 errors on Tue Mar 15 23:05:36 2022
config:

NAME STATE READ WRITE CKSUM
bigdata DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ONLINE 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 OFFLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

  • sudo zpool replace bigdata ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YDBBK

.-23:41 asonas@archlinux:~
`-% zpool status
pool: bigdata
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Tue Mar 15 23:41:40 2022
600G scanned at 60.0G/s, 1.05M issued at 107K/s, 6.00T total
0B resilvered, 0.00% done, no estimated completion time
config:

NAME STATE READ WRITE CKSUM
bigdata DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ONLINE 0 0 0
replacing-1 DEGRADED 0 0 0
ata-WDC_WD30EZRX-00D8PB0_WD-WMC4N1136787 OFFLINE 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YDBBK ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

.-8:01 asonas@archlinux:~
`-% zpool status
pool: bigdata
state: ONLINE
scan: resilvered 2.00T in 05:48:39 with 0 errors on Wed Mar 16 05:30:19 2022
config:

NAME STATE READ WRITE CKSUM
bigdata ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ONLINE 0 0 0
ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YDBBK ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5RPHLYU ONLINE 0 0 0
logs
sdc1 ONLINE 0 0 0

errors: No known data errors

おわってた

2022-03-16

.-12:52 asonas@archlinux:~
`-% zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
bigdata 8.17T 6.00T 2.17T - 13.6T 1% 73% 1.00x ONLINE -

  • zpool online -e
    • をする

.-12:55 asonas@archlinux:~
-% sudo zpool online -e bigdata ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0MK ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YDBBK ata-WDC_WD80EAZZ-00BKLB0_WD-CA0YD0SK .-[12:55](weblog-authoring-wiki://38) asonas@archlinux:~ -%
.-12:55 asonas@archlinux:~
`-% zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
bigdata 21.8T 6.00T 15.8T - - 0% 27% 1.00x ONLINE -