(Seemingly) Slow SAS drives & FreeNAS
So I recently wasted 70 hours waiting for a couple of badblocks
tests on my “new” NAS system at home. I’m using 2 TB SAS drives
(quite old, but quite cheap at ~$10 USD per drive) which I don’t trust
yet, so I figured that two runs of badblocks on each drive should be
enough to tell if they are good or not.
Problem was, instead of the expected ~24 hours per run (out of four
tests in each badblocks run, if that makes sense), I had just
finished the first test after 70 hours! Looking at iostat, my
writes were approx. 7 megabytes per second, WTF?
So I went on another round of googling, and eventually found a relevant thread on the iXsystems forums. So, it seems that the writeback cache isn’t enabled for my drives when using FreeBSD/FreeNAS, but the OP stated that this was the case when using CentOS (maybe a udev rule at play there?).
I tried to enable the writeback cache, and my write speeds (and writes per second) increased by 10-20 times. That’s better, but it doesn’t seem to stick between boots - but that’s easy to fix with some terrible bash scripting!
#!/usr/bin/env bash
for dev in /dev/da*; do
current_state=$(smartctl -g wcache $dev|grep -i "Writeback Cache is"|grep -iEow "(Enabled|Disabled)$")
if [ $current_state != "Enabled" ]; then
new_state=$(smartctl -s wcache,on $dev)
success=$?
if [ $success -ne 0 ]; then
echo "Something went south when enabling writeback cache on ${dev}"
echo "${new_state}"
elif [ $success -eq 0 ]; then
echo "Writeback cache enabled for ${dev}"
fi
elif [ $current_state == "Enabled" ]; then
echo "Writeback cache already enabled! (${dev})";
fi
done
iostat -Kx after enabling the writeback cache:
extended device statistics
device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b
ada0 0 0 0.0 0.0 0 0 0 0 0 0
da0 0 1725 0.0 110428.0 0 0 0 0 1 98
da1 0 1748 0.0 111924.1 0 0 0 0 1 98
da2 0 1783 0.0 114123.6 0 0 0 0 1 98
da3 0 1901 0.0 121702.2 0 0 0 0 1 97
da4 0 1756 0.0 112388.7 0 0 0 0 1 98
da5 0 1735 0.0 111097.2 0 0 0 0 1 98
pass0 0 0 0.0 0.0 0 0 0 0 0 0
pass1 0 0 0.0 0.0 0 0 0 0 0 0
pass2 0 0 0.0 0.0 0 0 0 0 0 0
pass3 0 0 0.0 0.0 0 0 0 0 0 0
pass4 0 0 0.0 0.0 0 0 0 0 0 0
pass5 0 0 0.0 0.0 0 0 0 0 0 0
pass6 0 0 0.0 0.0 0 0 0 0 0 0
Before enabling the writeback cache:
extended device statistics
device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b
ada0 0 0 0.0 0.0 0 0 0 0 0 0
da0 0 111 0.0 7145.1 0 8 0 8 1 100
da1 0 112 0.0 7174.8 0 8 0 8 1 100
da2 0 112 0.0 7191.8 0 8 0 8 1 99
da3 0 112 0.0 7204.5 0 8 0 8 1 100
da4 0 112 0.0 7191.8 0 8 0 8 1 99
da5 0 112 0.0 7174.8 0 8 0 8 1 100
pass0 0 0 0.0 0.0 0 0 0 0 0 0
pass1 0 0 0.0 0.0 0 0 0 0 0 0
pass2 0 0 0.0 0.0 0 0 0 0 0 0
pass3 0 0 0.0 0.0 0 0 0 0 0 0
pass4 0 0 0.0 0.0 0 0 0 0 0 0
pass5 0 0 0.0 0.0 0 0 0 0 0 0
pass6 0 0 0.0 0.0 0 0 0 0 0 0