Category: Hard disks

Disk Spin down (Tested with Bullseye 2022)

Even though everything concerning block devices in linux has shifted to unique identifiers, hdparm has not, and will still use the old /dev/sdx system

To control disk spindown, and to manually issue commands, you will need to have the package installed

apt-get install hdparm

There is a probelm with disk spindown via hdparm, the problem is that you must address a disk as /dev/sdc , which changes in the case of USB media and other disks, even when you add slaves,

hdparm -Y /dev/sdb will spin a disk down instantly

hdparm -S 240 /dev/sdb will set this disk to sleep when idle for 20 minutes (5 second units here)

or adding at the bottom of the file /etc/hdparm.conf a section such as

/dev/sdc {
spindown_time = 240
}

to make those changes persistent across reboots.

To check the status of a disk, here is what you do

hdparm -C /dev/sde

You could get one of the following results
When spun down…
drive state is: standby
When active
drive state is: active/idle

Don’t make your disks spin-down too often, 20 minutes is good for me almost in all circumstances.

If the disks don’t spin down, chances are that selftest is enabled…

Check if it is enabled with

smartctl -a /dev/sdb
if it reads
Auto Offline Data Collection: Enabled.

then you need to disable it with
smartctl --offlineauto=off /dev/sdb

then wait for them to finish (if a test is running) then spin down.

The lowdown: Yes you can, try the free OSFMount.

How i found out about it ? a friend sent me his laptop to un-dlete files for him, i didn’t have time to see how i can un-delete under windows, so (with his permission) i mounted his laptop hard drive on my computer (Linux), then DDd the whole drive to a 250GB image file, put the hard drive back where it was (in the laptop), and sent it back to him so that he can continue using it, once i found the time, i simply copied the image to a Windows computer, mounted it with OSFMount, then un-deleted everything with Recuva (the best un-delete software in my opinion), put his files on an external hard drive and sent it his way.

Images created with dd , ddrescue , or dd_rescue are not formatted, they are the direct copy of a whole disk, including boot records, partition tables, and file system, so mounting such images should not be hard at all, and indeed, turns out there is a program that can mount them under windows (i would not be surprised if it turns out there are hundreds that do that), but for now, this seems to be a champ, and it seems to be free.

Yet, this program seems to be more than a mounting tool for direct disk images, it also mounts CD images (i guess the one i currently use (virtual clone drive) is obsolete, creation of RAM disks, and can open a big bunch of other image formats (nrg, SDI, AFF, AFM, AFD, VMDK, E01, S01).

So there you are, all you need for your disk mounting needs in 1 program 😀

Cheers

1- Before using dd, ddrescue, or dd_rescue, you need to know which disk is which, you can do that by simply using the command “fdisk -l” in my case, the old disk turned out to be /dev/sdb and the new un-partitioned disk is /dev/sdc.

So, i have been cloning a 2TB hard drive ( WD20EARS ) to a WD20EARX, same disk, but with a few differences

WD20EARS is sata 2 and the other is sata 3, another difference is that using “hdparm -I /dev/sdb” the older WD20EARS reports (And should not be true)

WD20EARS

Logical/Physical Sector size:           512 bytes

wile with “hdparm -I /dev/sdc” the newer WD20EARX reports

        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes

The first clone did not work for a reason unknown to me, i cloned my NTFS disk with ddrescue (gddrescue) on a linux (because i don’t know how to clone on windows) and then plugged it into windows, where it simply did not work, and in disk management reported the disk as un-partitioned space, so now i want to do the thing again, but i don’t want that slow performance, so i increased block size to 4KiB. (UPDATE: THE NEW COPY WITH 4KiB DID WORK BUT I DONT KNOW IF THE 4KiB SIZE IS RELEVANT, MAYBE YOU SHOULD TAKE A LOOK AT THE SECOND DIFFERENCE BETWEEN THE DISKS UP AT THE BEGINNING OF THE POST)

For now, i will try the cloning with the command (Only change the block level for advanced format hard drives)

Note, block size no longer works, and it is called sector-size, but the short letter for it -b is still the same, so we will change this to the line below it
ddrescue --block-size=4KiB /dev/sdb /dev/sdc rescue2.log
ddrescue -b=4KiB /dev/sdb /dev/sdc rescue2.log

And if all of your data is important, you can ask ddrescue to retry every bad block 3 times (or as many times as you wish) with the -r command

ddrescue --block-size=4KiB -r3 /dev/sdb /dev/sdc rescue2.log
ddrescue -b=4KiB -r3 /dev/sdb /dev/sdc rescue2.log

And what do you know, the disk now works on my WINDOWS machine 😀 no errors and no nothing, great, so now to some details about the copy

The result up to now is that i am reading at a maximum of 129MB while the average (in the first 60 GBs is 93018 kB/s), if this continues, i will be done in less than 6 hours.

The part that does not make any sense to me is that western digital states clearly in the specs that the maximum (Host to/from drive (sustained)) is 110 MB/s for both drives, it must be that i need to wait a bit more and see what that actually means.

rescued:         0 B,  errsize:       0 B,  errors:       0
Current status
rescued:    74787 MB,  errsize:       0 B,  current rate:     119 MB/s
   ipos:    74787 MB,   errors:       0,    average rate:   93018 kB/s
   opos:    74787 MB,     time from last successful read:       0 s
Copying non-tried blocks...

Now, once done, you can have the OS reload the partition table without having to restart, you can simply use the command partprobe

partprobe
or
partprobe /dev/sdc

To use partprobe, you need to install parted

apt-get install parted

If it were a linux drive, an advanced format drive would not have it’s first sector on sector 63 but rather on sector 2048, which is at exactly 2KiB, it could (but usually does not) start at any other value divisible by 8.

Windows probably does something similar for our AF Disk, so asking parted about our ntfs disk, this is what parted says

Model: ATA WDC WD20EARS-00M (scsi)
Disk /dev/sdb: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  2000GB  2000GB  primary  ntfs

1049kB is 1074176 bytes, Which when divided by 8 is 134272 (divisible by 8).

NOTES:
-There is a tool specifically for cloning ntfs volumes called ntfsclone, i am not sure what extra features it provides that are specific to ntfs, i have never used it before, with my disk that has bad blocks, i can only rely on gddrescue.
-A block is 512 on regular drives, and 4096 on newer ones, if you want to backup the hard drive’s geometry, you can do one of the following
Backup the first 63 blocks (MBR + Bootloader). on a “non advanced format” drive

dd if=/dev/sda of=/mnt/storage/sda.vbr bs=512 count=63

On an advanced format drive, we can try

dd if=/dev/sda of=/mnt/storage/sda.vbr bs=4096 count=63

Which, will make us read 258048 bytes rather than the traditional 32256 bytes (around 250K rather than 32K)

This article is work in progress, i have started the ddrescue and waiting for it to finish before i go on with this post.

One thing you should notice that this is the GNU DDRESCUE, from the package gddrescue, not the old script dd_rescue that is a wrapper around the dd program.

It has been some time since i found out that dd_rescue has been replaced by the newer rewritten ddrescue from the gddrescue package, to be more specific, since i posted this back in march 2011.

So, now i have yet another disk that is busted, with 3 partitions, but not like that one, this one simply has so many bad sectors, it’s a 2TB western digital caviar black that is causing me trouble

So i couldn’t find a 2TB caviar black, so to be on the safe side, i got a 3TB western digital green, partitioned it and formatted it like i describe here.

So, now that i have a hard drive that needs rescuing, lets revise what we need to do

1- Install the new ddrescue tool gddrescue
apt-get install gddrescue
2- Run ddrescue, make sure to use a file to resume in case we get interrupted (sometimes saves days of rescue can be lost and need to be done again, if they have not been damaged with disk deterioration that is)

ddrescue /dev/sdb /hds/3tb/2tb.img /root/resumelog.log

If you lose power, or get interrupted, or need to restart your computer, ddrescue will resume ONLY if you use the same exact line above once again, it will then use the log file to append to the existing output file.

Now, we have an image, we can now mount that image and take a look, so we mount the image on a loop

You could have partitions on the original disk, in my case i had 3 EXT2 partitions, the data i need is on the third partition

So i enter parted (debian package), and did the following

Using /hds/3tb/2tb.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit
Unit?  [compact]? B
(parted) print
Model:  (file)
Disk /hds/3tb/2tb.img: 2000398934016B
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start          End             Size            Type     File system  Flags
 1      32256B         400077619199B   400077586944B   primary  ext2
 2      400077619200B  700144058879B   300066439680B   primary  ext2
 3      700144058880B  2000396321279B  1300252262400B  primary  ext2

So now i know that the partition i want (third) starts at 700144058880, that’s all i need to know to mount this as a loop device
I am mounting loop device simply because i want to run fsck (disk check) on the partition before actually mounting that partition.

First, what is the next free / available loop device number ?

losetup -f

this should produce output such as

/dev/loop1

So now we know that we need to mount this on loop1 since it is the next available (not in use) loop device.

losetup /dev/loop1 /hds/3tb/2tb.img -o 700144058880

So, now loop 1 has my partition, you should omit the -y if you want to manually agree to every repair fsck wants to make

fsck.ext2 -y /dev/loop1

Great, now we should have a clean file system to mount
Even though we can mount the attached loop directly, i will demonstrate how to mount the loop and how to mount the image on a loop directly. i am doing this so that this tutorial would have the complete command referance of what tools and command parameters you might need.

First methode, the detach / release the loop device then mount it again in one go, this is done as follows
(-d means detach)

losetup -d /dev/loop1

Then, we attach with the foolowing command, notice how we used the starting ofset exactly like we did when attaching to a loop device.

Your mount command here

The other way is simply to mount our already attached loop device as follows

mount -t ext2 /dev/loop1 /hds/img

Now, we can mount this partition

mount -o loop,offset=700144058880 harddrive.img /hds/img

Or if you like, you can mount it read only

mount -o ro,loop,offset=700144058880 harddrive.img /hds/img

mount | grep /hds/3tb/2tb.img
/hds/3tb/2tb.img on /hds/img type ext2 (ro,loop=/dev/loop1,offset=700144058880)

It is simple, here is what you need to know

You can format it EXT4, but ext2 and ext3 are also OK ! ext2 and ext3 allow up to 16TB disks, and file sizes of up to 2TB, ext4 allows much more.

Any linux kernel newer than 2.6.31 should work just fine with “Advanced format” drives using the exact same steps in this article.

MBR only supports 2TB drives, you need GPT, so let us get started

1- apt-get update
2- apt get install parted
3- parted /dev/sdc
4- mklabel gpt
5- Answer yes to: Warning: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? yes
6- mkpart primary ext4 0% 100% (to make a partition as big as the disk (will occupy starting from first megabyte (for alignment) to the end of disk))
7- quit

FYI, if you want multiple partitions, here are the 2 lines that should replace step 6
6- mkpart primary ext4 0% 40%
6- mkpart primary ext4 40% 100%

and remember to format both (sdc1 and sdc2) when you are done with parted

Now to formatting the drive

mkfs.ext4 /dev/sdc1

Before mounting it, i like ext4, but i don’t want a journaling OS on this drive that is not the system drive, so i will need do a few things to the drive first

Lazy writeback

tune2fs -o journal_data_writeback /dev/sdc1

No Journaling

tune2fs -O ^has_journal /dev/sdc1

Now to check what we have

dumpe2fs /dev/sdc1 |grep 'Filesystem features'

Or maybe if you want the whole thing on the screen

dumpe2fs /dev/sdc1 |more

if has_journal option exist when executing the first – you have journal on the file system

And there we are, Now we need to mount it at boot time by adding it to fstab, to do that, we will need the disk’s unique ID !

8- Now executing the following command will give you the unique ID of this new partition for use with fstab (The disk list we will edit below in step 10)
blkid /dev/sdc1
9- create the directory where you want to mount your hard disk, for example
mkdir /hds
mkdir /hds/3tb
10- Now, we add the following line to fstab, notice that noatime increases performance, but some applications might need or rely on it. postfix does not and i have verified that.

UUID=b7a491b1-a690-468f-882f-fbb4ac0a3b53       /hds/3tb            ext4     defaults,noatime                0       1

defaults and noatime are but only a couple of options, here are more options that you can add
nofail = If the disk is not present, continue booting
nobootwait = Limit the amount of time you plan to wait
noauto = Don’t mount it until I issue a “mount /dev/sdb1”, or mount “/hds/thisdisk” command

11- Now execute
mount -a

You are done,. if you execute
df -h
You should see your 2+TB hard drive in there !

To make sure the drive is aligned correctly, i like to write a file on it and see how fast that goes… so let us use a 2GB file

dd if=/dev/zero of=/hds/WD2000_3/deleteme.img bs=1M count=2000

Outcome came out (for a western digital black 2TB)
First run: 2097152000 bytes (2.1 GB) copied, 5.94739 s, 353 MB/s
Consecutive runs: 2097152000 bytes (2.1 GB) copied, 11.1405 s, 188 MB/s
Outcome came out for a western digital green 3TB
First run: 2097152000 bytes (2.1 GB) copied, 8.32337 s, 252 MB/s
Consecutive runs: 2097152000 bytes (2.1 GB) copied, 14.376 s, 146 MB/s

the consecutive runs give close results, what i printed here is the average

Section 1: My hard drive has bad sectors / Blocks / area

Do i need to change it ?
Not necessarily, but If it is in warranty, and they allow you to replace it, a new one is not a bad idea, otherwise read on

it all depends on whether the bad sectors are expanding or not, if they are not, they are probably caused by shock to the hard drive, usually, it is enough to mark them as bad using “chkdsk /r” on windows and leave the drive working.

To find out if your bad sectors are Spreading or not spreading, do a “chkdsk /r” four times, make sure the same number appears in the second and third and fourth time (Forget the first time), then, if the second is different but the third and fourth are the same, then do the test 2 more times, and make sure you get the same number of bad sectors for trials 3, 4, 5, 6, if so, your bad sectors are not spreading.

You did not mention backup in the answer before, do we need to backup ?
People would typically ask you to backup just in case, i say you should always have backup of your most important files, non spreading sectors of the hard drive, in my humble experience do not contribute negatively to reliability, so my answer is, backup should be done regardless

How do i know how many bad sectors are marked on an NTFS hard drive ?
There is a tool called nfi.exe that comes with a bundle Microsoft makes available here http://support.microsoft.com/kb/253066/en-us this tool is part of (OEM Support Tools), it can tell you everything about a disk formatted in NTFS

I accedientally pulled the power plug of a PC from the socket, and that PC was just starting to boot. The seagate hard drive inside stopped working, and the bad sectors turned out to affect the partition table, in any case, i slaved it on a windows vista PC, then into the Computer management, disk management panel, and what do you know, as if it has no paritions…

The solution to detect the boundaries of the 4 partitions it had is software called XXXX

Ran the software (The analyze option) , and what do you know, my partions exactly, 100MB made by Windows 7, a 479 GB partition for Windows, a 1GB swap partition for linux and an EXT3 partition for Linux…

So happily i asked the software to write the partitioning info to the disk, but the disk won’t hold the data, the bad sectors are where Windows writes the partition information

So, i ran down to the computer shop (In our building), and got the same exact drive (Seagate 500GB Model number xxx)

Mounted both on a Linux machine as slaves, both the damaged and the target.

To find out which one is SDC and which one is SDB, i watched as the linux machine booted, and as it booted, it threw in errors saying SDB all the time, so i know that SDB is the busted drive !

Installed gddrescue (apt-get install gddrescue), and ran it with the following command

ddrescue /dev/sdb /dev/sdc resumelog.log

(The additional log file helps us resume in case of interruption)

Once that is done, i put the new hard drive in a Windows machine, still can not see any partition info

1- Ran xxxx, it can see the 4 partitions, write changes… and what do you know, the partitions stick, we are good to go, i restart, but still, Windows can now see the partitions, but thinks drive G is not formated !

So i opened the command prompt (Elevated), then ran the command

chkdsk g: /f

the /f stands for fix, the thing took some time, but after the restart drive G works fine, all files are in there, and no one wants to kill me no more 🙂

The results of the commands on this post are Here

On windows, You can use speedfan, under linux you can read the smart data as follows

Use the smartctl command, but first you must install the package

apt-get install smartmontools

1- Checking for smart support, (All recent hard drives have it but you need to enable it in BIOS)

smartctl -i /dev/sdb

2- Enable reading it

smartctl -s on -d ata /dev/sdb

See it

smartctl -d ata -H /dev/sdb

Read more

smartctl -d ata -a /dev/sdb

Now some manufacturer specific smart atributes exist, you need to google them out, Also smart is not definitive, if S.M.A.R.T data says ok but there is a ticking noise in your hard drive, Don’t trust the hardware data, get backup.

Some hard drives like the Samsung Spinpoint come with extensive diagnostics software, My 3 2.5′ disks pass the test, pass the smart test, and tick when warm, i will let you know if it was the tick of death sometime soon

To run a self test…

smartctl -t <short|long|conveyance|select> -C /dev/sdb