FAQ of hard disk errors and data retrieval

Section 1: My hard drive has bad sectors / Blocks / area

Do i need to change it ?
Not necessarily, but If it is in warranty, and they allow you to replace it, a new one is not a bad idea, otherwise read on

it all depends on whether the bad sectors are expanding or not, if they are not, they are probably caused by shock to the hard drive, usually, it is enough to mark them as bad using “chkdsk /r” on windows and leave the drive working.

To find out if your bad sectors are Spreading or not spreading, do a “chkdsk /r” four times, make sure the same number appears in the second and third and fourth time (Forget the first time), then, if the second is different but the third and fourth are the same, then do the test 2 more times, and make sure you get the same number of bad sectors for trials 3, 4, 5, 6, if so, your bad sectors are not spreading.

You did not mention backup in the answer before, do we need to backup ?
People would typically ask you to backup just in case, i say you should always have backup of your most important files, non spreading sectors of the hard drive, in my humble experience do not contribute negatively to reliability, so my answer is, backup should be done regardless

How do i know how many bad sectors are marked on an NTFS hard drive ?
There is a tool called nfi.exe that comes with a bundle Microsoft makes available here http://support.microsoft.com/kb/253066/en-us this tool is part of (OEM Support Tools), it can tell you everything about a disk formatted in NTFS

Searching you hard drive for files or for a string in a file

On a linux box, there is a simple way to search for some text inside a file using regular expressions in the command line

So, the short answer, in Linux, you can simply execute this line

grep -R -B3 -A4 "MYTEXT" /var/ > /root/findingres.txt

this will show you 3 lines before the text and 4 lines after the string and the file name, it will search in the /var/ folder, and will store the searching results to /root/findingres.txt

The above is a simple example, you can use regular expressions to find more complicated stuff

Also, if you want to search the entier hard drive for a string, and you want to search only in one type of file that you want to specify using a file extension you can use the following

grep -R --include=*.txt "MYTEXT" /etc/

Now, if you want to search for a file by file name,

find / -type f -name "myfile.txt"

would look for an exact file name, if you are looking to find a file using wildcards, for example, any file that ends in a certain extension you would

find / -type f -iname "*.psd"

If you want to include directories in your search, remove the -type flag

Checking if a mail server is blacklisted anywhere

When i get delivery problems like the message i got today from a microsoft exchange server

I'm sorry to have to inform you that your message could not be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can delete your own text from the attached returned message.

host mail1.example.com[xxx.xxx.xxx.xxx] said: 550 5.7.1 Message rejected due to content restrictions (in reply to end of DATA command)

I usually run the test on the following locations

1- the road runner website allows you to check cloudmark… run a scan with
http://postmaster.rr.com/amIBlockedByRR

tells me about
Return Path Reputation Network Blacklist :
Spamhaus (ZEN) :
Road Runner Blocklist :
FBL Enrollment :
Road Runner Internal Name-Based Block :
Cloudmark Sender Intelligence :
Return Path Sender Score Reputation Score :
historical Road Runner Internal IP Address-Based block records

Now, MXtoolbox

http://www.mxtoolbox.com/blacklists.aspx

for

AHBL
BACKSCATTERER
BARRACUDA
BURNT-TECH
CASA-CBL
CASA-CBLPLUS
CBL
IMP-SPAM
INPS_DE
ivmSIP
ivmSIP24
LASHBACK
MAILSPIKE-BL
MAILSPIKE-Z
NIXSPAM
NOMOREFUNN
PSBL
RATS-Dyna
RATS-NoPtr
RATS-Spam
REDHAWK
SEM-BACKSCATTER
SEM-BLACK
SORBS-DUHL
SORBS-SPAM
SORBS-WEB
SPAMCANNIBAL
SPAMCOP
SWINOG
TRUNCATE
UCEPROTECTL1
UCEPROTECTL2
UCEPROTECTL3
WPBL

If i can not see anything in the blacklists / blocklists above, i will generally contact the server administrator

Dual SIM android phones

I like the idea of Dual SIM android phones because one SIM can provide data at a cheaper rate, the other can be my phone line, and i will not experience overages on my main phone like (billed at a premium when over quota).

The LOWDOWN, Get SAMSUNG, if you like get the ALCATEL, stay away from the LG, it is a disaster, 3 LG Phones suffered the same exact problems.

I have tried 3 Android dual SIM phones to date.

Samsung Galaxy Y Duos (S6102)
Excellent up to now, no problems with this phone at all, but we have only used it for 3 days up to now (We are 3 people with three phones)

ALCATEL ONE TOUCH BLAZE DUO OT-890D
A bit slow, dialer good, not very responsive, but works as a mobile phone, don’t expect to flawlessly play angry birds

LG Optimus NET duo – LG-P698
A complete disaster despite the excellent hardware (for the price), All 3 phones (now replaced with the SAMSUNG) have the same exact problems, software for all 3 phones was updated to the latest software, and still a disaster.

The best hardware rendered useless with LG’s very bad dialer, the dialer hangs, sometimes calls people and there is no way to stop the call, takes a good 10 seconds at best to hang up, and if you press hang up more than once, it will call the person back, all in all, the basics are hell in this phone.

The battery is also very very bad, 16 hours at best if you don’t use it at all (Had one 3G sim card and 1 GSM only SIM card), the power manegemnt of the phone is not all that good and that is probably the reason it depletes a battery bigger than the samsung battery in much less time.

The good side of it is that it is very fast when sharing the internet by becoming a potable hotspot (WIFI tethering), it’s browser is good, it’s hardware is good, it has 512MB of ram (That’s plenty), but it is a very bad phone.

The copy (cp) and move (mv) commands in Linux

For some reason, when you look for a way to move a folder and all it’s sub folders into another folder, overwriting the files if they exist on destination, you will find very little information, the reason behind that is that the mv command that both renames files and folders and moves them does not support the -R switch (Recursive), the answer is that you copy them, then delete the source

So, if i downloaded wordpress and i want to update my installation with the new files i would

cp -R /var/vhosts/wordpress/* /var/vhosts/example.com/

this will update my copy of wordpress by moving the contents of the wordpress folder into the virtual hosting directory of my website

KEEP IN MIND THAT THE mv AND cp COMMANDS OVERWRITES WITHOUT PROMPTING….

to make it prompt before overwriting you must provide a -i parameter

If you don’t want a prompt, and you still want to NOT overwrite destination files… you will need to also set the “–reply=no” (depreciated) parameter so that the mv command will answer NO to all overwrite situations

NOTE: –reply has been depreciated (no longer works)

the linux move (mv) command : Moving folders

to move a folder into another folder, assuming there is a file names text.txt inside the folder /hds/ssd/mysql/moveme/text.txt, after this command we will have the file in /hds/ssd/mysql/into/moveme/text.txt

mv /hds/ssd/mysql/moveme/ /hds/ssd/mysql/into/

To move the contents of moveme including sub-folders directly into the folder into without moving the folder moveme itself

mv /hds/ssd/mysql/moveme/* /hds/ssd/mysql/into/

Adding an FTP server and setting up users to access directories

This post is rather old, and everything is secure on the internet these days, so rather than FTP, it is recommended that you setup the new user with SFTP instead (Secure file transfer protocol), SCP is another option, but i have an sftp article ready for you here

So here is a quick guide to setting up a user to have access to a certain directory via FTP

You probably already use a Linux server, you access your files via SFTP or SCP, but you want to give someone access to a certain directory within.

Here is how it is done on a Debian squeeze machine

apt-get install pure-ftpd-common pure-ftpd

Then we need to add a group and default user for our program
groupadd ftpgroup
useradd -g ftpgroup -d /dev/null -s /etc ftpuser

pure-pw useradd test1 -u ftpuser -d /home/ftpusers/test1

pure-pw mkdb

This creates the file mentioned earlier called /etc/pureftpd.pdb, this file houses all information related to your virtual users

pure-pw passwd test1

Once password is set, update the database

pure-pw mkdb

To delete a user

pure-pw userdel test1

pure-pw show test1
pure-ftpwho

Create symlink to add PureDB to authentication methods

cd /etc/pure-ftpd/auth
ln -s ../conf/PureDB 50pure

Disable PAM authentication unless you need it

echo no > /etc/pure-ftpd/conf/PAMAuthentication

Disable UNIX authentication unless you need it

echo no > /etc/pure-ftpd/conf/UnixAuthentication

site would not delete from PLESK

Today, as i was managing my PLESK Version 9 for Windows server, i noticed that every website i put a tick beside and tried to remove (deleting all files and all related), i ended up disabling and it just won’t get removed

So here is the solution

Open the command prompt
in the command prompt (CMD) change the directory to the plesk bin directory like so

cd c:ParallelsPleskadminbin

Then, execute the command

domain.exe --remove domain.com

You should now find that the domain is no longer in the list of domains.

Sometimes i get an error deleting stating that a file (dll) in system32 directory is in use by…. the answer is reboot the server, then try deleting a few times with the command above until you see the success message

Just like the previous post, the reason why running from the command line works and from the control panel does not work is unknown to me.

reactivating websites on Plesk for windows.

On a PLESK for windows installation, there was a problem reactivating websites, everything seems fine when you activate, then you refresh the page and it is inactive,

the answer is to first make sure the user is still active (date for user not domain name), then to activate the domains from the command line. Why ? not sure, but this is certainly still true for version 9 of plesk panel and soon i will be testing it for version 10.

cd c:ParallelsPleskadminbin

Then

domain.exe –on domainname.com

(Type it in yourself, and notice that — is 2 consecutive dashes)

You should see

SUCCESS: Changing status of domain ‘domainname.com’ completed.

Directory and sub directory sizes and disk usage

Although this belongs to the Linux Command Cheat Sheet, this post is here to list all the Directory / Sub Directory and Disk drive usage

To begin with, if it is disk space you are looking for information about, the following command should display the useful disk usage information

df -h

If you want to know the size of all sub directories in a directory

Move to the directory containing the sub directories (if root, can take a long time)

du -sh *

Now, if you want to know all the directories above a certain size, in this case that measure in GBs, here is how you do it

du -h <dir> | grep '[0-9\.]\+G'

If the above lists directories not within the gigbytes domain, you can use the following instead

du -h <dir> | grep '^\s*[0-9\.]\+G'

Splitting INNODB files to separate .ibd files

Please note: most of you are probably visiting this post for the answer to whether you can use multiple disks with innoDB, the answer is, even though you can not move idb files around like when you move MYI MYD files, you can move an IDB file while the DB engine is not running, then create a symlink where the Innodb expected to find the original file, then start mysql again, so the short answer is that innodb does work with symlinks just like myisam.

The second answer is, you can not use more than one disk on the same table (even though you can for the same database by putting every table on a separate disk).

As you would probably know, innodb, by default stores all data into its central, single file data dictionary (on debian, /var/lib/mysql/ibdata1), you can ask innoDB to put every table’s data in a separate file (tablename.ibd) by simply adding the directive innodb_file_per_table anywhere under [mysqld] section.

Databases already in single file will stay there, the directive only affects the creation of new databases, InnoDB does not mind mixed mode regarding ibd files, dome tables in a single database can be in the shared file, and some can have their own files. files created while the directive is in effect will get separate files, you can then remove the directive from your my.cnf and the engine will start putting new databases back into the shared file.

Mind you, unlike ISAM tables, you can not move these files around, in MyISAM you can simply copy the 3 files that are any table, the .frm, .MYI and .MYD and you have your table elsewhere, in INNODB, this is not the case as much is still stored in INNODB’s ibdata1 , also, innodb files are connected to a certain database with internal strings, Yet, you can copy the whole data directory and have it work, remember, log files are also needed and must be kept.

To move an .ibd file and the associated table from one database to another, use a RENAME TABLE statement:

RENAME TABLE db1.tbl_name TO db2.tbl_name;

The main disadvantage of splitting data files per table is disk space, files in INNODB never get smaller, when in 1 file, other tables can use that space of deleted rows in another table, when split apart, you can only use the occupied space again if you add rows to the same table that had deleted rows.

The advantages are

1- You can utilize multiple disks for both space and performance (By using symlinks)
2- You can backup certain tables and leave others to add flexibility to your backup plan