QWORQS

There are many tools that precompile PHP to make it run faster, up to now, my favorite is APC which also serves as a very fast value cache (for persistence between requests), a value cache much faster than memcached (but not as distributed).

For some time, i have been optimizing further by asking APC to never check if the file is modified on the disk, and whenever my software is modified i would manually clear the APC cache so that the whole script can be compiled all over again (I say compiled loosely speaking, in reality, it is simply turned into bytecode).

In any case, when you have a server with plenty of ram, it would be convenient if the PHP engine can read the file itself in byte and skip that step for compiled files, and from the way the linux kernel works, those files would be cached from disk into ram (because when a file is read or written, linux keeps a cache of it in ram).

So, bcompiler should be a good extension to PHP that fits such a criteria, and is probably my new way of running my scripts.

Also, bcompiler hides my source code, but i am not interested in that to protect my intellectual property, usually i am not very concerned about my intellectual property because it takes a very good programmer to understand a program and take things from it, and if the person is such a programmer, well, he can also write his own, and with the help of google, he can arrive at something like what i am doing, so to make a long story short, i am interested in hiding my source code for application security reasons, or Security through obscurity as MircoSfot would put it

Note that if your kernel is before 2.6.33 you can check, but it won’t be working !

in the case that you don’t want to update your kernel, and you just want to trim your disk, try wiper.sh or fstrim, both are command line tools that you can run manually or put in a cron job. if you do want to update your kernel, here is how on debian squeeze

So for example, if you are on debian 6 squeeze, you need to get a kernel from the backports (add the line “deb http://backports.debian.org/debian-backports squeeze-backports main” to your /etc/apt/sources.list then apt-get update then apt-get -t squeeze-backports install linux-image-3.2.0-0.bpo.2-amd64) to get the new kernel, it will then work.

I assume you already have an ext4 file system with discard option in fstab as described on this website

Also note: Many modern SSDs will not reclaim the TRIMmed space., so if using the test below you see zeros, discard (trim) is working 100%, if you don’t. it may or may not be working… but if you wait fdor a significant amount of time, then reboot, the zeros should appear in that exact location even if the disk does not reclaim instantly … happyt trimming, now to the procedure

now, write a file to the ssd (random numbers)
dd if=/dev/urandom of=/hds/ssd300/myfile.bin bs=1M count=3

Find the location where the file begins
hdparm –fibmap /hds/ssd300/myfile.bin

Now, take note of the start address and use it in this command replacing xxxxx

hdparm –read-sector xxxxxx /dev/sdb
You should see random numbers

Delete the file
rm /hds/ssd300/myfile.bin
Sync with the command
sync

Wait for 2 minutes
the issue the same command to read again
hdparm –read-sector xxxxxx /dev/sdb
You should now see all zeros, if you do not, the disk has not been trimmed 🙂

I am writing this because the stuff you need is not in one place elsewhere, this is what you really need to know, and i want to keep this very short, if you like you can read more elsewhere, this one will only share what you really should know.

1- Do i need trim ?
For reading NO, so if you write once and read 102112913 times, you are good without trimming anything, without trimming, disk writes are slow, reading is absolutely not affected by trim.

2- What is the difference between the ext4 discard option and running fstrim myself manually every once in a while, or even put it in a cron job ?
on ext4 with trim enabled, blocks are trimmed (erased) whenever they are no longer in use by the file system, meaning, when data is deleted from a block, the physical flash memory is erased right after the data deletion, so your disk will remain trimmed all the time, the overhead is not much because the OS knows the block it just freed, so it simply does no more math other than issue a second command to trim, when you run fstrim, fstrim will read the whole file system, and whenever it finds an empty spot, it will trim (hardware erase).

3- i forgot to enable discard, do i just enable it and all is good, is that safe.
Yes it is safe, but enable it, then manually run fstrim only once, or you can wait, and eventually all spots will be trimmed after the get written to and erased again.

Ext4 (the new linux file system) supports TRIM when you mount the disk with the discard option, you can use tools to trim with ext2 or ext3, but it won’t be automatic and not as efficient.

1- But i want ext2 because i don’t want Journaling
ext2 is in fact ext3 without the journal, in ext4 you can remove the journal as well with no problems at all, there are no consequences, ext4 was designed to run with or without journaling

How do i format the disk in ext 4 and enable trim ?
For instructions on creating ext4 partitions, see here , as for the mounting, the line should have an extra option called discard and it should look something like this in your /etc/fstab

UUID=b7a491b1-a690-468f-882f-fbb4ac0a3b53       /hds/3tb            ext4     defaults,noatime,discard                0       1

You should be done, there is nothing more to do

3- I am stuck with ext2 and i don’t want to move, reformet and then move back again
before that, do you know that you can convert the drive to ext4 ?

4- I don’t want to convert anything to anything, i just want to manually trim
Thats easy, use the command
fstrim /hds/myssddisk

and you are done, but mind you, on anything but ext4, this will trim the whole unused space trimed or not trimmed

An application for windows that i wantin Linux is WINSCP, but it seems the author of WINSCP says (on his forum) that ” Sorry, there’s no chance for that.”

In any case, i have no doubts there are hundreds of applications that can do the Job, in fact the file browser that comes with your gnome or KDE already opens FTP and SFTP and SCP connections, so you need to look no further.

There are also applications that can mount a remote file system that is run on SSH, xxx is one such software

But truth be told since the days of Norton commander, i have always liked the two window view that winSCP is similar to.

So in this post, i will add screenshots of the applications similar to WINSCP, i will try both krusader and filezilla (Yes, filezilla does support)

apt-get update

then

apt-get install krusader filezilla

With Krusader, it is a good idea to install 

apt-get install kdiff3 kompare xxdiff krename rar unrar zip

Another software to be tested would be snowflake, confusingly it is being renamed to muon, which is already the name of a package manager for debian !, in any case, installing snowflake is as simple as downloading the deb file then installing it

wget https://github.com/subhra74/snowflake/releases/download/v1.0.4/snowflake-1.0.4-setup-amd64.deb

then install it

dpkg -i snowflake-1.0.4-setup-amd64.deb

Worth noting that on my 4K display which has a 200% setting, snowflake is not usable, the font is so small, and clicking on something is a challenge, so to work around this while the maintainers fix this for people who have settings like mine, i run snowflake from the terminal with

java -Dsun.java2d.uiScale=2.5 -jar /opt/snowflake/snowflake.jar

And now we have both, on my computer which is a fresh install, krusader was a 90MB download, in your case, it is probably much less because most of the things downloaded are libraries you probably already have.

in any case, let me take those screenshots of WINSCP’s alternatives and get back to this post

They say the number of rows you see in PHPMyADMIN is approximate and not very precise for innodb tables, i can tell you that depending on how the table has been used, the results can be very very inaccurate and sometimes irrelevant.

phpmyadmin wants to display a row count with the list of tables, in myisam, a value is maintained within the database saying how many rows are in the database, INNODB does not maintain such a value so it has to either respond accurately without regard to the delay and scans required or give an approximate reading, it will depend on how you ask, read on for more details.

Q: why are innodb row counts innacurate in PHPMYADMIN ?

Well, innodb would return an accurate count if it were asked formally with a

SELECT COUNT(*) FROM tablename"

But phpmyadmin can not execute such a statement with every listing of a database, because it means the database engine will have to read the whole database and scan all rows.

Q: in that case how come it works for MyISAM ?

A: MyISAM maintains a number stating how many rows are in there that is incremented and decremented with every insert and delete, so the database engine does not need to scan the whole table to give us a row count.

Q: Then where does PHPMYADMIN get those approximate numbers ?

For speed, PHP MY ADMIN would execute something similar (If not exactly)

SHOW TABLE STATUS WHERE 1;

This is when MyISAM would read it’s internal row count value, and Innodb would return an estimate because it does not maintain such a value.

You can also use the command to see a subset of the tables like this

SHOW TABLE STATUS LIKE 'wp_%';

to see status of all tables starting with ‘wp_’

Q: Why does innodb not maintain such a number

A: probably for performance reasons, to avoid the need to update such a number with every insert and every delete.

athurx is the atheros wireless driver, i have 2 atheros adapters on the same computer, one is 300Mb/s and the other is 150Mb/s

The reason for the blue screen is that the wireless interface driver(S) on my 64bit windows 7 (could be on any other platform too) is outdated, and it seems the old version of the driver has a problem in using 2 different atheros adapters on the same computer, the solution seems to lie in updating the drivers.

1- The Wireless N 150Mb/s (TP-LINK TL-WN722N, atheros AR9271 chip, should also apply to TL-WN721N that uses the same chip)
So, my atheros based TP-LINK TL-WN722N uses the Qualcomm atheros AR9271 chip, Windows installs driver version 2.0.0.32 dated 1/4/2010, and since Qualcomm atheros does not provide their drivers directly on there website, i downloaded the driver from TP-LINK, the new driver had version 2.0.0.62 and was dated more than a year later 4/20/2011, the problem with this driver is that it is NOT SIGNED, and therefore you have to manually install the driver then agree to install a non signed driver, simply asking windows to update from a directory will not update your driver, also check the driver versions before you update to see if your drivers have been updated.

2- The Wireless N 300Mb/s (tp-link TL-WN821N, Atheros AR7015)
This one also has an outdated driver that can be updated from the TP-LINK website, just like the one above.

Make sure you unplug the other driver as you update the software for the first, and once both have drivers installed, reboot and plug in both adapters, works like a dream up to now.

UPDATE: This post has been added at easywebdns : recovering ext3 hard disks

So, here are the tools you need

1- Checking a linux partition for bad sectors / Bad Blocks without deleting the data

badblocks -o /path/somefiletowriteto.txt /dev/sdb1

this will do a check and write the bad block numbers to a text file.

Please note that 6TB and 8TB hard drives will not work out of the box with badblocks program, you will probably get a message informing you that the boundary must fit a 32bit integer (Value too large for defined data type), in that case, you can always increase the block size with the (-b 4096) switch, this will give you 8 times more block addressing space (Since the default is 512 bytes.

If you want it to test the disks hard but you are OK with deleting the data (Say after you dd_rescue), then try the -w switch

badblocks -w -o /path/somefiletowriteto.txt /dev/sdb1

Or, to see information while it scans

badblocks -wsv /dev/sdd > /root/250bad-blocks.txt

Notice that the -w has to come before the -o since the -o must be followed by a file name.

2- fsck /dev/sdb1 , the all famous, need i say more ?

3- tune2fs

tune2fs -O ^has_journal /dev/sdb1

remove the Journal (Converting ext3 to ext2 file system)

4- Mounting a disk: mount -t ext3 /dev/sdb1 /adirectory

5- debugfs <- use it in Read only more, this thing is like brain surgery, 1 mistake and your FS is playing chess in heaven with all the other dead disks

6- dd_rescue /dev/sdb1 /directory/mydisk.img

7- Running fsck on a disk image

fsck -y /path/mydisk.img

8- Mount the image

mount /pathto/backup.img /mountpoint/data

this is a nice link for you http://www.linuxjournal.com/article/193 that already has info about the stuff i mentioned here

Those are probably the most common, will add more when i remember them

This tutorial was first written for Lenny, then tested on squeeze, wheezy… up to bullseye

In August 2021, I have revised this for Debian Buster (10) and Debian Bullseye (11).

If you have installed Debian Lenny with file sharing (in the lenny installer), you will need to add users to that installation and specify folders that need to be shared, here i will show you how to add a user and share a folder with that user.

If you have not installed file sharing while installing Debian lenny, you need to install them, the easiest way to do that is

apt-get install samba

Or if you like

run the command

aptitude

using the arrow keys and the enter key, expand “TASKS” then place the bar over File Server, then the plus sign to select it (Shift +), right after hit “g” then “g” on your keyboard to install them.

Once done, add the following lines to smb.conf

[public]
   comment = Public Folder
   path = /public
   writable = yes
   guest ok = yes
   guest only = yes
   force create mode = 775
   force directory mode = 775
[private]
   comment = Private Folder
   path = /private
   writable = yes
   guest ok = no
   valid users = @smbshare
   force create mode = 770
   force directory mode = 770
   inherit permissions = yes

Now to adding users, from the following 3 commands, After the second and after the third command, you will be asked to chose a password for the user joe

useradd yazeed -m -G users
passwd yazeed
smbpasswd -a yazeed

Now with this out of the way, you can also edit the following fields in the file /etc/samba/smb.conf

The following need to be edited

netbios name : Should become the name of the computer on the network, in windows, this is the machine’s name
hosts allow : If your network is like mine where PCs take IP addresses of the form 192.168.2.x, then this one should be 127. 192.168.2. where 127 is for the local machine, the following part of the IP is the part of the IP that prefixes the IP of all network machines that should be able to access this file server (usually you will change the 2 with a 0 or 1)
interfaces : 127.0.0.1/8 192.168.2.0/24, like above, if your IP subnet is not 192.168.2.x, change it here (usualy you will change the 2 with a 0 or 1)
remote announce : if your subnet does not start with 192.168.2. then change it to your own (usually you will change the 2 with a 0 or 1)
remote browse sync : if your subnet does not start with 192.168.2. then change it to your own (usualy you will change the 2 with a 0 or 1)

Now scroll down, you will see a sample folder, edit that to the folder you want to share, and copy it over and over again for any other folders that you want to share, You are done, now you should be able to open those folders from the network.

This article is work in progress, i have started the ddrescue and waiting for it to finish before i go on with this post.

One thing you should notice that this is the GNU DDRESCUE, from the package gddrescue, not the old script dd_rescue that is a wrapper around the dd program.

It has been some time since i found out that dd_rescue has been replaced by the newer rewritten ddrescue from the gddrescue package, to be more specific, since i posted this back in march 2011.

So, now i have yet another disk that is busted, with 3 partitions, but not like that one, this one simply has so many bad sectors, it’s a 2TB western digital caviar black that is causing me trouble

So i couldn’t find a 2TB caviar black, so to be on the safe side, i got a 3TB western digital green, partitioned it and formatted it like i describe here.

So, now that i have a hard drive that needs rescuing, lets revise what we need to do

1- Install the new ddrescue tool gddrescue
apt-get install gddrescue
2- Run ddrescue, make sure to use a file to resume in case we get interrupted (sometimes saves days of rescue can be lost and need to be done again, if they have not been damaged with disk deterioration that is)

ddrescue /dev/sdb /hds/3tb/2tb.img /root/resumelog.log

If you lose power, or get interrupted, or need to restart your computer, ddrescue will resume ONLY if you use the same exact line above once again, it will then use the log file to append to the existing output file.

Now, we have an image, we can now mount that image and take a look, so we mount the image on a loop

You could have partitions on the original disk, in my case i had 3 EXT2 partitions, the data i need is on the third partition

So i enter parted (debian package), and did the following

Using /hds/3tb/2tb.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit
Unit?  [compact]? B
(parted) print
Model:  (file)
Disk /hds/3tb/2tb.img: 2000398934016B
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start          End             Size            Type     File system  Flags
 1      32256B         400077619199B   400077586944B   primary  ext2
 2      400077619200B  700144058879B   300066439680B   primary  ext2
 3      700144058880B  2000396321279B  1300252262400B  primary  ext2

So now i know that the partition i want (third) starts at 700144058880, that’s all i need to know to mount this as a loop device
I am mounting loop device simply because i want to run fsck (disk check) on the partition before actually mounting that partition.

First, what is the next free / available loop device number ?

losetup -f

this should produce output such as

/dev/loop1

So now we know that we need to mount this on loop1 since it is the next available (not in use) loop device.

losetup /dev/loop1 /hds/3tb/2tb.img -o 700144058880

So, now loop 1 has my partition, you should omit the -y if you want to manually agree to every repair fsck wants to make

fsck.ext2 -y /dev/loop1

Great, now we should have a clean file system to mount
Even though we can mount the attached loop directly, i will demonstrate how to mount the loop and how to mount the image on a loop directly. i am doing this so that this tutorial would have the complete command referance of what tools and command parameters you might need.

First methode, the detach / release the loop device then mount it again in one go, this is done as follows
(-d means detach)

losetup -d /dev/loop1

Then, we attach with the foolowing command, notice how we used the starting ofset exactly like we did when attaching to a loop device.

Your mount command here

The other way is simply to mount our already attached loop device as follows

mount -t ext2 /dev/loop1 /hds/img

Now, we can mount this partition

mount -o loop,offset=700144058880 harddrive.img /hds/img

Or if you like, you can mount it read only

mount -o ro,loop,offset=700144058880 harddrive.img /hds/img

mount | grep /hds/3tb/2tb.img
/hds/3tb/2tb.img on /hds/img type ext2 (ro,loop=/dev/loop1,offset=700144058880)

My web server got hacked today, i know because my datacenter contacted me today telling me that there is a bruit force attack originating from my server to another server on a different network, so what is happening is that my server got hacked, then the hacker is using the server she hacked to hack other servers by sending FTP requests.

So, how come i got hacked when i am so obsessed with security, well, in reality, this is just an intermediate machine that i used to run a certain script that would move my mail server, and i did not (and did not see the need) to secure it.

What i usually do to secure my server is simply install fail2ban, in this case i did not out of lazyness but here is how i got hacked and how fail2ban would have protected me.

Before i show you the log files, this whole problem would not happen if i had a strong password combined with fail2ban

In the complaining partie’s log files

Tue Jul 24 22:28:27 2012: user: hauvouuc service: ftp target: yyy.yyy.yyy.yyy source: xxx.xxx.xxx.xxx
Tue Jul 24 22:28:27 2012: user: pkmcndgq service: ftp target: yyy.yyy.yyy.yyy source: xxx.xxx.xxx.xxx
Tue Jul 24 22:28:27 2012: user: malumdvc1 service: ftp target: yyy.yyy.yyy.yyy source: xxx.xxx.xxx.xxx

In my log files (auth.log):

Many lines like the following right below each other

Jul 24 18:03:08 run sshd[14229]: pam_unix(sshd:auth): check pass; user unknown
Jul 24 18:03:08 run sshd[14229]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=9.12-14-84.ripe.coltfrance.com 
Jul 24 18:03:10 run sshd[14229]: Failed password for invalid user ts3 from 84.14.12.9 port 41014 ssh2
Jul 24 18:03:11 run sshd[14231]: Invalid user ts3 from 84.14.12.9

Anod some lines like this

Jul 25 15:30:46 run sshd[10728]: pam_unix(sshd:auth): check pass; user unknown
Jul 25 15:30:46 run sshd[10728]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=217.119.29.135 
Jul 25 15:30:48 run sshd[10728]: Failed password for invalid user public from 217.119.29.135 port 34292 ssh2
Jul 25 15:30:48 run sshd[10730]: Address 217.119.29.135 maps to gamma2-7.cust.smartspb.net, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Jul 25 15:30:48 run sshd[10730]: Invalid user public from 217.119.29.135

Thousands of lines like this one

Jul 24 14:12:38 run sshd[2025]: error: connect_to 213.186.33.207 port 80: failed.
Jul 24 14:12:39 run sshd[2025]: error: connect_to 192.168.10.24 port 2110: failed.
Jul 24 14:12:39 run sshd[2025]: error: connect_to 195.130.65.50 port 80: failed.

OR

Jul 24 06:41:19 run sshd[9824]: error: connect_to 213.186.33.207 port 80: failed.
Jul 24 06:41:19 run sshd[13434]: Failed password for invalid user test from 202.28.123.191 port 37830 ssh2
Jul 24 06:41:20 run sshd[9824]: error: connect_to 213.186.33.207 port 80: failed.

And more like this

Jul 24 08:19:18 run sshd[20882]: pam_unix(sshd:auth): check pass; user unknown
Jul 24 08:19:18 run sshd[20882]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=puck748.server4you.de 
Jul 24 08:19:21 run sshd[20882]: Failed password for invalid user kk from 85.25.235.73 port 49213 ssh2
Jul 24 08:19:21 run sshd[20884]: Invalid user css from 85.25.235.73