QWORQS

The following is the shortest tutorial that should get you up and running…

1- install rtorrent

apt-get install rtorrent

2- download the torrents

wget http://cdimage.debian.org/debian-cd/7.0.0/amd64/bt-dvd/debian-7.0.0-amd64-DVD-1.iso.torrent

3- start rtorrent

rtorrent

Hit enter on the screen you are on, then hit tab, you should see a list of the torrent files you downloaded, type the first letter of your file then tab again, the program will either show you your file , or reduce the list to all the files that have that same prefix, add one more letter then hit tab again, keep going until you have your file.

Once done, use the arrow keys to select your torrent file, then hit ctrl+s to Start the torrent.

That’s all you need to know !

Using the top command, you can find out how much time your processor spends waiting for input and output from disk, the IO wait field, but how can we know what applications are responsible for that IO wait.

to examin disk io and iowait more closely, we need some tools, the following is a list of tools that can show you exactly what is causing all the IO wait so that you can start from there.

Sometimes, apps that use the disk with many random reads should go to the secondary disk. but to find that application, take a look at the following tools, i will be explaining each and every one of them as we go.

Here are a bunch of programs on linux that will tell you just that

• iotop – apt-get install iotop – simple top-like I/O monitor
• iostat – apt-get install sysstat – shows and monitors activity per disk – The sysstat package contains the sar, mpstat and iostat commands for Linux.
• isag – apt-get install isag – Interactive System Activity Grapher for sysstat
• dstat – apt-get install dstat – versatile resource statistics tool
• vmstat 1
• iostat 1
• lsof
• strace -e trace=open
• strace -e trace=open -p
• In ps auxf you’ll also see which processes are are in uninterruptible disk sleep (D) because they are waiting for I/O.
• fatrace – Amazing tool showing you which files are being accessed or written on the hard drive

examples

iostat -c -d -x -t 2 100

I got myself a new C850B797 toshiba laptop with an i7-3630QM processor and 16GB of ram (coz it’s cheap), the C850B797 is the grey edition in the C850 series, the processor is a 22nm Lithography and everything is very good as far as i can see, but then i realized that the laptop had one very serious issue, i am limited to 100mbit Ethernet, and i really do move plenty of files around all the time (many gigabytes), so i needed to get gigabit, but how ?

Usb speed is 480mbit/second, much of that goes overhead, so the Ethernet controller would probably be USB3 which would block my only usb port, so the answer is to get a USB3 hub and connect the USB gigabit ethernet to it, but it turns out that the hub followed by gigabit eithernet is already on the market from one of 2 providers.

The first, the one i got is the StarTech ST3300U3S USB hub and Ethernet controller, that uses ASIX AX88179 for gigabit Ethernet after a VIA VL811 USB3 hub, the question is, will i be getting the speeds i am hoping for from this ?

I will be running tests concerning reliability of the combo (since both are connected in serial) and let you guys know.

1- StarTech ST3300U3S USB hub and Ethernet controller – Power adapter included = $55, Also happens to be the same exact thing as the SYBA inf@zone (infozone) with the same chips and the same plastic casing, the SYBA infozone will sell at $33 if you buy 3 together, i found 3 for $98.99 on ebay, making it much cheaper than the $66 each for the same exact thing from StarTech, SYBA labels it as SY-HUB50056.)

AX88179 — USB3.0 to 10/100/1000M Gigabit Ethernet Controller
VIA VL811 – SuperSpeed USB 3.0 Hub Controller

The other options on the market are…
2- Kanex DualRole Gigabit Ethernet with 3-Port USB Hub (USB3GBITX) – Power adapter not included – $40
AX88179 — USB3.0 to 10/100/1000M Gigabit Ethernet Controller
No idea about the hub chip yet.

Please bare in mind that this same combo but with USB 2 and a 100Mbit eithernet adapter should not cost more than $10, it is the USB 3 and gigabit Ethernet that are expected to be this expensive.

Via also produces the VL812 which is almost identical to VL811

Disk Spin down (Tested with Bullseye 2022)

Even though everything concerning block devices in linux has shifted to unique identifiers, hdparm has not, and will still use the old /dev/sdx system

To control disk spindown, and to manually issue commands, you will need to have the package installed

apt-get install hdparm

There is a probelm with disk spindown via hdparm, the problem is that you must address a disk as /dev/sdc , which changes in the case of USB media and other disks, even when you add slaves,

hdparm -Y /dev/sdb will spin a disk down instantly

hdparm -S 240 /dev/sdb will set this disk to sleep when idle for 20 minutes (5 second units here)

or adding at the bottom of the file /etc/hdparm.conf a section such as

/dev/sdc {
spindown_time = 240
}

to make those changes persistent across reboots.

To check the status of a disk, here is what you do

hdparm -C /dev/sde

You could get one of the following results
When spun down…
drive state is: standby
When active
drive state is: active/idle

Don’t make your disks spin-down too often, 20 minutes is good for me almost in all circumstances.

If the disks don’t spin down, chances are that selftest is enabled…

Check if it is enabled with

smartctl -a /dev/sdb
if it reads
Auto Offline Data Collection: Enabled.

then you need to disable it with
smartctl --offlineauto=off /dev/sdb

then wait for them to finish (if a test is running) then spin down.

1- Installed the debian base system, debian wheezy (7) (But should work with debian squeeze (6))
2- apt-get install ssh openssh-server fail2ban
3- INSTALLED THE HARDWARE 8 CAMERA DVR CARD into the PCI slot
4- Modified the network interfaces to use a static IP, and pointed the DMZ of the router to that IP
5- installed some software (Even though some of them would have been installed anyway when i install zoneminder), but i do it anyway just in case they are optional in zoneminder

apt-get update
apt-get install apache2 php5-mysql libapache2-mod-php5 mysql-server
apt-get install ffmpeg libarchive-tar-perl libarchive-zip-perl libdate-manip-perl libdevice-serialport-perl
apt-get install libjpeg62 libmime-perl libstdc++6 libwww-perl zlib1g
apt-get install zoneminder

I can see close to the end of operation

ZoneMinder is stopped
invoke-rc.d: initscript zoneminder, action “status” failed.
Starting ZoneMinder: success

I hope it’s nothing too serious

Now, check if your card is supported, you issue lspci -v as root… Since my 8 port card is supported, i see eight stanzas of this

I got eight of the following labeled 03:02.0 03:02.1 03:02.2 03:02.3 03:02.4 03:02.5 03:02.6 03:02.7

03:02.2 Multimedia video controller: Techwell Inc. TW6816 multimedia video controller (rev 10)
Subsystem: Techwell Inc. TW6816 multimedia video controller
Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 3
Memory at febfe800 (32-bit, non-prefetchable) [size=1K]
Capabilities: [44] Power Management version 2

If you dont see your card’s devices, well, your out of luck, maybe you should try a newer kernel, i am using 3.2 from debian wheezy (7)

Now, to the second test

ls -ls /dev/video*

NO love, nothing, If i were lucky, i would get something like

ls -ls /dev/video*
0 crw-rw----+ 1 root video 81,  0 30. Jul 15:18 /dev/video0
0 crw-rw----+ 1 root video 81,  2 30. Jul 15:18 /dev/video1
0 crw-rw----+ 1 root video 81,  4 30. Jul 15:43 /dev/video2
0 crw-rw----+ 1 root video 81,  6 30. Jul 15:43 /dev/video3
0 crw-rw----+ 1 root video 81,  8 30. Jul 15:17 /dev/video4
0 crw-rw----+ 1 root video 81, 10 30. Jul 15:17 /dev/video5
0 crw-rw----+ 1 root video 81, 12 30. Jul 15:17 /dev/video6
0 crw-rw----+ 1 root video 81, 14 30. Jul 15:17 /dev/video7

But i did not, so i need the driver to my techwell card.

Simple, there is a nice person who made a driver already, let’s see how we can install it

First, we need GIT so that we can download the driver

apt-get install git
apt-get install linux-headers-$(uname -r)
apt-get install make sudo

(without installing sudo, the compile gives the error /bin/sh: 1: sudo: not found which is because the guy probably wrote the driver with ubuntu in mind)

git clone git://gitorious.org/tw68/tw68-v2.git
cd tw68-v2
make
make install

So, i was expecting 8 cameras, i got the following

ls -ls /dev/video*
0 crw-rw---T 1 root video 81, 0 May 12 23:12 /dev/video0
0 crw-rw---T 1 root video 81, 2 May 12 23:12 /dev/video1
0 crw-rw---T 1 root video 81, 4 May 12 23:12 /dev/video2
0 crw-rw---T 1 root video 81, 6 May 12 23:12 /dev/video3

probably a driver issue, oh well, lets see what we can do with 4 then i will see about this issue once i have 4 working:D

ln -s /etc/zm/apache.conf /etc/apache2/conf.d/zoneminder.conf

adduser www-data video

/etc/init.d/apache2 restart

Now, visiting 192.168.2.101/zm/ allows me to add cameras, the matter is very intuitive from that point on, all you need is a browser, remember you will need to password protect things too :D so that people from the outside can not look at your cameras.
	

This post is somewhat old, and kept here for historical reasons, if you want to run LXC containers on Debian Bookworm (12), I have composed a much more useful post here

Yes, Wheezy is out to the public, and openVZ is out of Wheezy, so what to do.

Basically, what i am doing now is investigating the alternative LXC, i have no time to learn right now, so i am going to have to do this fast.

I have a gut feeling that LXC is better than openVZ, after all, it is in the mainline kernel, and it is supposed to be marvelously easy to install, so let me start working on this with everyone here.

NOTES: if you want to give away LXC containers to people, you will need to use AppArmor with it, here, i run my containers, so i will not be installing AppArmor in this tutorial, but maybe soon i will add a tutorial for the AppArmor part.

So, LXC here we come, to completely replace openVZ, with something more open (Sorry Parallels Virtuozzo, welcome IBM), something that can keep up with the kernel and not keep us behind.

I will be turning this post into a tutorial on installing and running LXC on debian wheezy (7) with memory allocation to containers and with the kernel that shipped with wheezy, i should be done creating this tutorial in a few days, and it will remain an incremental effort where i will be adding more and more as i learn about this.

NOTES: memory allocation is not compiled with the kernel by default but disabled, you enable it by adding a parameter to grub. (Not anymore, now memory allocation works out of the box)

1- Install base system of wheezy (debian 7)

2- Install some stuff i can never do without

apt-get update

apt-get upgrade

apt-get install ssh openssh-server fail2ban

fail2ban is a very important application that will prevent outsiders from bruit force cracking your server, it is very important, without it you will be hacked sooner or later (especially if you are in a datacenter), hackers look for servers to send spam from all the time.

Now, we need to specify a hostname for this machine (the LXC HOST), i want to call mine server5.example.com

echo server5.example.com > /etc/hostname

/etc/init.d/hostname.sh start

hostname

hostname -f

apt-get install ntp ntpdate

Now, we need to setup networking for LXC, every physical NIC (Network adapter) will need a bridge.

To create a bridge, you need to install

apt-get install bridge-utils

Then your /etc/network/interfaces file must look like this

------------------------------------------------
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
  auto lo
  iface lo inet loopback
# The primary network interface
  #allow-hotplug eth0
  #iface eth0 inet dhcp
#Bridge setup
auto br0
  iface br0 inet static
  bridge_ports eth0
  bridge_fd 0
  address 192.168.2.121
  netmask 255.255.255.0
  gateway 192.168.2.1
  dns-nameservers 8.8.8.8
------------------------------------------------

apt-get install lxc

You will be presented with the following prompt, i myself accept the default /var/lib/lxc

Please specify the directory that will be used to store the Linux Containers. If unsure, use /var/lib/lxc (default). LXC directory:

mkdir /cgroup

Add the following line in /etc/fstab using a text editor:

cgroup /cgroup cgroup defaults 0 0

mount -a

Now, to make sure everything is working like it should

lxc-checkconfig

------------------- OUTPUT OF lxc-checkconfig ----------------START

Kernel config /proc/config.gz not found, looking in other places...
Found kernel config file /boot/config-3.2.0-4-amd64
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled

--- Control groups ---
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

--- Misc ---
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
File capabilities: enabled

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig.

------------------- OUTPUT OF lxc-checkconfig ------------------END

And on the host machine, you need to enable IP forwarding befor you fire up any of those LXC containers

 echo 1 > /proc/sys/net/ipv4/ip_forward

But to make that peppermint you need to edit the file /etc/sysctl.conf where we can add a line containing net.ipv4.ip_forward = 1

/etc/sysctl.conf:

net.ipv4.ip_forward = 1

You might find that the entry is already there but with the value 0, in that case just flip the zero to a 1, or you might find it there but commented out, in that case, delete the # that precedes that line to enable it.

To enable the changes made in sysctl.conf (And you don’t if you already executed the echo 1 statement above) you will need to run the command:

sysctl -p /etc/sysctl.conf

Now that LXC is officially installed, there is more than 1 way to create containers, debootstrap is one of them (you will need to install it, and the container config will need to be done manually by adding a few lines into a file you create inside the container area), while i will use the LXC way by using the application lxc-create you are free to use any tool, including importing containers from vmware (copying vmware containers will work).

Also worth mentioning, i use apt-cacher so when i am asked about the urls of the distro, i simply modify it to read http://192.168.2.133:3142/ftp.us.debian.org/debian/ which is how i accerss apt-cacher to speed up things and not re-download everything every time.

So, lets start

lxc-create -t debian -n vm33

On a newer releast (7.7), the above gave me an error, so the following was the error and the solution (needed command)

MIRROR=http://ftp.us.debian.org/debian lxc-create -n vm10 -t debian -- -r wheezy

Or if you want to use apt-cacher

MIRROR=http://192.168.10.237:3142/ftp.us.debian.org/debian lxc-create -n vm10 -t debian -- -r wheezy

1- Preseed file anyone? Enter (optional) preseed file to use: <== leave this one empty

2- Chose the distro (debian wheezy for me)

3- 64 or 32, i use 64

4-
Archives.

[*] Debian Security

[*] Debian Updates

[*] Debian Backports

[ ] Debian Proposed Updates

5- Mirror.

i modify this to read http://192.168.2.133:3142/ftp.us.debian.org/debian/ in order to use my apt-cacher, you can put any mirror here, or leave the default one (http://ftp.debian.org/debian/ Mirror Security http://security.debian.org/ and Mirror Backports) provided for you. Archive areas Main, Packages (leave blank or specify the packages you want, you can install them later with apt-get), then the root password

You must keep in mind that even after you see the message ‘debian’ template installed ‘vm33’ created, the config file for vm33 is not really ready, you need to enable networking in it manually. so, let’s edit the file /var/lib/lxc/vm33/config and add networking support

vi /var/lib/lxc/vm33/config

NOTE: THE BELOW IS FOR TYPICAL SETUPS, FOR HETZNER DATACENTER, PLEASE SEE THE POST ON LXC NETWORK SETUP WITH HETZNER.

then add the lines right before #Capabilities and after the lines of ## Container

lxc.network.type = veth

lxc.network.flags = up

lxc.network.link = br0

lxc.network.name = eth0

lxc.network.ipv4 = 192.168.2.125/24

Also, before we start the container, there are a few things we need to do…

there seems to be an issue with the ssh keys, so what we will do around this issue is copy the keys from the host, (We will generate new ones for the conatiner later)

EXECUTE ON HOST

cp /etc/ssh/ssh_host_dsa_key /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_dsa_key
cp /etc/ssh/ssh_host_dsa_key.pub /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_dsa_key.pub
cp /etc/ssh/ssh_host_ecdsa_key /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_ecdsa_key
cp /etc/ssh/ssh_host_ecdsa_key.pub /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_ecdsa_key.pub
cp /etc/ssh/ssh_host_rsa_key /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_rsa_key
cp /etc/ssh/ssh_host_rsa_key.pub /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_rsa_key.pub

Then, they won’t work without proper permissions

chmod 0600 /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_dsa_key.pub
chmod 0600 /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_ecdsa_key 
chmod 0600  /var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_rsa_key

Now i reboot the server just to be on the safe side, then i do the following

lxc-start -n vm33 -d

lxc-info -n vm33

When you run the command for information, you should see the word RUNNING and a pid.

Just SSH to the host !

Now if you want to create new host keys for SSH just do the following

delete the files

/var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_dsa_key.pub
/var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_ecdsa_key
/var/lib/lxc/vm33/rootfs/etc/ssh/ssh_host_rsa_key

execute

dpkg-reconfigure openssh-server

—————————————

Making LXC auto start at the system boot
The old Way – create a symbolic link, should still work, but i have not tried

ln -s /var/lib/lxc/vm34/config /etc/lxc/auto/vm34_config

The new way that provides better control of the order they are started in.
Set lxc.start.auto == 1 in the config

Then, the following will tell the system what containers to start first, and when

Update:

BCOMPILER is no longer supported past PHP 5.3, BUT, the same developer went back to developing APC, and now, there are APC functions that can do what bcompiler used to do.
I will come back to explain how to use it later (it works perfectly for me without the original files all together. the trick is 2 functions, apc_bin_load and apc_bin_dump.

The other good option is PHC (of phpcompiler.org).

A third option would be obfusc.com

End of update (2014-05-10)
—————————–

For encoding, there are 3 options

Before i waste your time, what i do is this, i obfuscate the code so that the function names, variable names, and other things are useless and meaningless to the reader, then i use bcompiler. Now here are the choices and programs i used through the years.

Zend Guard (formerly Zend Encoder)- Expensive ($600) – Some say they have cracked it but i have never seen any evidence of that, requires the free Zend Optimizer or Zend Guard Loader to run the encoded code
Ioncube – I have a license for this one, does what zend encoder does, needs it’s loader on the server to work, provides a cheaper encode-online edition for those with small projects, lowest license is for 200 dollars
bcompiler – Free, and probably faster than the other two, it is simply native PHP opcode, meaning you spare PHP the need to compile your code (interpret), making your code run faster, but reversing it is not very hard. to make reversing harder, you need to obfuscate your code (change variable names and function/method names to make the code unreadable, read below for software to obfuscate that are available online.
BENCODER v1.6 – Encode your PHP script using bcompiler – to facilitate using bcompiler

PHP Obfuscators

Here are just a few, if you google PHP obfuscator you will find hundreds 😀

truebug – $45
codeeclipse – free service, obfuscation happens online on the website
phpprotect.info – Simple free tool that obfuscates variable names, which is enough most of the time, it is very hard to read code with non descriptive variable names
iddin.sourceforge.net – all in German, but translating reveals that it is an obfuscator

I usually use OSFmount for mounting files made with DD, ddrescue, dd_rescue and gddrescue, for virtual machines in vmware i can mount those directly with OSFMount as long as it is a single file disk, when the thing has multiple files, i can not do that

Turns out vmware gives a way a tool to consolidate those disks into 1 disk.

cd C:Program Files (x86)VMwareVMware Workstation
vmware-vdiskmanager -r c:my_in_file.vmdk -t 0 file_out.vmdk

Now i can mount them directly onto my host machine.

Note that, even the  http://www.vmware.com/support/developer/vddk/ (The vmware vmdk development kit) which is a recent tool can not mount mutiple split disks, and requieres a single disk to mount on host

First, calculating memory

I want 1GB of ram, how many pages would those be ?
To calculate the numebrs for an openvz conf file, you should know that in container config files, the numbers there are pages, not bytes, you should probably know that a page of memory is 4k which is 4096 bytes

So a gigabyte is 1073741824 bytes

Number of pages = 1073741824 / 4096

So a gigabyte is 262144 pages (which open VZ should get in the config file to end up with a gigabyte of ram).

Now To what values in the openvz config file mean.

So, a memory block in oppenvz is measured by pages not bytes or megabytes, so the word pages generally means memory, now let us look at some variables in the openvz config file, vm means virtual machine (which is your container)

Values are specified in the form of
parameter=”barrier:limit”
For example
VMGUARPAGES=”67584:2147483647″

vmguarpages = VM Guaranteed Pages, the barrier is the effective number (The limit should be set to maximum integer, 2147483647 on 32bit and 9223372036854775807 on 64bit even though it is not defined or useful yet), If the current amount of allocated memory space does not exceed the guaranteed amount (the barrier of vmguarpages), memory allocations of container’s applications always succeed. If the current amount of allocated memory space exceeds the guarantee but below the barrier of privvmpages, allocations may or may not succeed, depending on the total amount of available memory in the system. if allocations exceed privvmpages, allocations will fail.
oomguarpages = Out Of Memory Guaranteed pages (Total for all machines should not exceed the available memory on the server) this is like the above, but assuming the system has no choice but to kill procesees, the system will weigh this in to chose what VM to kill from.
privvmpages = Private VM pages (The barrier and the limit of privvmpages parameter impose an upper limit on the memory allocations)

So in my Window => preferences window, i searched for “content assist”, and under content assist for PHP, i made the value 50 rather than 200, this did not help at all as there was no change.

So i searched for text editors and enrich after delay to enrich immediately, and magically everything worked on PDT, this does not make sense, what does hovering have to do with content assist, but there you are, now as soon as i write anything, choices star appearing 😀

Again, Switching from Zend Studio (paid) to PDT (Eclipse) which is ironically made by Zend as well was a very good move, i feel no issues whatsoever, i think this one is even better than the old version of Zend studio i was using, and Zend studio offered me no advantage at all, even though it would probably be beneficial to some who know hot to use the zend tools and get the studio to compliment those tools.