QWORQS

Just because this took a lot of research from me (Looking at specs), I am posting this here for my own reference.

Note: CF cards will work on both 3.3V and 5V. so there is a bit of a mystery on why we need a voltage step down chip and a jumper for this particular adapter, other drives are clearly incompatible IDE interfaces.

it turns out that when combined with a converter using the fc1307a chip to create CF out of 2 SD cards, it will not work without a voltage step down, with the wrong setting, or with cards with no step down, a computer will see the combined cards, but will fail when trying to format them for some reason.

The 1.8″ adapter (Sideways IDE interface) is 3.3V, the normal 2.5″ disks are 5 Volt, the adapter we have that turns CF cards into 1.8″ or 2.5″ compatible drive, has a JUMPER to select, this jumper effectivly activates or deactivates the voltage step down chip (The only chip on the board)

So there you have it, 1.8″ and 2.5″ PATA drives are not compatible regardless of the fact that the PIN interfaces are the same. the adapters with a voltage step down will work on both, but you need to set them up first with a JUMPER

For those who want a real lowdown right away, here is the final verdict, as of the time of writing this, there are no faster options, the limitations here are imposed by the creators of PI and it’s chips (The Broadcom GPU)

You have 2 options for capturing stills in burst mode. In the case i am addressing here, taking photos of rapidly moving objects in broad daylight, so the shutter speed is very fast, the ISO is not so high (Because of the sun).

1- Through the GPUs stills part of the processor, you would get less than 2 frames per second, you can use the timelapse function.

2- Through the video port on the GPU, this will allow a maximum of 15 frames per second at 5MP resolution, no binning, no sensor cropping, but the still port gives somewhat better photo quality

Considerations

The speed of storage counts, if your storage becomes a bottleneck, 15FPS will not be possible.

1- You can use faster USB storage, the USB port is bound to a maximum of 45MB/Second

2- A fast memory card, currently 25MB/s comes on expensive “Professional” SD cards, i would rather use the USB port

3- Splitting the bandwidth between storage and Ethernet is not a good idea, the Ethernet adapter on the raspberry PI is connected through USB, so it will share the USB bandwidth 😉

4- Splitting the storage or buffering some of the frames on the SD card can be a good idea, as the SD card is connected directly to the CPU not through USB

5- Lowering the photo quality through increased performance will give you good mileage, something equivalent to what other programs call 80% quality will reduce the size pretty well without sacrificing much.

How to get through the stills port

raspistill -o %04d.jpg -ss 5000 -ISO 300 -t 10000 -tl 100 -q 10 -th none

So, the questions that come to mind instantly are, how come quality is a value from 0 to 100 and we are using 10 ? it is a good question, and this is something that is not making sense i know, but judging with my eyes, -q 10 will give us good compression ratio and the photo is still good, the range between 10 and 100 does not make much difference, and then under 10, things go very bad rapidly, no idea why

Now to the next method, which is getting photos from the video port… here is a quick and dirty picamera python script

If you don’t know how python works, please don’t change the indentation, indenting the print statement will add the print to the previous code block which is something we don’t want.

#!/usr/bin/python3
import time
import picamera

frames = 60

with picamera.PiCamera() as camera:
    camera.resolution = (2592, 1944)
    camera.shutter_speed = 5000
    camera.iso = 100
    camera.framerate = 10
    camera.start_preview()
    time.sleep(2)
    start = time.time()
    camera.capture_sequence([
        '/usbstick/image%02d.jpg' % i
        for i in range(frames)
        ], use_video_port=True)
    finish = time.time()
print('Captured %d frames at %.2ffps' % (
    frames,
    frames / (finish - start)))

So simply put, if you are running raspbian, you need to get into the pi config (command is raspi-config), and enable the camera then reboot

Also, don’t forget to run “rpi-update” or apt-get upgrade

Now if you try to take a photo you would enter the command

raspistill -o cam.jpg

But this will take a long time to take one photo (5 seconds by default), this is because it needs to set the exposure, if you want to give it less time you could run the command

raspistill -t 1500 -o test.jpg (One and a half seconds)

But that’s is too long, i already know that it is broad daylight, and that i need a very fast shutter speed, So how do i manually set things ?

Taking a photo of fast moving things in the SUN, i would run the command as follows

raspistill -t 1 -ss 1000 –awb sun -o test1.jpg

Taking photos of moving things in cloudy weather, i would run the command as

So, here are the manual things

–ss 1000 (1 ms, good for a sunny day oudoors, 4ms is good on a cloudy day, with dim light, i would recommend bumping this up a bit)

–ISO 300 (Sensitivity, Back in the day, the film dictated this property), the faster the shutter (Lower value), the higher the needed sensitivity (ISO VALUE), the range is 100 to 800

–nopreview

Prsets for Automatic white balance (AWB)

–awb sun

APC is gone, no longer maintained, now there are alternatives.

For opcode cache, PHP 5.5 (5.6 shipps with jessie) the opcode cacher ( OPcache ) module is installed and enabled by default, as for key cache (Persistent across pages), we have an alternative called APCu (Just the user key value cache), once installed, the apc_ functions return to PHP so it is a drop in replacement, no no program modifications needed.

To install APCu, you run the following

1- Install the tools
apt-get install apache2-threaded-dev php5-dev php-pear make
2- Before you get APCu-4.0.7, you should check what the latest version is !
pecl install channel://pecl.php.net/APCu-4.0.7

Now you are done, all you need to do is add
extension=apcu.so
to the php config file, in my case, what i do on debian jessie is add it in a file here
/etc/php5/apache2/conf.d/apcu.ini

There you have it, you are back on track.

Over provisioning a Samsung 1TB 850 EVO

Mind you, Don’t follow this tutorial step by step unless you have a 1TB Samsung 850 EVO, if you have a smaller disk, you need to adapt the numbers to your SSD 😉

Over provisioning a flash disk is simply some un-partitioned space a the end of the disk, but you need to tell the SSD’s controller about that free space that it can use to do it’s housekeeping, You also need to find out if the Tejun Heo’s on-demand HPA unlocking patch applies to your distro, if it does, you need to get kernel patching first.

First of all, the controller will usually use the cache RAM to so the over provisioning, or at least this is what i understood from some text on the Samsung website, you can make things faster by allowing it to use FLASH space while it deletes a 1.5MB flash area to put the data in.

1- How big should the over provisioning area be ?

Samsung recommends 10% of the disk’s space. Somewhere hidden in a PDF on their websites, they explain that OP space should be anywhere between 7% and 50% ! we will use 10 as our writing patterns are not that harsh. but mind you, a database that alters a few rows every second can probably make the most use of such OP space.

2- Won’t that 10% wear out before the rest ?

No, there is a mapping function inside the controller where that space is in fact wherever the controller thinks is appropriate, the wear leveling algorithm kicks in at a stage after the logical stage of partitions etc… it is blind to the file system or the over provisioning area, it will simply remap any address you give it to a random address that is not already mapped, at flash erase, those mappings are deleted, and other areas of the disk will be assigned to that area, i have no idea whether it uses a random algorithm, or simply has a record of flash chip usage (At the size of the sample, that won’t make any difference.)

3- Are you sure we are informing the controller and not just telling Linux what the last address is ?

Sure I’m sure, ask the controller DIRECTLY yourself with the command

smartctl -i /dev/sdb

Before the operation we are doing in this article, it will say 1000204886016, and after it will say

User Capacity:    900,184,411,136 bytes [900 GB]

Meaning that now, the disk’s S.M.A.R.T. attribute tells us that this much is available for the user after the over provisioning operation

So, how do we over provision in linux

See the last secrot of your ssd,

hdparm -N /dev/sdb

In my case, my samsung 850 EVO has the following, notice that the number is repeated twice. x out of x is the same, and HPA is disabled..

max sectors = 1953525168/1953525168, HPA is disabled

Now, 1953525168 * 512 = 1,000,204,886,016 (1 TB !)

Now, we want to set a maximum address, anything after this address is a PROTECTED AREA, that the controller knows about, I will multiply the number above with 0.9 to get the maximum address, take the integer part alone

hdparm -Np1758172678 --yes-i-know-what-i-am-doing /dev/sdb (As hdparm -Np1758172678 /dev/sdb will ask you if you know what you are doing)

 setting max visible sectors to 1758172678 (permanent)
 max sectors   = 1758172678/1953525168, HPA is enabled

Now again, hdparm -N /dev/sdb

max sectors = 1758172678/1953525168, HPA is enabled

Now, to make sure we are not suffering that dreaded bug, let’s reboot the system and check again after that, I am using debian Jessie, so it is unlikely that i am affected

Yup, hdparm -N /dev/sdb still gives us a smaller maximum address than the actual physical

Now, we seem to be ready to talk fdisk business.

fdisk /dev/sdb

Now, if you O (Clean), then P, you should get a line such as

Disk /dev/sdb: 838.4 GiB, 900184411136 bytes, 1758172678 sectors

This means that FDISK understands.and asking it to create (the n command) will yeild this

/dev/sdb1 2048 1758172677 1758170630 838.4G 83 Linux

Arent we happy people.

Now, lets mount with trim support, and enjoy all the beutiful abilities an SSD will bless us with.

tune2fs -o journal_data_writeback /dev/sdb1
tune2fs -O ^has_journal /dev/sdb1

NOTE: in the event that you are presented with an error such as the following

/dev/sde:
 setting max visible sectors to 850182933 (permanent)
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 04 51 40 01 21 04 00 00 a0 14 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 max sectors   = 1000215216/1000215216, HPA is disabled

The most likely cause is the SATA controller (Try executing the hdparm -Np command using a different SATA controller), another possible cause of errors is that some disks require being trimmed before this action

One little problem about modern VGA cards is HEAT, they consume over 30W on IDLE, those 30 watts are going into the case, so i looked into my old computers, and found a computer that dates back to 1995-1996, I pulled out the VGA card from it, and installed it on a modern I3 computer for testing pending the installation on an I7 with 64GB of ram and what have you.

On ebay, you can find such PCI cards for around $10, Cirrus Logic, SIS, ATI, OR S3, they should all work, if the promotion card works, those should work too.

Now i ran the Debian Jessie installer, the installation went fine, when rebooting, the system boots with the PCI card, but then switches to the embedded graphics system (Comes with the I3 CPU), the BIOS does not allow me to disable that, so, rather than looking for a solution, I will test the adapter on an I7 (Does not come with built in VGA).

I have a good feeling that it will work right away, here is some information about my 20 year old graphics card (Will post some photos too when i plug it out)

    Made by: Alliance
    Codename: ProMotion 6410
    Bus: PCI
    Memory Size: 1MB
    Max Memory Size: 4MB
    Memory Type: FPM
    Year: 1995
    Card Type: VGA
    Made in: USA
    Owned by: Palcal
    Outputs: 15 pin D-sub
    Power consumption (W): 1.5
    Video Acceleration: MPEG-1 (VCD)
    Core: 64bit
    Memory Bandwidth (MB/s): 213
    Sold by: miro
    Press info: Freelibrary

You can find

It is simple, but i am issuing no guarentees, this will probably work for you, but there is a chance that something could go wrong, you have been warned

Loosing network connectivity: One thing that sometimes happens is that the Ethernet number can change from eth0 to eth1 for example (this could happen if your system has not been exposed to other cards, in which case it can turn from eth0 to eth6), What you can do about this is either be physically present to fix it (fix in /etc/network/interfaces) or add another interface to the file so that it can work out of the box, or have KVM over IP, or something of the sort, if you have LXC installed, the containers will not be able to fire up because they must have the following two extra lines in their config, /var/lib/lxc/CONTAINER_NAME/config

lxc.autodev = 1
lxc.kmsg = 0

apt-get update
apt-get dist-upgrade

edit your apt sources (vi /etc/apt/sources.list)

replace the word wheezy with jessie (Wherever you find it)

apt-get update
apt-get dist-upgrade

You should be done

Using enhanceio (the flashcache fork)

First, on a debian system, you need to compile enhanceio because debian have not yet released anything for it (2015-05-13)

apt-get install git make gcc pkg-config uuid openssl util-linux uuid-dev libblkid-dev python
apt-get install build-essential
apt-get install linux-headers-$(uname -r)

Now let us download enhanceio

git clone https://github.com/stec-inc/EnhanceIO.git
cd EnhanceIO/Driver/enhanceio/
make && make install

On make install you will see
make[1]: Leaving directory '/usr/src/linux-headers-3.16.0-4-amd64'
install -o root -g root -m 0755 -d /lib/modules/3.16.0-4-amd64/extra/enhanceio/
install -o root -g root -m 0755 enhanceio.ko /lib/modules/3.16.0-4-amd64/extra/enhanceio/
install -o root -g root -m 0755 enhanceio_rand.ko /lib/modules/3.16.0-4-amd64/extra/enhanceio/
install -o root -g root -m 0755 enhanceio_fifo.ko /lib/modules/3.16.0-4-amd64/extra/enhanceio/
install -o root -g root -m 0755 enhanceio_lru.ko /lib/modules/3.16.0-4-amd64/extra/enhanceio/

cd /lib/modules/3.16.0-4-amd64/extra/enhanceio/
insmod enhanceio.ko
insmod enhanceio_fifo.ko
insmod enhanceio_lru.ko

#Now check that it is loaded as a kernel module with (HINT: Will it be there after reboot ?)
lsmod | grep enhanceio

cd ../../CLI/
cp eio_cli /sbin/
cp eio_cli.8 /usr/share/man/man8

Now that we have installed and compiled enhanceio, we will simply use it, we are all done, unlike bcache and dmcache, there is minimal setup for this kind of ssd/flash thumb stick.

Now in my setup, i want sdb to act as an SSD cache for sdc1

eio_cli create -d /dev/sdc1 -s /dev/sdb1 -p lru -m ro -c main_disk_cache

The output of that command was as follows, this result can also be obtained with the command “eio_cli info”, or for super detail “cat /proc/enhanceio/main_disk_cache/stats”

Cache Name       : main_disk_cache
Source Device    : /dev/sdc1
SSD Device       : /dev/sdb1
Policy           : lru
Mode             : Read Only
Block Size       : 4096
Associativity    : 256
ENV{ID_SERIAL}=="WDC_WD1001FALS-00J7B0_WD-WMATV0098355", ATTR{partition}=="1"
ENV{ID_SERIAL}=="INTEL_SSDSA2CW120G3_CVPR1481061P120LGN", ATTR{partition}=="1"
Cache created successfully

Now to see the block size
blockdev –getbsz /dev/sdb1 (512 was the result)

The controller on this SSD is the jmf612, along with the RAM chip (Cache) hynix H5PS1G63EFR (SDRAM) (64m X 16BIT) = 1Gbit which is 128MB (128 MBYTE).

This chip does not suffer from the latency problem in the JMF601A/602A chips, (jmf612 is 2009 while JMF601A is 2006).

The memory that comes with the drive is made by intel (29F32G08AAMDB), 4GB per chip, With 16 such chips, we have 64GB of flash on the board, the photos will follow once i get them off my camera and onto this blog.

The reason why i am keeping record here is that this drive is probably going to be used as a bcache cache device, and i am concerned about reliability, Granted that it is 90% read scenario, some websites are claiming that data corruption is a downside, but not providing any elaboration, http://alatest.co.uk/../corsair-reactor-r60-jmicron-jmf612-ssd/po3-116271270,383/ (The crosair reactor-r60 is identical to this drive)

The drive is claimed to have speeds of read 240 write 170, compression is not a factor in this as other drives use compression to gain speed, something that is only good for certain types of data

There is also a new firmware at the adata website that you should make sure is installed, ADATA S596 Firmware Update Tool V1.0

NOTES:
Drives with this controller include

• Patriot Zephyr SSD
• Corsair Reactor-r60
• Active Media Predator-X7
• Western Digital SiliconEdge Blue
• Kingston V-series

Please note, i am keeping this thread for myself as a reminder, so if sometimes you feel that it is getting too general, it is probably more of a reminder for me.

Also note that i am organizing this post as i go as i spend some time creating my new MySQL machine every day, so this is an ACTIVE work in progress

—————————————————
First note, the benefits of RAID 0 on the SSDs is more about wear than it is about speed.
Here is why and how

1- All modern disks use wear leveling to prolong the life of your disk, bcache does nothing in this regard, so using thumb drives is not very smart.
2- Your SSD has to be larger than the active data set, otherwise, bcache or any other will start kicking out data and writing in it’s place too often, wearing your disk, yet, if the data that is always accessed on your hard drive is about 500GBs of your 3TB disk, and your cache drive is more than 500GBs, odds are that your disk will probably send this data, then spin down because everything we need is already on the SSD reducing the wear to almost nothing. If you do not have a large enough SSD, odds are you will save their life with a RAID setup, either software or hardware, it makes no difference.
3- Larger disks have longer lives in general with or without raid for all purposes, the reason is arithmetically simple, the larger the disk, the linger it takes for a flash chip to get it’s turn for a write again because the others need to get their turn first, the larger the disk, the more the banks, hence, the longer the useful life.
4- Erase commands are expensive, And even with a workaround, trim support means nothing on a busy server
—————————————————
My own setup

1- My computer has 6 sata ports (ASUS P9X79).
2- I prefer to never group spinning hard drives in one metal case that does not have vibration isolation, so the mainboard SATA will host the 5 SSDs, and the main hard drive (2TB black).
3- I have 2 PCIe 2 port esata cards (silicon image supporting port multiplier)
4- 1 SATA port multiplier (Silicon image) that even though has raid function, is probably more reliable in JBOD mode, where the 5 spinning disks other than the main disk will need to live.

Most data on the 6 spinning disks is only stored and seldom accessed. furthermore, most of this data does not get cached because it is accessed sequentially.

The boot partition of the hard drive (Spinning) is not cached, the second partition is, this is safer as the system can boot anyways, then the raid array and the bcache initialized with no unforeseen effects, boot time is not so important as the PC is always on, we reboot it once a year

—————————————————
Note for self:
1- What are the possibilities of 2 cache layers, one with write-back, the other is only for read cache.
Possible Advantages:
You can use less reliable flash thumb drives with no wear leveling mechanism as an extra cache layer, where failure will not cause data corruption, but will leave more space for the SSDs
Overcache problem considerations ?
The SSD will initially get the same cache as the thumb drive, but when the hit rate on the thumb drive increases, the SSD will remove that data, so we will only have 1 copy
The SSD with write-back can make writing a hell lot faster for database data.
Bcache only allows one SSD per spinning partition. you can not combine, while the other methods could ! and since they are read only, you could even RAID the thumb drives
Caching software available
Device mapping
bcache (part of the new kernels)
dm-cache (part of the new kernels)
Flashcache (Facebook’s)
EnhanceIO (Fork of flashcache)

EnhanceIO can be attached to any block device on-the-fly even
when device is already mounted.
dm-cache is faster, but bcache is safer

If so…
we will probably be using bcache in FIFO mode, and enhanceio in most (least recently used (LRU)).

Also take a look at https://github.com/Feh/nocache
—————————————————
You probably noticed that there are not many tutorials and instructions on bcache for Debian online, the reason is that the user tools are not packaged for Debian, not even for Jessie, but that is a simple thing, the hard part has already been done at the kernel level so you dont need to recompile the kernel, but you need the instructions on how to setup bcache !

So here is a step by step, no theory and no explinations, just the procedure

bcache, the simple and safer way

So here is how i setup bcache on my linux server for MySQL usage.

1- create 2 partitions on my 2TB disk, one would be 200GB and the other 1800GB (With fdisk or parted you chose)

2- Delete the first 200GB partition leaving only the other

3- Run the debian installer and instructing it to use the continuous free space, which is now the small first partition

apt-get install git make gcc pkg-config uuid openssl util-linux uuid-dev libblkid-dev

git clone https://github.com/g2p/bcache-tools.git

cd bcache-tools

make

make install

-------------------------------------
install -m0755 make-bcache bcache-super-show /usr/sbin/
install -m0755 probe-bcache bcache-register /lib/udev/
install -m0644 69-bcache.rules /lib/udev/rules.d/
install -m0644 -- *.8 /usr/share/man/man8/
install -D -m0755 initramfs/hook /usr/share/initramfs-tools/hooks/bcache
install -D -m0755 initcpio/install /usr/lib/initcpio/install/bcache
install -D -m0755 dracut/module-setup.sh /lib/dracut/modules.d/90bcache/module-setup.sh

-----------------------------------------------------

Now, use wipefs to delete the filesystem on the big partition (/dev/sda2) so that we can format it as bcache rather than ext4

wipefs -a /dev/sda2

Take note of the UUIDs that come out of thie following 2 commands.

Why ? well, caching devices are the SSDs, backing devices are the normal sata disks, what happens when you run the following 2 commands is that you tell the kernell that one is a backing device, and the other is a caching device, you can have as many backing and caching devices as you want, but pay attention to this, you can not use 2 SSDs to cache one disk, although you can use 1 SSD to cache multiple disks, so for example, you can have 2 SSDs caching 5 disks, 1 SSD can cache 3 disks for example and the other can cache 2 disks.

Also, this is debian, so this step is done for you automatically, you will have /sys/fs/bcache/somediskuuid off the bat, that does not apply to all linux systems (some systems might requiere you to do the following to the backing deice ... echo "the bcache uuid" > /sys/fs/register)

First step, initialize my 1.8TB partition into a bcache block, -B means this is a backed device

make-bcache -B /dev/sda5

mkfs.ext4 /dev/bcache0

Then initialize the SSD (120GB intel SSD) into a bcache block, -C means caching device

make-bcache -C /dev/sdb1

--------------------

UUID: 4e9aed54-bf48-4d7a-b5b2-b041f2a811f8
Set UUID: 13544b4e-99de-42a0-905a-c6efbe669151
version: 0
nbuckets: 228944
block_size: 1
bucket_size: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1

----------------------------

Now we have a backing and caching device, let's connect them together, this step is called "attaching"

first,

cd /sys/fs/bcache/

ls

the ls command will give you the caching devices UUID as a folder, which is actually a folder.

Now, Mount the bcache partition.

mount /dev/bcache0 /hds/cacheddisk (Check that you have bcache0 not bcache1 for example).

A friend of mine had a problem with a MacBook Pro, the problem was that it would not turn on, so i took a look at it and here is what i came up with

The battery was completely dead, effectively disabling the SMC (System Management Controller).

Without the SMC, the system will refuse to boot unless you bypass the SMC.

To bypass the SMS and make the computer run, you do the following

1- Disconnect the power cable.

2- Hold down the power button for 10 seconds (or more), then while still holding the power button down, connect the power cable and keep the power button pressed down for another 10 seconds (or more), then once the 20 seconds are over (10 without cable and 10 with cable), release the button and press it again like you normally would to start the computer.

Your fans will run at full speed (resulting in a loud computer), it will be slow, and it might crash or restart itself, in short things will not be working fine, what you need is a new battery.

------------------------------------------------------------
Stuff that only applies to my friend, so skip this part
------------------------------------------------------------
Here are the specs of my friends laptop.



When you want to look for a battery online, you should be looking for

Macbook Pro A1286 Early 2011 battery (Does not apply to all readers).

I did a search, but knowing that you will be taking it to a service person to install it means that a bad choice on my end (even with a money back guarantee) could mean paying twice to the repair person, so it is not as simple as it seems for me (Since you won't be installing it yourself).

The original battery (OEM) costs $110 it seems, batteries from other providers seem to cost anywhere from $25 to $70, with the $25 batteries having super bad reviews, and the ones for $70 being ok it seems. still, the complications (having to take it to someone to install it) leave me confused. so a safe bet would be the OEM (Original apple battery) but then that's more than double the price
------------------------------------------------------------

Once you install the new battery, you will need to reset the SMC, the repair shop will probably do the reset for you, but just in case.

Shut down the computer.
Plug in the power adapter.
On the built-in keyboard, press the Shift, Control, and Option keys on the left side and the power button, all at the same time.
Release all three keys at the same time.
Turn on the computer.

You are done