Shrinking a disk partition under Debian 11 bullseye

As usual, I will start by getting to the bottom of it, then explain everything

first, you need to first shrink the file system, then the partition where the filesystem resides, replace /dev/sda4 with whatever you partition is named

1- Shrinking the filesystem

Unmount the partition to be resized,

umount /mountpoint

otherwise you will get a message such as

Filesystem at /dev/sda4 is mounted on /mountpoint; on-line resizing required
On-line shrinking from 30453104 to 98098 not supported.

The following commands are relevant to the program resize2fs, they are hands on examples of use, take a close look at the description of what each does before you proceed by picking how you want to use the command.

* Show the minimum size we can squeeze this partition to without losing data
resize2fs -P /dev/sda4
* do the filesystem resize to the MINIMUM possible size (the number you ended up with in the previous command)
resize2fs -M /dev/sda4

The command above moves all data to the beginning of the filesystem/drive, then shrinks it to the smallest possible size.

2- Shrinking the partition

2.1- Find the boundaries of the file system with fdisk

3- You are DONE

If this is it, why is there much more in this tutorial, Simply put, what is above does very little explaining, if you want to understand what we did, you will need a bit more

the assumption, I have a partition that only has 5% data, I would like to shrink the partition to ten percent of it’s size.

Unlike windows, where your luck of where the data resides, you can always shrink a Linux partition to whatever size fits the data that is on it (without losing data)

in this tutorial, I will assume the partition is /dev/sda4, you will need to replace that with whatever your partition is.

1- collecting information about our partition

fdisk /dev/sda
then the p command for print

df -h
this should show you all the partitions, info about them and where they are mounted and how much space is used

the file system can be shrunk with resize2fs

the command “resize2fs -M” should first move the data to the beginning of the drive, then shrink it

first, how large is the file system ATM
tune2fs -l /dev/sda2 then multiply by block size

New firmware for my Western Digital “My Book Live” NAS storage device

The WD My Book Live is a NAS device based on Debian Linux, Since Debian stopped supporting this processor (APM82181), the device has received no updates and will probably never, so the next best thing to do in my opinion is to install openWRT.

WARNING: I recently got a second MyBook Live device, tried installing 23.05.0 but for some reason, i could not get networking to work, So i simply installed V21, then upgraded to 23… there was probably just something I was missing, but i could not be bothered figuring it out, this was a faster way…

Before you start

1- Only the first few paragraphs of this tutorial (STEPS 1 THROUGH 6) are the instructions you need, the remaining is just for extra reference and in short you don’t need to read it to have your device running, but I do recommend YOU SKIM THE WHOLE THING BEFORE YOU START.
2- This procedure requires you to take the disk out and install it on a PC to switch the firmware, then put it back
3- The upgrade will delete all your data, You will need to move your data that is already on your WD NAS drive somewhere else while the upgrade is ready.

Step 1: Move any existing data BEFORE TAKING APART.

Move any data you may have on the drive to a temporary location outside the NAS drive. this has to be done before taking the drive apart as the unconventional 64 kB block size of the disk will be nothing but trouble if you want to extract the data while mounting the disk to a linux PC for example.

Step 2: Take the disk apart

I have included photos to help you do that, it is not rocket science.

Step 3: Mount the disk on a linux PC (Windows and MAC should work)

and mount it to a linux PC (Windows might work with software such as etcher, but i have no guarantees).

Step 4: Download the openWRT firmware

Go to the drive’s page on the openwrt website (Here), and download it to your Linux (Or windows) PC

Step 5: Write the firmware to the disk.

Decompress the file, then copy it to the drive with a command similar to the command below, but make 100% sure to replace sdx with your own drive designation

 dd if=/root/wdsata.img of=/dev/sdx bs=64k

Write the firmware to the disk, overwriting it, and effectively loosing any data you did not backup in step 1

Step 6: Put the drive back in the enclosure

Nothing to say here, this is the reverse of step 2

Once it is in the enclosure, you can not just connect it to your router as it in itself has this port defined as 192.168.1.1 and is serving dhcp !

Step 8: Create the data partition

At this stage, your device will boot, but you will need to create/expand the data partition, the partition that should not be overwritten when you upgrade the firmware for example.

opkg update
opkg install gdisk blkid openssh-sftp-server block-mount
gdisk -i /dev/sda

As soon as gdisk opens, you may be presented with the following message, if so

Found valid MBR and corrupt GPT. Which do you want to use? (Using the
GPT MAY permit recovery of GPT data.)
 1 - MBR
 2 - GPT
 3 - Create blank GPT

Chose 1 to maintain the 2 partitions we have, Now hit the command (w) to write and confirm, then quit, gdisk has just switched your disk to GPT from MBR, now run gdisk again the same way (gdisk -i /dev/sda)

n for new partition, accept the (3) for partition number, use the number (2097152) for alignment with 4K sector advanced format nearest to the 1GB mark

mkfs.ext4 /dev/sda3
mkdir /share
blkid /dev/sda3

You might find a file named fstab in /etc, this is not the file that needs to be edited, the one you are seeking is in /etc/config/fstab in my case, the UUID was as follows UUID=”9643bd00-f117-4074-a252-7ea30a5174e2″ yours will certainly be different, so in my fstab i added the following lines near the end

config mount
option target '/share'
option uuid '9643bd00-f117-4074-a252-7ea30a5174e2'
option enabled '1'

Now, network sharing is what i was originally interested in when i got this unit, and it is why I am replacing it’s firmware, so to installing samba

opkg update && opkg install samba4-server luci-app-samba4

Now, add the following line to /etc/passwd to add me as a user to the system

yazeed:*:1000:65534:yazeed:/var:/bin/false

Or, if you do not want to add the user manually, you can install the adduser package, and add the users through it like so

opkg install shadow-useradd
useradd yazeed
Unfortunately, this command won't do and you will have to edit it in the passwords file

Now, for either method from the above, run the command

passwd yazeed
smbpasswd -a yazeed

Now, since this is a NAS device, disk tools may be a good idea

opkg install hd-idle luci-app-hd-idle hdparm

To check if disk is spinning, try the command
hdparm -C /dev/sda
The responce active/idle means it is spinning

You are done.

FAQ

Is the hardware and the new openWRT firmware compatible with my 8TB hard drive

Yes it is, I have found many people asking if the hardware supports drives over 2TB, the answer is yes, but you will have to use the GPT rather than the MBR (See steps above)

about the original firmware

What is that vulnerability about

it comes from WDs cloud service, bottom line is that many devices were completely wiped remotely by malicious users and it is unknown if the data itself leaked, so yes, it is very serious

What is the difference between quick factory restore and full factory restore

Quick factory restore is probably what you are looking for, the later seems to do a zero fill on the hard drive after performing a factory restore to disallow data retrieval (For example before you sell it), you can verify this by logging in using SSH, and by the fact that the tool tips state something to that effect.

Inspecting the device

To begin with, I logged in via SSH and inspected some stuff, to enable SSH access on the My Book Live original firmware, you will need to visit a page at a URL such as http://mybooklive/UI/ssh or http://192.168.2.116/UI/ssh (Replace the IP with your own)

the system is based on the following CPU

CPU
processor       : 0
cpu             : APM82181
clock           : 800.000008MHz
revision        : 28.130 (pvr 12c4 1c82)
bogomips        : 1600.00
timebase        : 800000008
platform        : PowerPC 44x Platform
model           : amcc,apollo3g
Memory          : 256 MB

With that out of the way, A look at /etc/apt/sources.list revealed that it is a Debian Distro, the only problem with this is that debian stopped supporting this CPU some time ago, so you can’t go past Debian 8 (Jessie)

deb http://ftp.us.debian.org/debian/ squeeze main
deb http://ftp.us.debian.org/debian/ wheezy main
#deb-src http://ftp.us.debian.org/debian/ wheezy main
#deb http://ftp.us.debian.org/debian/ sid main

Checking the disk info with hdparm revealed that the disk is a WDC WD20EARX-00PASB0, which is as i expected a Caviar Green (SMR disk)

parted (The new fdisk so to speak) shows the following partition scheme for the existing system.

Model: ATA WDC WD20EARX-00P (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system     Name     Flags
 3      15.7MB  528MB   513MB   linux-swap(v1)  primary
 1      528MB   2576MB  2048MB  ext3            primary  raid
 2      2576MB  4624MB  2048MB  ext3            primary  raid
 4      4624MB  2000GB  1996GB  ext4            primary

And a “df -h” reveals

Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              1.9G  555M  1.3G  31% /
tmpfs                 5.0M     0  5.0M   0% /lib/init/rw
udev                   10M  6.7M  3.4M  67% /dev
tmpfs                 5.0M     0  5.0M   0% /dev/shm
tmpfs                 100M  4.6M   96M   5% /tmp
ramlog-tmpfs           20M  4.5M   16M  23% /var/log
/dev/sda4             1.9T  2.1G  1.9T   1% /DataVolume

A good alternative for this Gigabit Lan network attached storage might be openWRT, the same firmware I use for my routers !

there are things you need to know in advance though, first of which is that changing the firmware will require you to delete everything on the drive ! as Western Digital have used an unconventional bunch of things such as a 64 kB block size !

With that out of the way, you can skip down to the installing openWRT about the upgrade process step by step (Including backing up your system), then come back to why etc…

What if i want to revert back to the WD software ?

That is indeed a good question, and to make it easy to do that, I have already backed up the entire disk to another while I am sure that i don’t want to go back. Also worth mentioning that the latest firmware on the WD website dates back to 2015 ! which is at the time of writing 6 years ago !

Where can i find the up to date openWRT distribution for this drive ?

OpenWRT has a page dedicated to this drive, both the single and the Duo here (https://openwrt.org/toh/western_digital/mybooklive)

What are the benefits of the NAS box (enclosure), why not just take out the hard drive and put it in a PC somewhere.

The Western Digital My Book Live has a super low power CPU, and when the disk is spun down, it consumes very little energy (Not a significant load to your UPS for example), It is also fan-less, so it is with the exception of the spinning drive when it is spinning silent, which is also a nice thing, So i would argue that keeping it by updating it’s software is a good idea

Another reason is the amount of relevant software provided through openWRT packages, covering many more things than the original firmware (miniDLNA included).

Errors and resolution

1- I have this error that i have not resolved yet

mv: setting attribute 'user.DOSATTRIB' for 'user.DOSATTRIB': Permission denied

2- The NAS box will not accept many files that windows creates such as Thumbs.db, to allow such files to be stored, This can easily be resolved by editing the samba template and commenting the “veto” files line, then make sure the config is regenerated from the template

How do i keep the system up to date

If you come from a debian background, you would normally apt-get update then apt-get upgrade and that is that, in OpenWRT, there is no such upgrade command, the upgrade command in openWRT is meant to upgrade 1 package specified by name, so the solution is the following line

 opkg list-upgradable | cut -f 1 -d ' ' | xargs -r opkg upgrade

Forcing color change for visited links

Do you remember the days when your browser would always tell you what links you have followed before ?

For a very long time now, CSS has allowed webmasters to chose the same color for the link that has been visited as the link you have never visited, favoring aesthetics over functionality, well that is okay, but i don’t like it that way, so what to do

As a firefox user, I have an option to create a file inside my firefox profile ({firefoxProfile)/chrome/userContent.css), but it will not work out ofg the box, you also need to open about:config, then switch “toolkit.legacyUserProfileCustomizations.stylesheets” to true

That’s it, but you might also need to restart FireFox

Getting started with water cooling

I am no expert on water cooling a PC, but i have done a bit of research, and I am keeping things here for my reference

1- Do use distilled water, there is nothing wrong with it, unlike what people who sell coolant will try to tell you, it is working perfectly for me, I added about 20% car coolant to it for both color and other relevant properties

2- BOIL the distilled water to get the oxygen out of it, otherwise, over time the water will become still in the system, forming bubbles.

My pump already has a small reservoir

The system is very simple, Pump to block to radiator then back to pump, took much less time to assemble than i thought

Seagate Backup Plus HUB internal disk

I got an external USB3 8TB disk from Costco, namely the STEL8000401 Seagate Backup Plus Hub, the enclosure includes a USB3 HUB, and an 8TB 3.5 inch hard drive inside

upon opening the shell, I was presented with an 8TB SATA 3 6Gb/s drive with the model number ST8000DM004

According to the datasheet of the hard drive, this is an SMR drive *(Shingled magnetic recording)

Seagates markets this hard drive as a desktop hard drive, but beware, SMR drives are slow on writing, so when you are using the drive for a mostly read scenario, it may be good, but if your application is write intensive, this is not a good hard drive for that purpose

Also, from the datasheet, this drive spins at 5400RPM, opposite to my older seagate barracuda 8TB (ST8000DM002) which is a 7200 and is not an SMR drive

I will be doing some more experimentation with this drive in a bit, and post the results here

Sequential Write

The lowdown: 130 MiB/s sequential Write, and 132 sequential read ! the older drive has a 230MiB/s sequential read

The first experiment is using PV, for those who do not know, PV is like DD but somewhat more advanced, I will be copying the drive ST8000DM002 to the new ST8000DM004, after passing the 10GB mark, the transfer speed from one disk to the other was around 130 MiB/s (Speed of writing to the new SMR drive). Again, regardless of the data, this is a byte for byte copy including blank areas, therefore this is an example of a sequential write. for comparison, writing the same data to /dev/null (read on the old drive) results in 230 MiB/s meaning the bottleneck is writing to the SMR drive. reading from the newer SMR drive to /dev/null results in 132 MiB/s which is surprisingly similar to its read speed.

Linux on first generation I7 with no graphics card

Linux would not normally boot as soon as you take away the graphics card, the solution to this is to set it to serial mode, with Grub 2 (Such as debian buster and bullseye), there is no menu.1st file, and you should not edit /boot/grub/grub.cfg by hand, what you need to edit is /etc/default/grub, in this file you will need to

1- Change GRUB terminal to console and ttyS0. This will provide one GRUB to a monitor display and serial console.
2- Remove hidden parameter for avoiding “no video mode activated” error. And change GRUB timeout from 8 seconds to 1 second.
3-Change linux kernel console to tty1 and ttyS0. This setting will be taken over to userland, and there will be two login prompt for tty1 and ttyS0.

On my debian setup, the file looks like this, and it works, but make sure to take a backup of the file just in case before you modify anything

Now you will need to run the command

update-grub

Arab Bank Jordan Swift and BIC

After some research, It turned out that the Arab bank of Jordan has a unified SWIFT code (Same as BIC) for all their branches, which is ARABJOAX100, you will need to combine that with your account number which is 13 digits, all other swift codes for the branches are obsolete

بعد البحث، تبين ان كل فروع البنك العربي تستخدم سويفت كود (بيك كود) موحد وهو
ARABJOAX100
، جميع السويفت كودز القديمه للفروع القديمه غير مطلوبه، بالاضافه الى السويفت كود ستحتاج الى رقم حساب وهو رقم من 13 خانه (بالاضافه الى شحطتين تفصلان الارقام)

An account number would have the format

xxxx-xxxxxx-xxx

رقم الحساب يكون على النسق

xxxx-xxxxxx-xxx

WiFi for Arduino

Even though this looks like a long post, I have composed it for a friend and unlike mostly everything else on this blog, this is not just for my own reference, so it should be easy to follow and understand (I hope).

What for ?

This is a very valid question, Why would i use a slower Arduino and connect it to WiFi using an ESP8266 you ask, why not just use the ESP8266 or even ESP32 as both the WIFI and the microcontroller to run our code?
There are many situations where you would want to, the most common of which is the analogue and digital pins on an Arduino board, the friend I am writing this tutorial for is looking to use the 50 digital pins on an Arduino Mega Pro Embed as select lines for 50 Arduino pro mini boards, another might be the analogue pins on an Arduino (8 or 16 depending on the board), so digital and ADC pins on an Arduino might be needed.

You might ask why not an ESP32, it has a bunch of digital and analogue pins, the answer is that sometimes they are not enough, especially when you find out that the analogue pins on the ESP32 are divided into 2 groups, one of them is not usable if you enable WiFi.

Another valid reason is all the shields that have Arduino libraries but those libraries do not function with ESP, which is probably even more common of a problem than the pins problem.

So in short, even though the need might not arise very often, it does exist.

The ESP8266 as an Arduino WIFI shield

Arduino does not come with WiFi, there are shields from Arduino that provide WiFi, and those shields are based on ESP8266 which is a very cheap WiFi enabled microcontroller. but there is nothing stopping you from using any ESP8266 board and connecting it to your Arduino,

Which one: They should all work, and you probably already have one since you are here, I am personally using the slightly more expensive $4.6 boards that come with a USB-TTL chip and power regulator built in, if you want to use the cheaper boards (esp8266-01), you might want to connect it to the 3.3V output of your Arduino, but you will still need a level shifter, I would expect you also have a UART USB to serial board.

Price: models from the 01 ($2.5 each when you get 5 boards for $12 ) up to the 12E or 12F ($4.6 each when you buy them as 3 for $14). not bad for a WiFi enabled microcontroller !

Communication between Arduino and ESP8266

Arduino can talk to the shield either via UART or via SPI (Given the libraries written for this), SPI is up to three times faster than UART, but most of the time your application, be it sensor data or the like, will not be able to flood any of those 2 buses, In this post, I will cover both, SPI first then serial.

The components (hardware)

1- ESP8266 (Any variant should do)
2- Logic level shifter, since Arduino is 5V and ESPs are 3.3, I have been told that the ESP 12E and 12F are 5 volt logic tolerant, but I would think going with a logic shifter might save me something down the road, hours of debugging, or a new board, or something i fail to foresee
3- An Arduino, I am using a mega, but an UNO should do just fine (I will cover it)
4- Wires to connect all the above, and probably a breadboard (I like to solder things to a universal PCB board, but not everyone likes to do this)
5- A power supply, in my case a couple of micro USB cables and a 5V source that is my a power supply.

Software on the ESP8266

1:SPI: If you are going with SPI, you will need to flash JiriBilek / WiFiSpiESP onto your ESP8266, fortunately, this comes with an ino file that you can use your Arduino software to flash directly

2:UART-Serial: If you are going with serial, you might want to go with jeelabs / esp-link, mind you, Arduino themselves forked this before for their own WiFi shields, but since then, the jeelabs esp-link has added many features, so i would recommend you go with the original jeelabs.

Software on Arduino

1:SPI: if you have installed the SPI software from above on your ESP8266, the accompanying Arduino software would be JiriBilek / WiFiSpi, The library implements almost the same functions as the Arduino WiFi library.

2:UART-Serial: there is no library to go with this case that is beyond your regular serial bus if you want to exchange serial info, so if this is a 3D printer, software on your PC should be able to translate the data into serial, and it would be transparent, but what if you want to use WiFi from within Arduino, like a client that downloads pages or sends post data to pages,

Choice of UART-Serial vs SPI

UART-SERIAL, has certain advantages and disadvantages, with serial, i can simply update the software on the Arduino over the air over WiFi, I can get serial messages and use WiFi at the same time both as client and server, SPI on the other hand is faster, but it is not out of the box compatible with serial messages. Another disadvantage of SPI is that it needs a bit of extra code to allow the board to boot

Implementing WIFI over SPI

SPI – The hardware, how to connect

The H.SPI (On the ESP8266) is connected to the SPI on the Arduino like you would connect any SPI bus, with the addition of a logic level shifter (Red part in the photo), We connect Clock to clock, Slave select to select line, MOSI to MOSI and MISO to MISO, there is nothing to it. I have added a table for the Uno (Same for Arduino Pro Mini) and the Mega for your convenience

 NAME | ESP8266 | MEGA | Uno      | Logic Analyzer |
 SS   | D8      | D53  | D10      | CH0      | SS
 MOSI | D7      | D51  | D11      | CH1      | MOSI
 MISO | D6      | D50  | D12      | CH2      | MISO
 SCK  | D5      | D52  | D13      | CH3      | SCK

Now assuming you are done with the connection above, it is time to load some software.

SPI: Installing the WiFiSpiESP on the ESP8266

First, we need to load the software to ESP8266, the JiriBilek / WiFiSpiESP comes with a .ino file, so all you need to do is load that into Arduino studio, connect your esp8266, compile and upload, now this part is done, no modifications are needed to this code since all the control is passed on to the Arduino, compile and upload.

If you are having trouble uploading the code or selecting the board, my 12E board works in Arduino studio as NODEMCU V1.0, if you don’t have any ESP8266 boards in your boards list, you will need to add it, there are many tutorials on using Arduino with esp8266.

SPI: software on the Arduino

On the Arduino side, you will have to include the library (WiFiESP), then include it in your code, the library should be readily available in your libraries menu of your Arduino Studio.

NOTE: Both the library and the software you installed on your ESP need to have the same release number (0.2.5 at the time of writing) or it would not work, the software is hard coded not to work if they don’t match, you will be presented with the error (Protocol version mismatch. Please upgrade the firmware) in your serial console during runtime, I know this because a couple of weeks ago, I contacted the author (Jiri) through GitHub, and he brought both versions of the software and the library current so that they would match, it was a small thing but if you ever get this error in the future, you know where to go, he was quick to fix it within hours.

Now to the Arduino code, inside the library, there are examples, all you need to do is upload one of those examples, most likely, you would want to start off with the WiFiWebClient, this example that comes with the library needs to be modified in two locations, the first is the credentials to your WiFi, and the other is to change the server you are connecting to from www.example.com to wherever that web server is. this should get you started on most projects.

In my case, I have had to modify a few things in the script to make it work, first of all, a short delay needs to be inserted before we check if the WiFi is connected, the other is to not have it die but rather try again if it is not for a set number of times

WiFi using UART-Serial

UART-SERIAL should be the as easy, I should be back here

The ESP8266 has a TX and RX pin that should be connected in reverse to the ones on the Arduino, RX (Receive) should be connected to send, and send to receive, both boards need to share a common ground (reference voltage), and an Arduino mega should be able to provide 3.3 volts with sufficient current for the ESP8266 if you plan to power the ESP from the MEGA, if you have an ESP8266 with an onboard voltage regulator, you can simply add it to the power supply directly through the VIN pin (rather than the 3.3V pin)

Uploading jeelabs esp-link to the ESP8266

Start by downloading the zip file from GitHub,