QWORQS

This blog has plenty of posts about AI, some are about AI tools, others are about installing AI locally, so this post is where I am putting all the AI stuff I have ever blogged about in one place !

The section Local AI is about creating your own AI server using freely available sources, the API section lists all the services that provide an API but can not be installed locally, and the Online Services is where you can get things done via AI online (That can’t be installed locally or accessed programmatically via API)

Continue reading “Everything AI – TOC”

Langchain is an abstraction layer so that you don’t have to call the APIs of every AI provider separately

Originally in Python, langchain ports such as Dart and Rust exist

One reason for writing this post is that I can not seem to find a way to update the backup BIOS on the system, this means that if my BIOS ever gets corrupt, the system will revert to a very old BIOS, and I will have to do it all again, this post should save me a lot of time if that happens.

This is a computer as old as time that has always been giving me trouble, was used for a few years then went and found a home in the basement with all the other electronic garbage… but lately, I have been short on ram, and this machine had 24 Gigs of it, so i thought i might move some processes to it, so i fired it up and it fired just fine, then i noticed a problem with the RAM speed !

The CPU on the device is a first gen I7-980 (The 6 core one), and the ram is all identical (Kingston KVR1333D3N9/4G)

BIOS Update

So before i go about bashing Gigabyte (the deserve it as you will see through this post), I decided to start by updating the BIOS, I had the FA, and on the website, there were 3 versions, all of them newer than FA, namely FF, FG1, and FH, FH is obviously the one we are aiming for

So the motherboard is advertised to have a built in bios update function, so i started with that (Q-flash), trying to update to FH failed (Error about invalid bios file), So i decided to take them one by one, started by updating to FF which worked, but then went to FG1 and that failed, turns out FG1 is where EFI was added to the bios effectively doubling its size.

I am mentioning EVERYTHING here to spare you the hassle of trying unnecessary steps if you have the same motherboard

So, i created a DOS boot flash disk, and tried their update utility, that too failed with an error complaining that it is of the wrong size…

This is sort of a problem, I now need a copy of Windows 7, and the only copy of windows I have is on my laptop, Windows 11, and that will surely not boot on a motherboard without UEFI !

Eventually, i found another very old computer with windows, and what do you know, the update was only successful with Gigabyte’s @BIOS program.

Before using the tool, I followed the instructions about disabling hyper threading on the CPU from the BIOS.

So, now i have an up to date bios (And by up to date I mean 2012),let us get to the actual problem !

RAM issues

Ever since day 1, the RAM would report 1333 in the bios, but (400 MT/s) in the operating system (Cpu-z on windows, and “lshw -short -C memory” and “inxi -mxxx”)

the CPU itself only handles ram up to 1066, So i was expecting the motherboard to fall down to that !

but when running any of the commands above, it reports 400MHz and 400MT/s !

400 is a very low number ! especially when another computer from the same era reports (1600 MT/s)

Not only that, but I can’t seem to access the manufacturer name for example.

So after hours of trying to convince the system to use different values, I was surprised when i ran sysbench (sysbench memory run), sysbench did not yield the expected low results ! but instead, reported a data transfer of 5GB/s

So, up to now, I am under the impression that when the CPU slows down, it takes the RAM speed down with it, I will test this theory very soon, but for now, who cares, it is working and misreporting and that is all I am interested in this very minute

Below is almost the same post, i started writing it, then forgot i wrote it and wrote it again, leaving it here in hopes that it may be useful….

Now to the BIOS !

This was a real pain, putting the motherboard in production again (I need ram on the network), I notices that there is no EFI ! turns out that would require the bios to be upgraded from FA to FH ! (Through FF and FG1), the tools bundled with the firmware image don’t work ! I created a dos disk, and tried, but the updater that comes with the firmware claims the image is not compatible….

Turns out I have one choice to get the update going, install windows 7 !!!

So i found a hard drive with Windows 7, and used @bios to upgrade the bios

This is a question that I get all the time from people who think I know my way around computers. So i will try making it as simple as possible because people who ask me are usually not the savviest of people when it comes to technology.

A table comparing DRAM-less SSDs to SSDs with DRAM is available here (Nothing extra ordinary, just to give you a feel of what variables are compared)

Storage space

There is honestly very little that I can say about storage that you don’t already know, your workload, and what you store on your computer is something you probably know all about, if you have plenty of video content, you need a big disk, if you just need to browse the web, a very small 128GB disk would do, one more thing to keep note of is durability, in the world of SSDs, the TB-Written endurance/longevity rating increases as the disk grows bigger, simply because flash cells have a limited number of write cycles, the larger the disk, the more flash cells, the more data you can write to a disk without it going bad, Also, some performance parameters change slightly between different sizes of the same model, but I will not be making a choice based on those numbers, So the space consideration will boil down to “What are you planning to store on the disk”, and how much space will that need

Speed

The last time we spotted people to whom speed meant freedom of the soul was in 1997, ever since then speed has come to imply “Time savings” more than anything, So this is the area where things will get a bit complicated, the list of keywords you see here are what will be covered in this post

• Max transfer rate
• IOPS
• Max sustained rate
• Burst rate
• DRAM
• SLC Cache
• 4K reads/writes
• HMB (Host Memory Buffer)
• NAND flash memory for caching

Whether we are talking about spinning disks or SSDs, there is the maximum speed, which is basically a sequential read or write to the disk, which now, with NVMe (PCIe connected SSDs) has reached crazy speeds, and there is IOPS (Input/Output operations)

Cheap SSDs don’t come with DRAM, and for the average user, there is no problem with that, most data is for reading, and the SSD is hardly ever pressed to a workload where the difference is noticeable, so do I need a disk with DRAM ?

If you are just someone who plays games, runs a web browser, and boots windows, the short answer is NO, the difference in time is not worth it.

Gradio is all about user interface, it is a startup that got acquired by Hugging Face !

In your python code, you execute the following line

import gradio as gr # The way people usually call it

now, here is an example that turns your text to uppercase

def to_uppercase(text):
    return text.upper()

Now, to get a user interface, run the following code

gr.Interface(fn=to_uppercase, inputs="textbox", outputs="textbox").launch()

Now, you should be getting 2 boxes, one for your input, and the other to display your text in uppercase

Now, imagine the to_uppercase function being a call to an AI, so there you have it

Here is a variant with big boxes

# Inputs and Outputs

view = gr.Interface(
    fn=to_uppercase,
    inputs=[gr.Textbox(label="Your message:", lines=6)],
    # outputs=[gr.Textbox(label="Response:", lines=8)],
    outputs=[gr.Markdown(label="Response:")],
    flagging_mode="never"
)
view.launch()

At this point, when you run this, you should get the link to the URL with that user interface !

LM studio is a great tool to run models locally on your machine, but it is somewhat more than that

According to their intro, it is…

• A desktop application for running local LLMs
• A familiar chat interface
• Search & download functionality (via Hugging Face 🤗)
• A local server that can listen on OpenAI-like endpoints
• Systems for managing local models and configurations

This is a great front end for Stable Diffusion

the official GIT repo for this is here, to install it alongside stable diffusion and everything else you need, make sure you are running in your local environment with conda or pip, and run the following command

wget -q https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh

then make it executable and run it with the following command

./webui.sh --listen --api

You can add Automatic1111 to your openwebUI (Go to settings/images)

1- Installing as docker container

• Install docker like you would, by adding its repositories or however you are used to installing it
• sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
• docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
• to check if it is running, run sudo docker ps
• Now, localhost:8080 should have your OpenWebUI running
• Do a signup or create account (Local instance bound), the first account you create will automatically become an admin account !

At this stage, you should be good to go,

Every model is a different story, but before you start looking into models, there are a few useful things to know, and a few useful tools that you can use.

NVIDIA

Nvidia is a very popular AI hardware provider, the cool things about modern AI models is that they can be split into layers, hence, you can have more than one card doing the work ! So, I have 2 x 4090 cards doing the work, you can combine the ram to see if your model fits on both when split in half, some models even provide an option to offload some of the model’s data onto the system ram, other have the option to load part of the data into VRAM as needed ! but that is a story for another day (Another post)…

To inspect the GPU and RAM usage of your GPUs, you can use the following command

watch -n 0.5 nvidia-smi

The command should show you what processes live in your VRAM (VRAM is your cards ram)

From other posts like this one, I seem to always be complaining about the 209 E200 W211, The thing i never mentioned is that I have an even older Mercedes E200, A W124 that has been parked for 8 years and decided just now to revive it.

for one, there is a water leak in the cooling system, for another, it has no gas in it, and the fuel pump is not working !

I recall from 8 years ago someone telling me “I can smell gas” While the car was parked near us ! So i am guessing, there is a small leak that is leaking out the gas

Not long before it got parked (8 years ago), the car had a serious leak (Leaked a full tank in a day) and I had to change the hose that connected the fuel tank to the fuel pump that is under the car !

Here are the specs, and this is post is all about how I plan to bring this car to life in a few days !

My Mercedes 200 E (W124 – E200)

• Transmission: 5 Speed manual transmission
• Engine: 111.940 (Wikipedia) (Check if part is 111 010 8700 on xentry/das)
• It has that different looking steering wheel and shifter, If i recall right, this variant’s name has the word sport in it (probably sports line), but I am not sure what it was called exactly, I recall this info because I had to get the steering wheel shipped from Poland as no used or new parts dealer has ever seen it before

Diagnostics/OBD2

Well, this 1994 E does have a diagnostics port, but it is nothing like OBD2 (It is OBD1 not 2), it is very easy to read, just connect any LED (use a 12V LED, or adapt an LED for 12V with resistors obviously).

The 16 pin connector can be accessed with banana plugs connecting the LED to the flowing

Pin-3 = ECU (BLINK… LED+Button)
Pin-6 = K-line (BLINK… LED+Button) (Ex: VAG-COM USB reader)
Pin-1 = GND
pin-16 = 12V

The users manual does not go into sufficient detail concerning the (HFM Sequential Multiport Fuel Injection/Ignition System (HFM-SFI))

The OBD plug can be accessed with a scan tool (Ex: HfmScan) using the diagram below, if you insist on going into that rabbit hole, there are adapters (OBD2 to banana) like the one you see here, which connects the OBD connector to the following pins (OBD breakout cable 4 colours)

Now, I prefer using the LED method

Degrading wires (Mercedes and Porsche !)

Why I think claiming that the degrading wires were “intentionally installed for easier recycling” is a bullshit attempt to cover Mercedes benz’s unforgivable failure on those models

• Having an arcing wire right next to the fuel pump is an even worse failure than the failing to predict that the wires will bio degrade
• Giving a rust insurance of 30 years, and having the wires degrade after 10 ? doesn’t make any sense, You are implicitly telling the consumer you are expecting the car to live for 30 years, having its cables degrade starting after 10 years with the most dangerous ones, and eventually degrading to the extent that fixing it is not feasible contradicts the assumption of expected longevity !
• Assuming the above is not correct in the sense that not all wires degrade, only those exposed to heat etc… defeats the purpose they claim the wires were scheduled to degrade for

To revisit

When your primary concern is getting it to run, you will come across stuff that needs attention after, here is a small list

• The wires to the fuel pump need shrink wraps
• Distilled water to be swapped with antifreeze

Shopping list

I don’t think this will be a long list, but I am mentioning what I needed here so that if you are in a similar situation, this might remind you of what you might need

• Both kinds of polish compounds
• Retouch paint, and application thingie
• Thinner (To clean spray gun after)
• Clear coat
• Clean, lint free cotton
• Masking paper
• Electrician’s tape
• Gasoline ! since it probably evaporated
• A 12V tester (Usually in the form of a screw driver with a wire hanging out from the back to go to ground/Negative)
• A 12V car battery (The one in the car is 100% toast)
• Distilled water for the radiator (Add antifreeze later)

I found this adapter on Ali Express, the KUWEE Bluetooth 5 wired keyboard and mouse adapter, it should connect my beloved wired keyboard and mouse to my PC through Bluetooth 5, to any of 8 Bluetooth enabled devices

I got a few of them for a very nice project of mine, but more on that in another post !

According to the seller, here are the features of the device

• Brand: BRDRC
• Product size: 67x40x14MM
• Interface type: USB
• Effective distance:30M
• Supported system: Windows, ios. Android, OS.linux. TVOS, etc color:Black
• Material: plastic
• Package Contents: 1 x Gaming Keyboard Mouse Converter.

I am still looking for all the specs, but i think I found what I was looking for and am here to share this

Note: The device works fine with my wireless keyboard and mouse in 1 dongle (I used the keyboard port for the dongle), I understand it defeats the purpose (2 wireless hops), but just in case you have a reason to want it to work with 1USB that carrys both keyboard and mouse, there you have it, IT WORKS

1- Circulating through output: Switching output devices using the physical button on the device is inconvenient, you have to go through up to 7 clicks to get back to the one right before the one you are on (This is assuming all 8 slots are programmed, if you only programmed two devices, you only have to click twice to come back to the device you are on) !

So the answer is a shortcut, something a la KVM devices that intercept keyboard strokes as shortcuts, so the 8 inputs can be mapped directly to the 8 numbers on the keyboard with the following shortcut

Ctrl+Alt+Shift+1/8

2- Resetting the device: another shortcut that might come in handy is how to reset the device, here is the shortcut

device physical button + F1 on keyboard for 15 seconds

So, There you have it, the most useful two shortcuts that can mean the difference between convenient and inconvenient !

3- Multimedia buttons !

According to a random screenshot on my phone, the device adds multimedia buttons when you don’t have any physical ones !

• Ctrl + Alt + Shift + Esc = Multimedia Keys on / off
• Ctrl + Alt + Shift + F1 = iOS soft keyboard on / off
• Ctrl + Alt + Shift + F2 = Volume Down
• Ctrl + Alt + Shift + F3 = Volume Up
• Ctrl + Alt + Shift + F4 = Mute
• Ctrl + Alt + Shift + F5 = Previous
• Ctrl + Alt + Shift + F6 = Next
• Ctrl + Alt + Shift + F7 = Play / Pause

4- The USB output at the top: I don’t think it has any function other than sharing the power source to for example recharge a phone or something, not a very useful feature, but knowing what it is would at least answer your curiosities !

5- Size

For our American friends who would rather use any unit of measurement except for SI, the device itself is two fingers wide and less than one finger thick, and 3/4 index finger long, like in the photo below, as for everyone else, the dimensions of the device are 67x40x14MM

Here are some scans and photos i took for my future reference

So, here are a few photos of the device from the website, here for my reference

There is also another branding of the same external box at a much lower price (Photo below), sometimes less than half the price, but since i will have to wait a couple of weeks for it to arrive, I had to get the above because it has many more reviews, I would bet the other box is exactly like the one I got, but oh well, just a few dollars difference to spare me disappointment isn’t a big deal

And there also seems to be another device that shares the same input and output and description and I would bet it is them same device in a different shell, it looks like the one below, and it is also available in the KUWEE brand…