How to unpack bzip2 faster using parallel approach?

There are multiple tools which claim option to decompress bzip2 in parallel:

• pbzip2
• lbzip2

Let's compare pbzip2 performance with reference singe thread bzip2:

$ time bzip2 -d /tmp/rib.bz2  --stdout > /dev/null

real 0m52.188s

user 0m52.019s

sys 0m0.160s

$ time pbzip2 -d /tmp/rib.bz2  --stdout > /dev/null

real 0m49.380s

user 0m49.473s

sys 0m0.241s

You may notice that we have no speed improvement at all which means that pbzip2 cannot do decompression in parallel for standard bz2 compressed files.

But lbzip2 actually can do it and it offers great performance improvement:

$ time bzip2 -d /tmp/rib.bz2  --stdout > /dev/null

real 0m52.790s

user 0m52.549s

sys 0m0.224s

$ time lbzip2 -d /tmp/rib.bz2   --stdout > /dev/null

real 0m8.604s

user 1m8.099s

sys 0m0.420s

It's 9 seconds vs 53 seconds. It's 6 times improvement on 8 CPU server. 

Conclusions: use lbzip2 for parallel decompression. 

How to disable systemd-resolved on Ubuntu 18.04 server with Netplan

NB! This guide is not applicable for Ubuntu 18.04 with Desktop environment, please use another one as you will need to change Network Manager configuration too.

In our case we decided to disable it because of non RFC compliant resolver in customer's network:

Jan 18 18:19:05 fastnetmon systemd-resolved[953]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying  

First of all, confirm current DNS server:

sudo systemd-resolve --status|grep 'DNS Servers' 

Currently default configuration is following:

ls -la /etc/resolv.conf 

lrwxrwxrwx 1 root root 39 Mar  2 17:23 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

You will need to stop and disable resolved:

sudo systemctl disable systemd-resolved.service

sudo systemctl stop systemd-resolved.service 

Then remove symlink:

sudo rm /etc/resolv.conf 

And add customer's configuration (replace x.x.x.x by IP address of DNS server in your network):

echo 'search companyname.com' | sudo tee -a /etc/resolv.conf

echo 'nameserver x.x.x.x' | sudo tee -a /etc/resolv.conf

echo 'nameserver 8.8.8.8' | sudo tee -a /etc/resolv.conf

echo 'nameserver 1.1.1.1' | sudo tee -a /etc/resolv.conf

After that, I can recommend rebooting and checking that DNS resolution works fine on this server. 

 

The start job is running for Wait For Network to be configured with Ubuntu 20.04

I've got this issue on Digital Ocean VM with Ubuntu 20.04. Apparently, it started happening after minor upgrade via apt-get update / apt-get upgrade. 

Full text of error:

And network did not start at all. In console I was able to see that eth0 is active but had no IP. I think this issue is related with some cloud-init bugs triggered by upgrade.

I was able to fix it short term via KVM / recovery console by trying sudo ifdown eth0 and then sudo ifup eth0 but it failed again after reboot. 

As long term fix I've disabled cloud-init via special file:

touch /etc/cloud/cloud-init.disabled

Then it fixed network configuration but did not address 2 minute delay before ssh start.

Source of fix: https://www.digitalocean.com/community/questions/after-upgrading-to-20-04-lts-network-now-takes-2-minutes-to-start-because-of-cloud-init 

How to install hass.io aka Home Assistant on vmware esxi?

It's not very hard but you need to set a few more flags described in this guide.

When I started using Go?

Apparently I've started using Go for production projects in around November 13th of 2013. My first project was REST-like daemon to manage container based OpenVZ virtualisation. It was called VzAPI and was build using Go 1.1!

How to decode IPFIX on non standard port in tshark?

 You can do it easily:

mkdir /root/.config/wireshark

echo "cflow.ipfix.ports: 4739,4740" > /root/.config/wireshark/preferences

Easy way to control sound input and output in Ubuntu

 I have an external sound card Focusrite for my microphone and headphones but I use USB soundbar for output and all the time I need to switch between them.

And I'm happy to share this nice widget for Gnome shell. 

asciinema cast to SVG

Yay! We've got tool for it: svg-term-cli

How to convert asciinema screen casts into video mp4?

We have got great tool to save us: asciicast2movie.

I'll use Ubuntu 20.04 for my tests.

Install dependencies: 

pip install moviepy pyte 

Install fonts:

sudo apt-get install -y fonts-symbola fonts-droid-fallback fonts-dejavu

Start conversion:

python3 asciicast2movie.py ~/Downloads/community_installation.cast  ~/Downloads/community_installation.mp4

Unfortunately, it produced quite low res video :( 

How to associate .exe files with wine on Ubuntu 20.04?

 To do it you will need to install:

sudo apt install -y wine-binfmt

And then you can add wine as option in "open with" dialog:

sudo cp /usr/share/doc/wine/examples/wine.desktop /usr/share/applications

In addition to that you can call exe apps normal way after setting executable bit:

~/Documents/winbox64.exe  

How to improve wine applications appearance on hidpi / 4k displays

I've used this guide, it's very simple and you just need to run wine-cfg and select DPI value which works for you.

And then I've got following pretty look of Mikrotik's WinBox:

How to deploy Mikrotik in Google Cloud / GCE

You can use Mikrotik's official guide for start. To use all gsutil commands from VM in GCE you will need to set following permissions for it:

Also, you may experience issues during attempt to create bucket:

gsutil mb gs://cloud-hosted-router-images

Creating gs://cloud-hosted-router-images/...

ServiceException: 409 A Cloud Storage bucket named 'cloud-hosted-router-images' already exists. Try another name. Bucket names must be globally unique across all Google Cloud projects, including those outside of your organization.

It happens because all bucket names must be unique and you need to change default name of bucket from official guide.

In addition that I can recommend using bigger instance type:

gcloud compute instances create "chr-1" --zone "europe-north1-a" --machine-type "e2-medium" --image-family=cloud-hosted-router

To debug issues with image itself you can use serial output:

And example output output will look this way:

In addition to that I can recommend enabling "connecting to serial ports" to simplify debugging and keep access to serial console directly from Google Compute Console:

And you will be able to control Mikrotik CHR using local console (default login: admin, password: empty):

 

createrepo on Ubuntu 24.04

Unfortunately, createrepo command which can create RPM repositories was removed in Ubuntu 24.04 LTS.

Let's try building C based version of createrepo from: https://github.com/rpm-software-management/createrepo_c 

First of all, install all dependencies:

sudo apt install -y libcurl4-openssl-dev libbz2-dev libxml2-dev libssl-dev zlib1g-dev pkg-config libglib2.0-dev liblzma-dev libsqlite3-dev librpm-dev libzstd-dev python3-dev cmake

Then build it:

mkdir createrepo_folder; cd createrepo_folder

git clone https://github.com/rpm-software-management/createrepo_c

cd createrepo_c;

mkdir build

cd build

cmake .. -DWITH_ZCHUNK=NO -DWITH_LIBMODULEMD=NO

make -j 

cp src/createrepo_c /opt/createrepo

cp src/libcreaterepo_c.so /opt 

cp src/libcreaterepo_c.so.1 /opt 

cp src/libcreaterepo_c.so.1.1.4 /opt

Then we need to override RPATH to use libraries from /opt:

sudo apt install chrpath

chrpath /opt/createrepo -r /opt

 Do test run:

/opt/createrepo /var/public_repositories/

Directory walk started

Directory walk done - 18 packages

Temporary output repo path: /var/public_repositories/.repodata/

Preparing sqlite DBs

Pool started (with 5 workers)

Pool finished

Please do not remove original folder "createrepo_folder" because it has dynamic library needed for app itself. 

Firmware upgrade on Thinkpad X1 Extreme Gen 2 on Ubuntu 20.04

 It's actually pretty simple process, just run this in Terminal:

fwupdmgr update

Output may look the following way:

Upgrade available for Thunderbolt Controller from 42.00 to 59.00

Thunderbolt Controller and all connected devices may not be usable while updating. Continue with update? [Y|n]: Y

Downloading 59.00 for Thunderbolt Controller...

Fetching firmware https://fwupd.org/downloads/0ec44a6ae2c11843d80147271d99901d41a7489ff29444c820c918e46a8eac25-Lenovo-ThinkPad-X1E-Gen2-Thunderbolt-Firmware-N2OTG12W-Secured.cab

Downloading…             [***************************************] Less than one minute remaining…

Decompressing…           [***************************************]

Authenticating…          [***************************************]

Updating Thunderbolt Controller…                                 ]

Restarting device…       [***************************************]

Successfully installed firmware

Ubuntu 14.04 LTS kernel with drop_monitor support

In this guide, I'll describe how you could build kernel for Ubuntu 14.04 with CONFIG_NET_DROP_MONITOR option. It's pretty standard reference copy-n-pasted from official manual of Ubuntu with very small changes.

If you got this error:

sudo ./dropwatch 
Unable to find NET_DM family, dropwatch can't work
Cleaning up on socket creation error

Then it means that your kernel does not have CONFIG_NET_DROP_MONITOR option compiled.

I use Ubuntu 14.04 and could provide guide how you could rebuild kernel with this option.

Rebuild kernel:

apt-get source linux-image-$(uname -r)

Install build deps:

sudo apt-get build-dep linux-image-$(uname -r) libncurses5-dev

Start process:

cd linux-lts-vivid-3.19.0 

Change options:

chmod a+x debian/rules
chmod a+x debian/scripts/*
chmod a+x debian/scripts/misc/*
fakeroot debian/rules clean 

fakeroot debian/rules editconfigs # you need to go through each (Y, Exit, Y, Exit..) or get a complaint about config later

Then specify it:

Do you want to edit config: amd64/config.flavour.generic? [Y/n] Y

Required option you could find here:

Networking support - Networking options - Network testing - Network packet drop alerting service - [M]

Build it:

fakeroot debian/rules binary-headers binary-generic binary-perarch 

Check it:

cat ./debian.vivid/config/amd64/config.flavour.generic
#
# Config options for config.flavour.generic automatically generated by splitconfig.pl
#
CONFIG_HZ=250
# CONFIG_HZ_1000 is not set
CONFIG_HZ_250=y
# CONFIG_IRQ_FORCED_THREADING_DEFAULT is not set
CONFIG_NET_DROP_MONITOR=m
# CONFIG_PREEMPT is not set 

CONFIG_PREEMPT_VOLUNTARY=y

And finally load it when you restart to new kernel:

modprobe drop_monitor