Monday, October 10, 2016

A strange note on bash aliases

Today I encountered a strange scenario (rather common for most of programmers) while adding function in my .bashrc file. For confidentiality purpose, I have slightly modified the use-case to present the scenario. Here is what I did

First, I added alias. But it did not suite my needs wherein I wanted to append folder name to the complete path.

alias my_func="echo /home/nandakumar/funny/"

What I achieved with this alias was

nandakumar@heramba:~$ source ~/.bashrc
nandakumar@heramba:~$ my_func nanda
/home/nandakumar/funny/ nanda


My expectation was to print /home/nandakumar/funny/nanda i.e. without added space. However, aliases do not have such capability.

After scouting the digital knowledge book (aka internet), I realized my use case needed function

So I transformed alias to bash function as below

And the output:

nandakumar@heramba:~$ source ~/.bashrc
nandakumar@heramba:~$ my_func nanda
/home/nandakumar/funny/ nanda


Strange!! Output Unchanged!!! I almost pondered 20 mins wondering what went wrong. I checked syntax, the commands umpteen times. Alas! No luck :-(. Eventually I closed and re-opened shell and to my surprise, here was output

nandakumar@heramba:~$ my_func nanda
/home/nandakumar/funny/nanda


The output is clean now 8-).

Later when I re-did the experiment, alias entry was still present in bash despite wiping out from .bashrc file

nandakumar@heramba:~$ alias my_func
alias my_func='echo /home/nandakumar/funny/'

I stumbled upon one of stack-overflow query later in day and figured out that this is expected. A quote states as below from: http://stackoverflow.com/questions/7131670/make-bash-alias-that-takes-parameter

"If you are changing an alias to a function, sourceing your .bashrc will add the function but it won't unalias the old alias. Since aliases are higher precedent than functions, it will try to use the alias. You need to either close and reopen your shell, or else call unalias <name>. Perhaps I'll save someone the 5 minutes I just wasted."

Ha!! HA LOL :D. All I needed was explicit unalias to achieve what I wished for. One more important lesson learned. Hope you also gained some insight.

This may look familiar to many, but for me it's new learning.

Tuesday, April 19, 2016

Exploring Linux Kernel with Virtualbox - Part 2 - Compiling kernel and install guest additions

Continued from Part 1


I am back!

This post primarily concentrates on following aspects.

(i) Kernel compilation
(ii) Installation of guest additions
(iii) Sharing folders for kernel installation
(iv) Tips

Before starting with mainline topics, I have one more Machine Settings for boot order. Not sure, how much improvisation apparently it yields, but worth the try.

Goto Machine Settings-->System-->Motherboard-->(Uncheck Floppy option)

The irrelevance of Floppy drives in contemporary technology, inspired me to un-check such an option. However, it might be useful if you want to pamper with older systems like DOS.

Also move up Hard disk boot order above all and Optical drive as next priority.



Kernel Compilation

Once the source tar ball is extracted. You need to install libcurses to open up menuconfig.

sudo apt-get install libcurses-ocaml-dev

Once done with installation, enter the below command in extracted source directory to configure kernel.

make menuconfig

Following configurations are required for debugging kernel.

Note: * represents the option is enabled. Press Spacebar to toggle option.

Enter the Kernel Hacking sub-option and enable following. There are umpteen options for debugging but as of now these two enough for basic exploration.



(i) Enable Kernel Debugging: By default this option is enabled, if not make sure to enable


(ii) Enable Stack backrace support: This is required to dump stack trace at required code points



Few tips to fasten up kernel compilation. I will update, if I spot anymore.
  1. Disable container and namespace support: I prefer to enable it for future purpose.
  2. Disable unnecessary file system support: This can save lot of time. You do only need basic filesystem support and others required only if you wish to know the filesystem structure. Since the linux follows VFS, the flow is identical from User space except for lower layer implementation.
  3. Disable excess driver support: This requires careful attention. Many drivers such as apple device support etc.. can be easily omitted. Make sure you do not disable basic drivers such as Mouse, Keypad, USB etc..
Once done, keep pressing Exit till it returns to main menu. Then press Save to save configuration. Let the default name be retained.


The below picture confirms the saved configuration.



Now we are ready to compile! Execute below command to start compilation

make && make modules

It will require 3-4 hrs depending on hardware configuration and kernel configuration to complete the process. Sit back and listen to few uplifting music, sip a coffee, eat churmuri or watch a movie even [Kung Fu Panda 3 preferred :)].

Once completed you will get success message.


Now we have the kernel image and ready to explore.

Install the new kernel image with following commands.

sudo make modules_install (To install modules)

sudo make install (To install the new kernel bzimage)

Note: Make sure to install the modules first and then kernel image


Reboot the machine!

After reboot check if the kernel version is what you compiled.

vmguest1@heramba:~$ uname -a
Linux heramba 4.2.8-ckt3 #4 SMP Sat Apr 9 11:02:58 IST 2016 i686 athlon i686 GNU/Linux
vmguest1@heramba:~$ date
Sat Apr  9 11:24:45 IST 2016
vmguest1@heramba:~$





Hurray!! It is done.

Installing Guest Additions:

In my previous post, I preferred to defer installation of guest additions. The reason is, installation of guest additions involves compilation of kernel modules. Hence if we had done installation with vanilla build, it would have required recompilation of guest additions (since we installed our own compiled kernel). In order to mitigate the time involved, it is better to install guest additions after kernel is compiled.

Once ubuntu is booted, click on Devices --> Insert Guest Additions CD Image



You will get prompt as below and click on Run



The process involves compilation of guest modules with installed kernel and later installation.




Once done, reboot the guest OS and you will recognize the change in display rendering of guest OS. Now the guest OS display fits into your native display.

Please note that in my case guest additions were pre-installed and hence one does not see the default display rendering of VM

The guest additions are required for following purposes. This is non-exhaustive list of guest additions' usage.
  1. Native display support
  2. File sharing between guest and host OSes
  3. Clipboard data sharing
Next part will explain few tips and short debug session.

Friday, April 8, 2016

Exploring Linux Kernel with Virtualbox - Part 1 - The environment setup


Finally few experiments is about to fire with Linux Kernel. I am sharing those learning curve with a blog-post to invite readers for feedback. I have setup virtualbox to debug linux kernel and it has been fun so far. I want to make it more interesting by debugging/exploring major functionalities of linux kernel. Here is my initial start to this festival and I hope it will last long!

For all the demotivators, stop right here and read:

There are humongous online references and in form of printed text. So why is this juggling required ?

You know! experiences ingrain to brain more than reading theory. When you dont have such related work in your office, you buy those experiences by motivating yourself!


It's just beginning with exploration towards networking aspect of kernel. Hope I will come up with various other parts of linux not limited to networking.

The initial few sessions concentrate on bringing up setup and a tiny debug session.

My setup

Host: Ubuntu x86_64
Guest 1: Ubuntu i686
Guest 2: Ubuntu i686 (As of now not much in use. Primarily used to simulate network forwarding)

What you need in Host OS?

a) VirtualBox
b) VirtualboxGuestAdditions
c) Ubuntu i686 ISO (from Ubuntu site)

Install above utils with

sudo apt-get install virtualbox virtualbox-guest-additions-iso

Once installed, it is time to create virtual machine. I am not elaborating the process involved since lots of tutorials point at this procedure. Here are the steps leading to installation of guest OS.

Creating a virtual network

Create a Host only adapter Network interface under File-->Preferences-->Network.

Here I have added vboxnet1 as well but it is not required as of now.

Edit the settings (as shown in figure below) of added interface and disable the DHCP server and assign a manual address of your choice.




Choosing RAM size

I have healthy 16GB of RAM and hence chose 2GB for my VM. You can choose appropriate size that fits to your actual physical memory or choose size recommended by virtualbox.




Choosing disk space

Make sure to give at-least 24GB of disk space to accommodate guest OS. I prefer fixed size since we will not do much installation other than linux kernel. VDI image was chosen for storing VM disk. I have not experimented with other formats. If you wish to inter operate with other VMs, you can chose other formats.


  



Networking between host & guest

After VM is created, click on created VM and click on Machine Settings in Toolbar.

Note: There are two types of settings

(i) Global Settings which is applicable to all VMs
(ii) Per VM settings which refers to VM specific settings.

When I refer as "Machine Settings" it means VM specific settings.



Enable Adapter1 as NAT for connecting towards Internet.

Note: If Internet requires proxy, the proxy has to be set inside guest OS.




Enable another Host-only adapter interface and choose already created interface in Creating Virtuan Network section



Enable clipboard sharing

This is very important if you need to work with clipboard data between host & guest (copy & paste)

Goto Machine Settings-->General-->Advanced, choose "Bidirectional" for both parameters as shown below.



Enable file sharing

This enables you to share files between host & guest. To make use of this functionality, Goto Machine Settings-->Shared Folders and add a folder of your choice.



Installing OS

Boot the machine and it will guide you do browse for virtual boot disk. At that time, point to ISO file downloaded for installation. Make sure to disable downloading updates from internet.




The installation is like any other linux OS. The only recommendation from me is to have custom partitions as below.

(i) Dedicated /home partition for kernel compilation (about 14GB)
(ii) Dedicated / partition for installing new kernel (about 8GB)
(iii) Rest 2GB can be for swap

Note: The below picture is just an illustration and does not conform to the above mentioned parameters




Installing Guest Additions

Once installation is completed, first step is to Install guest additions on guest OS. But I prefer to defer this step for the moment. Why? Soon the secret will be unfolded ;).

Once the guest OS is booted, first and foremost, download the kernel source.

sudo apt-get install linux-source

This will be place /usr/src directory as compressed bz2 file.

Create a directory in /home/<user>/ say name "kernel" and copy the bz2 file to that directory. Once done extract the kernel source. Below file name is just for illustration. It could be something else in your native system.

tar -xf linux-source-4.2.0.tar.bz2


Now the system is ready for kernel game and we are ready to cheer! The sequel will be explained in subsequent blog post.

Sunday, January 17, 2016

Welcome back Ubuntu!

My prolonged tryst with Linux Mint is finally over. It has been a splendid relation with Mint but had to be discontinued due to valid reasons. The transition has been smooth too! The only thing that held me back from using Ubuntu was its Mac Look & Feel. Consequently, this time too I had installed Mint 17.3 Rosa but was disappointed with its performance. Being descendant of Ubuntu, I am not sure why it does not match up-to vanilla Ubuntu editions. The primary reason for ditching is that newer mint releases are planned with older stable releases of Ubuntu which is major hindrance to upgrading latest softwares. Newer versions of utilities I use (Darktable, KDENLIVE etc..) cannot be installed with newer Mint releases or requires lot of juggling with source repositories. One more point of highlight is that I felt Mint is not leveraging the hardware potential rightly. This was more profound after installing 17.3 which was sluggish in processing power. All these point tempted me to try ubuntu again.

The initial hurdle of installing ubuntu was undoubtedly its OSX LAF :(. I was heavily annoyed to see its desktop environment. But there is a way to get around. After installing "gnome-session-flashback" and making few settings, my desktop environment is back to legacy look. Also "unsettings" app made sure I have required icons on the desktop. The other bottleneck there is no "Hibernate" alongside other power options. User needs to manually test hibernate functionality using "pm-hibernate" and configure if the hibernation is successful. There are articles which help you to get around hibernate. One more tweak required is that we need to create templates in "Templates" directory for showing up some options in nautilus context menu. Eventually, I have faster running ubuntu with desired OS & UI settings. Hope this relation stays for a while. Not sure when will I reconcile with Mint again! Here is my new mate in picture :). It is faster, smoother and glossy too :D

Tuesday, September 1, 2015

A note on pushd and popd commands

Users of Linux shell are familiar with pushd & popd commands. But wait a minute, how does it work? How does it remember the directories which it traversed. Just a stack functionality right?
As you are aware, bash does fork+exec on the commands it runs and hence it is impossible for pushd & popd to store the directory list if spawned afresh. The solution is none other than the method used by bash for 'cd' command. Yes, the commands 'popd' & 'popd' too nestle within the bash executable ;). Of-course the concept of shared memory can pitch-in but why complicate when there is simpler solution ;)

The experiment as mentioned in my 'cd' post

heramba@heramba-MS-7640 ~ $ which pushd
heramba@heramba-MS-7640 ~ $ which ls
/bin/ls
heramba@heramba-MS-7640 ~ $ which popd

Saturday, April 18, 2015

A NULL pointer trivia

Off-late, I have been pondering on the NULL checks in 'C'. As I skimmed through programming forums, one aspect that arose was the checking of NULLness of pointer.

For ex:

char *k = NULL;

if (!k)
{
}
else
{
}

What do you think? Will it ever fall through the else block? Some say, Yes! It is possible to have NULL as a pointer which is reserved for systemic purpose (not even kernel). So the ideal way would be

if (k == NULL)
{
    //bail out
}

Moving a step ahead, we nowadays have grandeur of VMAs which means expression "!k" is always comparing with VMA rather than physical address. However, NULL is constant expression defined by compilers rather than OS by itself which means first example may sometimes mislead.

So what do you say, the first one is not right way? May not be! The C standard says NULL is integer constant expression with a value zero, and resolving to generic pointer expression. This means the first block is always safe provided the compilers follow the standards. Also note that first is quite optimized version since assembly requires to check JNZ instruction while the other (k==NULL) requires copying the address to register and comparing!

What is your opinion?

Monday, September 15, 2014

A simple read-write lock implementation - Problem 2

This is the continued analysis of simple read-write lock implementation problem. This time it is starvation of writer locks.

Let me modify the thread routines and remove the sleep() after unlock, which means that readers or writers are assumed to have no pause in between. Here are modified thread routines.

void*
thread_routine1 (void *arg)
{
    for (;;) {
        my_rwlock_read_lock(&g_rw_lock);
        printf("Grabbed read-lock -- 1\n");
        sleep(10);
        my_rwlock_read_unlock(&g_rw_lock);
    }
}

void*
thread_routine2 (void *arg)
{
    for (;;) {
        my_rwlock_read_lock(&g_rw_lock);
        printf("Grabbed read-lock -- 2\n");
        sleep(10);
        my_rwlock_read_unlock(&g_rw_lock);
    }
}


void*
thread_routine3 (void *arg)
{
    for (;;) {
        my_rwlock_write_lock(&g_rw_lock);
        printf("Grabbed write-lock -- 1\n");
        sleep(5);
        my_rwlock_write_unlock(&g_rw_lock);
    }
}

void*
thread_routine4 (void *arg)
{
    for (;;) {
        my_rwlock_write_lock(&g_rw_lock);
        printf("Grabbed write-lock -- 2\n");
        sleep(5);
        my_rwlock_write_unlock(&g_rw_lock);
    }
}


What is the output?

Grabbed read-lock -- 1
Grabbed read-lock -- 2
Grabbed read-lock -- 2
Grabbed read-lock -- 1
Grabbed read-lock -- 2
Grabbed read-lock -- 1
Grabbed read-lock -- 1
Grabbed read-lock -- 2


:). As you see the writers are severely starved! As of now, I am not still have zero thoughts to fix this issue. If you find any suitable approach please let me know in the comment box.