auf.suno
Connector, investor, futurist, geek, software developer, innovator, sportsman, libertarian, business enabler, cosmopolitan, autodidact, funny finch, tech evangelist,
purist, agnostic, Kärnten fan, foodie, artist, globetrotter, social liberal but fiscal conservative, Schöngeist... elegantiorum litterarum amans oder studiosus...

This is the website of Markus Gattol. It is composed, driven and secured/encrypted exclusively by Open Source Software. The speciality of this website
is that it is seamlessly integrating into my daily working environment (Python + MongoDB + Linux + SSH + GIT + ZeroMQ) which therefore means it
becomes a fully fledged and automatized publishing and communication platform. It will be under construction until 2014.

Open Source / Free Software, because freedom is in everyone's language...
Frihed Svoboda Libertà Vrijheid เสรีภาพ Liberté Freiheit Cê̤ṳ-iù Ελευθερία Свобода חרות Bebas Libertada 自由
auf.suno
Website Sections
Home
FAQs
About Me
Tweets by @markusgattol
Bourne-again Shell
Status: Done except for the "Bash Programming" section.
Last changed: Saturday 2015-01-10 18:31 UTC
Abstract:

Bash is a free software Unix shell written for the GNU Project. A Unix shell, also called "the command line", provides the traditional user interface for the Unix operating system and for Unix-like systems. Users direct the operation of the computer by entering command input as text for a shell to execute. The name Bash is an acronym which stands for Bourne-again shell. The name is a pun on the name of the Bourne shell (sh), an early and important Unix shell written by Stephen Bourne and distributed with Version 7 Unix circa 1978, and the concept of being "born again". Bash was created in 1987 by Brian Fox. In 1990 Chet Ramey became the primary maintainer. Bash is the default shell on most GNU/Linux systems as well as on Mac OS X and it can be run on most Unix-like operating systems. It has also been ported to Microsoft Windows using the POSIX emulation provided by Cygwin, to MS-DOS by the DJGPP project and to Novell NetWare. The Bash command syntax is a superset of the Bourne shell command syntax. The vast majority of Bourne shell scripts can be executed by Bash without modification, with the exception of Bourne shell scripts referencing a Bourne special variable or those using builtin Bourne commands.
Table of Contents
Colorized Shell Prompt
The Rationale
What it looks like
Code
Setup
Bash History
Involved Files / Variables / Parameters
Files
Variables and Parameters
Configure / Tune
Test the Settings made
Exploring the History Features
Using Event Designators
Search the history using C-r
Repeat the previous Command
Execute a specific Command
Subtitute words from History Commands
Security Considerations
The Idea
Bash Programming
Miscellaneous
Bash Completion
GIT Bash Prompt
GIT + Virtual Environment
Some .bashrc customization I did

Colorized Shell Prompt

I use the CLI a lot. Especially when the number of concurrent terminal windows (and therefore terminal sessions) exceeds two or so, folks tend to confuse terminal sessions with one another — all look the same (color, prompt, etc.) at first sight...

What we want/need is to recognize terminal windows/sessions just by taking a glimpse i.e. are we root with the current session or not, is the current terminal session used for working remotely or locally and if working remotely on some server, are we doing so using SSH (Secure Shell)?

Since Debian ships Bash as its default shell (which I think is an excellent choice; yeah I know zsh folks think otherwise ;-]) I decided to finally shorten my todo list for one item and to colorize my bash prompt.


Update: Debian moved to dash for its default shell. Go here for more details about how to make Bash the default shell — dash is more or less a sub-set of Bash in terms of functionality i.e. making Bash the default shell again is just fine.

The Rationale

Anyone who ever wanted to reboot his workstation using init 6, reboot or something akin but instead caused a remote server to reboot will agree that having easier to distinct terminals sessions is not just a blessing but a must-have for any CLI junkie.

I use commands like shutdown to shutdown my subnotebook or workstation all the time. I use SSH to work on some remote machine and of course I use the CLI for pretty much anything else too. Even though using the CLI boosts productivity it is also something to fantastically shoot yourself into the foot. Because I do not want that to happen, coloring my prompt is a good thing to do.


I want to be able to easily distinguish if I am root or not (some standard user e.g. sa). I also want to easily distinguish if I am working on some local machine (e.g. subnotebook, workstation, etc.) or if I am working on some remote machine e.g. father's desktop across the desk or some server hundreds of miles away in some datacenter. Last but not least, even by making all this magic things happen, I do not want to heavily change Debian's default prompt appearance i.e. I am only going to colorize it but not to change its behavior nor its typical appearance which is username@host:~$ e.g. sa@wks:~$ (me using my standard user sa on my workstation).

What it looks like

The left image shows me working locally on my subnotebook, using my standard user sa. Then I become root and what changes is that # appears red — that is the only thing I changed here compared to Debian's default setting i.e. Debian's default prompt for root looks exactly the same except that # is not red but also white as is the rest of the prompt after becoming root.

With the right image, at first I am working with my standard system user on my local machine. Then I use SSH to connect to some remote server i.e. I am now working remotely but I am still sa. As can be seen, compared to using sa locally where the whole prompt is white it has become entirely green now. Then I become root on the remote server — as with becoming root locally — I get a red # but the rest of the prompt stays green.

The explanation for what can be seen above is as follows:

  • root no matter where (locally or remotely) gets his red #
  • working locally means the prompt appears white
  • working remotely
    • provides me with a green prompt in case of SSH
    • provides me with a blue prompt in case of telnet
    • etc.

It took me about 2 days to feel comfortable and get used to this new setup. Before that I used Debian's standard setup which is the same except for colors. My opinion is that, aside from looking really nice, it really helps me a lot with not confusing terminal sessions and therefore doing something stupid.

Code

There is not much one needs to know in order to understand the basics of how to colorize the Bash prompt. The Bash manual (man 1 bash) file for example is an excellent source of information. Aside from it, the WWW (World Wide Web) is full of hints how to colorize a prompt. The actual code I use to make things colored can be seen below:

sa@sub:~$ grep -v \# .my_bash_settings

THIS_TTY=$(ps aux | grep $$ | grep bash | awk '{ print $7 }')
SESS_SRC=$(who | grep $THIS_TTY | awk '{ print $6 }')
SSH_FLAG=0

SSH_IP=$(echo $SSH_CLIENT | awk '{ print $1 }')
if [ $SSH_IP ] ; then
  SSH_FLAG=1
fi

SSH2_IP=$(echo $SSH2_CLIENT | awk '{ print $1 }')
if [ $SSH2_IP ] ; then
  SSH_FLAG=1
fi

if [ $SSH_FLAG -eq 1 ] ; then
  CONN=ssh
elif [ -z $SESS_SRC ] ; then
  CONN=local
elif [ $SESS_SRC = "(:0.0)" -o $SESS_SRC = "" ] ; then
  CONN=local
else
  CONN=tel
fi

if [ $(/usr/bin/whoami) = "root" ] ; then
  USR=root
else
  USR=notroot
fi

if [ $CONN = local -a $USR = notroot ] ; then
  PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
elif [ $CONN = local -a $USR = root ] ; then
  PS1='${debian_chroot:+($debian_chroot)}\h:\w\[\033[01;31m\]\$\[\033[00m\] '
elif [ $CONN = ssh -a $USR = notroot ] ; then
  PS1='\[\033[01;32m\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$\[\033[00m\] '
elif [ $CONN = ssh -a $USR = root ] ; then
  PS1='\[\033[01;32m\]${debian_chroot:+($debian_chroot)}\h:\w\[\033[01;31m\]\$\[\033[00m\] '
elif [ $CONN = tel -a $USR = notroot ] ; then
  PS1='\[\033[01;34m\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$ \[\033[00m\]'
elif [ $CONN = tel -a $USR = root ] ; then
  PS1='\[\033[01;30;45m\]${debian_chroot:+($debian_chroot)}\h:\w\$ \[\033[00m\]'
fi

export PS1

sa@sub:~$

Setup

Now that we have the code available with ~/.my_bash_settings we need to setup the whole shebang. I am not going to detail why I did what I did below because the afore mentioned manual file does so already.

Fact is, we want to make this work for root and non-root users as well i.e. sa in my case. Therefore I make little changes to /root/.bashrc and ~/.bashrc as can be seen:

1  sa@sub:~$ grep -A0 -B2 my_bash_settings .bashrc
2  # set a fancy prompt
3  #PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
4  source /home/sa/.my_bash_settings
5  sa@sub:~$ grep -A0 -B2 my_bash_settings /root/.bashrc
6  # Changed by Markus Gattol
7  #export PS1='\h:\w\$ '
8  source /home/sa/.my_bash_settings

As can be seen in lines 4 and 8, source is used to... well, source ~/.my_bash_settings every time a new terminal session is started/spawned. Hint, those who do not know about the source command might first try type source followed by help source... and no, there is no manual file for source because... well, again, type source :-]


The changes to both /root/.bashrc and ~/.bashrc only affect the machine locally i.e. if we want a colored prompt locally on our workstation then it has to be setup there.

Likewise, if we want it on a remote server which is accessed via SSH (Secure Shell), it has to be setup there as well i.e. putting ~/.my_bash_settings into place and alter /root/.bashrc and ~/.bashrc as shown above in lines 1 to 8.


Last but not least, in order for bash colorization to take effect no matter if we use an interactive login shell or just an interactive shell that is not a login shell, we should uncomment a few lines in

10  sa@sub:~$ grep -v \# .bash_profile | grep .
11  if [ -f ~/.bashrc ]; then
12      . ~/.bashrc
13  fi
14  sa@sub:~$
 

Everywhere (locally and remote machine, virtualized or non-virtualized environment) where we have a terminal session and want to have a colored shell prompt, need we set up the whole shebang — sounds like a lot of work but actually it is not, it really is not...

Bash History

The Bourne Again Shell's history mechanism, a feature adapted from the C Shell, maintains a list of recently issued commands on the CLI (Command Line Interface), also called events, providing a quick way to reexecute any of the commands/events in the list. This mechanism also enables us to execute variations of previous commands and to reuse arguments. We can replicate complicated commands and arguments that we used earlier — either in this login session or in a previous one — and enter a series of commands that differ from one another in minor ways. The history list also serves as a record of what we have done i.e. it can prove helpful when we made a mistake and are not sure what we did, or when we want to keep a record of a procedure that involved a series of commands.

Involved Files / Variables / Parameters

In order to set up respectively configure bash history behavior to our likings, we need to know about a few files and variables/parameters which influence heavily what happens for which users under which circumstances.

Files

It is always good to know with whom we are dealing with:

  • /bin/bash, The bash executable
  • /etc/profile, The systemwide initialization file, executed for login shells
  • /etc/bash.bashrc, The systemwide per-interactive-shell startup file
  • /etc/bash.logout, The systemwide login shell cleanup file, executed when a login shell exits
  • ~/.bash_profile, The personal initialization file, executed for login shells
  • ~/.bashrc, The individual per-interactive-shell startup file
  • ~/.bash_logout, The individual login shell cleanup file, executed when a login shell exits
  • ~/.inputrc, Individual readline initialization file

It is utterly important to understand the meaning of those files with regards to what kind of shell (login, interactive, etc.) respectively user (particular user or systemwide) we want to provide settings for. As we will later see, we are only focusing on my normal user (sa) but both major types of shells i.e. login and interactive. Therefore we are going to touch ~/.bash_profile and ~/.bashrc or to be more precise /home/sa/.bashrc and /home/sa/.bash_profile.

Variables and Parameters

As can be read in detail with man 1 bash, we are interested in

sa@sub:~$ echo $HIST
$HISTCMD       $HISTCONTROL   $HISTFILE      $HISTFILESIZE  $HISTSIZE
sa@sub:~$

We want to get our history list bigger plus smarter. Bigger can be done with HISTFILESIZE and HISTSIZE, smarter with HISTCONTROL, HISTIGNORE and HISTTIMEFORMAT. Now is a good time to go and read the man file for bash in order to understand the meaning about those afore mentioned variables. In short

  • HISTSIZE, The number of commands/events to remember in the command history during a session. The default value is 500.
  • HISTFILESIZE, The maximum number of lines/commands/events contained in the history file ($HISTFILE) i.e. maximum number of events saved in between sessions or in other words the number of total history events saved in a persistent manner. When this variable is assigned a value, the history file is truncated, if necessary, by removing the oldest entries, to contain no more than that number of lines — the default value is 500, same as for HISTSIZE.
  • HISTCONTROL, A colon-separated list of values controlling how commands are saved on the history list. If the list of values includes ignorespace, lines which begin with a space character are not saved in the history list — this is cool if we want some commands to NOT be logged. A value of ignoredups causes lines matching the previous history entry to NOT be saved. A value of ignoreboth is shorthand for ignorespace and ignoredups. A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved. Any value not in the above list is ignored. If HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the value of HISTIGNORE. The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTCONTROL.
  • HISTIGNORE, A colon-separated list of patterns (e.g. HISTIGNORE=ls:ll:la:l:cd:pwd:exit:mc:su:df:clear) used to decide which command lines should be saved on the history list. Each pattern is anchored at the beginning of the line and must match the complete line (no implicit * is appended). Each pattern is tested against the line after the checks specified by HISTCONTROL are applied. In addition to the normal shell pattern matching characters, & matches the previous history line. & may be escaped using a backslash — the backslash is removed before attempting a match. The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTIGNORE.
  • HISTTIMEFORMAT, If this variable is set and not null, its value is used as a format string for strftime to print the timestamp associated with each history entry displayed by the history built-in (issue type history && help history for more information). If this variable is set, time stamps are written to the history file so they may be preserved across shell sessions.

Configure / Tune

As I said within the File section above, I want the settings to take effect for one user with his login as well as interactive shells. First the stuff that does the magic

sa@sub:~$ pwd
/home/sa
sa@sub:~$ cat .my_bash_history_settings
# This file provides bash history settings. It is loaded/sourced in
# ~/.bashrc and ~/.bash_profile with `source
# $HOME/.my_bash_history_settings'.

shopt -s histappend           #append to the end of $HISTFILE
export HISTFILESIZE=20000     #number events saved in total
export HISTSIZE=20000         #number events saved during a session
export HISTTIMEFORMAT="%A  %Y-%m-%d  [%T %z] "
export HISTCONTROL=ignoreboth:erasedups


# Local Variables:
# mode: shell-script
# End:

And then how to apply it for both, interactive login shells as well as interactive shells for a single user (sa in our particular case).

sa@sub:~$ grep -v \# .bash_profile | grep .
if [ -f ~/.bashrc ]; then
    . ~/.bashrc
fi
sa@sub:~$ grep my_bash_history .bashrc
    source $HOME/.my_bash_history_settings
sa@sub:~$
Hint
As of now (September 2008) the erasedups thingy does not seem to work in between sessions i.e. only does it erase duplicates made within the same shell session. Once we end a session (e.g. closing the terminal, logging out, etc.), shopt -s histappend ensures that the in-memory history is appended to $HISTFILE.
What does this mean? Well, nothing more or less but that $HISTFILE actually might contain duplicates if issued in different shell sessions.
A command to get rid of them in a persistent manner is tac ~/.bash_history | gawk '!seen[$0]++' | tac > ~/.bash_history_no_dups && mv ~/.bash_history_no_dups ~/.bash_history (please test before using it).

Test the Settings made

How it looks like

 1  sa@sub:~$ history 10
 2    506  Wednesday  2008-09-03  [17:15:46 +0200] source .bashrc
 3    507  Wednesday  2008-09-03  [17:18:47 +0200] history -w
 4    508  Wednesday  2008-09-03  [18:33:04 +0200] tail .bash_history
 5    509  Wednesday  2008-09-03  [18:33:46 +0200] head .bash_history
 6    510  Wednesday  2008-09-03  [18:34:01 +0200] history | head
 7    511  Wednesday  2008-09-03  [18:34:12 +0200] head .bash_history
 8    512  Wednesday  2008-09-03  [18:35:06 +0200] tail -n20 .bash_history | grep -v ^#
 9    513  Wednesday  2008-09-03  [18:35:13 +0200] history | tail
10    514  Wednesday  2008-09-03  [18:36:31 +0200] man history
11    515  Wednesday  2008-09-03  [18:40:22 +0200] history 10
12  sa@sub:~$ history | tail
13    506  Wednesday  2008-09-03  [17:15:46 +0200] source .bashrc
14    507  Wednesday  2008-09-03  [17:18:47 +0200] history -w
15    508  Wednesday  2008-09-03  [18:33:04 +0200] tail .bash_history
16    509  Wednesday  2008-09-03  [18:33:46 +0200] head .bash_history
17    510  Wednesday  2008-09-03  [18:34:01 +0200] history | head
18    511  Wednesday  2008-09-03  [18:34:12 +0200] head .bash_history
19    512  Wednesday  2008-09-03  [18:35:06 +0200] tail -n20 .bash_history | grep -v ^#
20    513  Wednesday  2008-09-03  [18:36:31 +0200] man history
21    514  Wednesday  2008-09-03  [18:40:22 +0200] history 10
22    515  Wednesday  2008-09-03  [18:40:27 +0200] history | tail

As can be seen, lines 1 and 12 actually do the same thing. Nice to note is that, erasedups works just as expected (line 9 is gone after issuing line 12). Note, that history pools the in-memory history whereas

23  sa@sub:~$ tail -n20 .bash_history | grep -v ^#
24  exitexit
25  exit
26  source .bashrc
27  source .bashrc
28  history | tail
29  history -w
30  history | tail
31  tail .bash_history
32  head .bash_history
33  history | head
34  sa@sub:~$

asks for the on-disk history which lags behind in time — we can see that the in-memory (lines 2 to 11 and 13 to 22) history is the same (except for ignoreboth and erasedups actions) with the on-disk history (lines 24 to 33) in lines

  • 1 to 5 respectively 13 to 17 and
  • lines 27 to 33

What is also nice is the result (e.g. Wednesday 2008-09-03 [18:40:27 +0200]) of using export HISTTIMEFORMAT="%A %Y-%m-%d [%T %z] ". So far I am not using HISTIGNORE but I am sure I will some time.


[ a few days went by...]

So, has my bash history grown already i.e. does $HISTFILE contain more than just the default amount of lines/events which we know is 500. Yes! It works just fine as can be seen:

sa@sub:~$ history 12
  764  Thursday  2008-09-04  [20:40:05 +0200] cat /etc/security/time.conf
  765  Thursday  2008-09-04  [20:40:43 +0200] cat /etc/security/access.conf
  766  Thursday  2008-09-04  [20:42:52 +0200] cat /etc/security/limits.conf
  767  Thursday  2008-09-04  [20:55:17 +0200] ban
  768  Thursday  2008-09-04  [21:52:05 +0200] uws
  769  Thursday  2008-09-04  [21:53:12 +0200] ups
  770  Friday  2008-09-05  [07:41:45 +0200] ll
  771  Friday  2008-09-05  [07:41:50 +0200] file alltray.desktop
  772  Friday  2008-09-05  [07:41:54 +0200] cat alltray.desktop
  773  Friday  2008-09-05  [08:39:21 +0200] echo $HISTFILESIZE
  774  Friday  2008-09-05  [08:39:30 +0200] history 22
  775  Friday  2008-09-05  [08:39:58 +0200] history 12
sa@sub:~$

Exploring the History Features

This section is about a tiny portion of all the fun stuff we can do using the Bash (Bourne again shell) history and how we can boost our productivity.

Using Event Designators

I am just providing a few examples. Detailed information can be found with man 3 history and man 1 bash.

sa@sub:~$ echo "white cat"
white cat
sa@sub:~$ !!
echo "white cat"
white cat
sa@sub:~$ !echo
echo "white cat"
white cat
sa@sub:~$ echo "black duck" !#
echo "black duck" echo "black duck"
black duck echo black duck
sa@sub:~$

Search the history using C-r

I strongly believe, this may be the most frequently used feature of history. When we have already executed a very long command, we can simply search history using a keyword and re-execute the same command without having to type it fully again.

Since we played around with some white cat above, pressing Control+R and typing for example white gives us a prompt with echo "white cat" ready to be executed if we hit enter. If we wish to execute echo "red cat", we do not hit enter immediately but left or right arrow, edit the former command i.e. replace white with red and then execute with hitting enter. Wow... cool ha! ^^... I am talking about the red cat of course ;-]

Repeat the previous Command

Sometime we may end up repeating the previous commands for various reasons. Following are the 4 different ways to repeat the last executed command.

  1. Use the up arrow to view the previous command and press enter to execute it.
  2. Type !! and press enter to execute it.
  3. Type !-1 and press enter to execute it.
  4. Press C-P will display the previous command, press enter to execute it.

Execute a specific Command

sa@sub:~$ type uws
uws is aliased to `unison ws_pim_blog.prf'
sa@sub:~$ history | head
    1  Wednesday  2008-09-03  [17:12:34 +0200] man vzcpucheck
    2  Wednesday  2008-09-03  [17:12:34 +0200] man vzmemcheck
    3  Wednesday  2008-09-03  [17:12:34 +0200] man vzsplit
    4  Wednesday  2008-09-03  [17:12:34 +0200] man vzquota
    5  Wednesday  2008-09-03  [17:12:34 +0200] man usermod
    6  Wednesday  2008-09-03  [17:12:34 +0200] man pushd
    7  Wednesday  2008-09-03  [17:12:34 +0200] pushd --help
    8  Wednesday  2008-09-03  [17:12:34 +0200] uws
    9  Wednesday  2008-09-03  [17:12:34 +0200] man rmadison
   10  Wednesday  2008-09-03  [17:12:34 +0200] ssh rh0
sa@sub:~$ type !8
type uws
uws is aliased to `unison ws_pim_blog.prf'
sa@sub:~$ history 5
  515  Wednesday  2008-09-03  [19:39:02 +0200] echo "white cat"
  516  Wednesday  2008-09-03  [19:39:30 +0200] history | head -n22
  517  Wednesday  2008-09-03  [19:40:12 +0200] history | head
  518  Wednesday  2008-09-03  [19:40:36 +0200] type uws
  519  Wednesday  2008-09-03  [19:41:08 +0200] history 5
sa@sub:~$ !515
echo "white cat"
white cat
sa@sub:~$

With the above example, I also wanted to show that substitution (type !8) works just fine — not just picking a command per its numbered history entry (!515). The alerted reader might have also noticed, this works for in-memory as well as on-disk history i.e. for the entire history.

Subtitute words from History Commands

When we are searching through history, we may want to execute a different command but use the same parameter from the command that we have used already.

In the example below, the !!:$ next to the touch command gets the argument from the previous command to the current command.

sa@sub:/tmp/test$ ll
total 0
sa@sub:/tmp/test$ echo tiger
tiger
sa@sub:/tmp/test$ touch !!:$
touch tiger
sa@sub:/tmp/test$ ll
total 0
-rw-r--r-- 1 sa sa 0 2008-09-03 19:51 tiger
sa@sub:/tmp/test$

In the example below, the !^ next to the touch command gets the first argument from the previous command (i.e echo command in our current case) to the current command (i.e touch in our current case.

sa@sub:/tmp/test$ echo blue tiger
blue tiger
sa@sub:/tmp/test$ touch !^
touch blue
sa@sub:/tmp/test$ ll
total 0
-rw-r--r-- 1 sa sa 0 2008-09-03 19:54 blue
-rw-r--r-- 1 sa sa 0 2008-09-03 19:52 tiger
sa@sub:/tmp/test$

So, now we know how to pick the first word from some former executed command but what if we want to pick the last one or the third out of six? No problem, watch me

sa@sub:/tmp/test$ ll
total 0
-rw-r--r-- 1 sa sa 0 2008-09-03 19:54 blue
-rw-r--r-- 1 sa sa 0 2008-09-03 19:52 tiger
sa@sub:/tmp/test$ echo blue tiger fish with green stripes
blue tiger fish with green stripes
sa@sub:/tmp/test$ touch !echo:3
touch fish
sa@sub:/tmp/test$ touch !echo:$
touch stripes
sa@sub:/tmp/test$ ll
total 0
-rw-r--r-- 1 sa sa 0 2008-09-03 19:54 blue
-rw-r--r-- 1 sa sa 0 2008-09-03 20:10 fish
-rw-r--r-- 1 sa sa 0 2008-09-03 20:10 stripes
-rw-r--r-- 1 sa sa 0 2008-09-03 19:52 tiger
sa@sub:/tmp/test$

Security Considerations

Suddenly, for all the fellows scratching their heads over security concerns about what I said above... I have not forgotten you guys ;-]

Basically, we would alter the same variables as we did already but assign quite different values. Before I walk us through this subject, what I did above is providing comfort on boxes as for example workstations and subnotebooks or other machines used for non critical stuff and/or only used by a single person... I would not, and in fact I do not use this type of setup on some server on the Internet simply because of security concerns.

First we need to decide, do we want to limit our actions to all users on a particular system or just to a single user? Whatever one picks, he has to put his magic into the correct files (see above).

The Idea

For security purposes, we should not use a .bash_history file or limit what is recorded into it, because secrets such as passwords, keys, or other sensitive data could possibly be written to the disk — even these days were the Debian installer enables the novice to set up block-layer encryption many folks do not do it... no comment!

Once something is written to the disk, it is theoretically always going to be recoverable — even after deleting the file, writing over the saved data an arbitrary number of times, and physically damaging the platters. The only known way to completely destroy data is to obliterate the platters themselves, i.e. melt 'em down ;-]

The underlying reason is slightly complex and I will not go into detail here. So, if we need to keep our data secure and security is critical or we just want to thwart snooping of our data in the case it is physically stolen then practices like this and lots of good harddrive encryption is the way to go. On another note, under most circumstances it is probably still safe (and useful) to keep history available in our ram memory. Of course, disabling .bash_history means that when we logout or close the terminal session history will be lost. Putting that into code it looks something like this (e.g. in /etc/profile)

export HISTFILESIZE=1
unset HISTFILE
export HISTSIZE=30    #or less
export HISTCONTROL=ignoreboth:erasedups

One might add a HISTIGNORE line to the above (see above). Finally, for the more paranoid ln -s /dev/null ~/.bash_history.

Bash Programming

WRITEME

Miscellaneous

This section is used to collect bits and pieces that simply do not justify to provide them with a dedicated section.

Bash Completion

This goes out to all the lazy animals ;-]... Real wizards are utterly fond of the CLI simply because time matters. So, why not speeding up things a bit more i.e. enabling bash completion. In case we want to enable it for all users on the system, we have to uncomment a few lines in

sa@wks:~$ grep -A4 -m1 "bash completion" /etc/bash.bashrc
# enable bash completion in interactive shells
if [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
fi

sa@wks:~$

In case we want to enable it just for one particular user, we need to alter his ~/.bashrc i.e. also uncommenting those lines responsible for bash completion. Once done,

sa@wks:~$ source .bashrc
sa@wks:~$

makes the new settings valid for our current session i.e. no log-out and log-in necessary as often heard. Ah, and last but not least, one should of course install the package bash-completion.

GIT Bash Prompt

SCM (Software Configuration Management) is a big topic and should be a given for any serious system administrator and/or software engineer.

There is a nice way to have detailed information about a GIT repository displayed within our prompt. The following will use code that is probably installed on our system already anyway plus, it will be fully dynamic i.e. only add information when we enter a directory containing a GIT repository (or any subdirectory thereof) and remove it again when we leave it — it is therefore not going to make our prompt longer/bigger at all times but only when it makes sense.


Assuming we have the package bash-completion installed, the details can be found in /etc/bash_completion.d/git. At first, below is what it looks like before and after we change into a directory containing a GIT repository — note how the Bash prompt automatically changes from sa@wks:~$ to (master *) sa@wks:~/0/0$:

The same as screendump and some more explanation:

 1  sa@wks:~$ cd 0/0
 2  (master *) sa@wks:~/0/0$ type pi; pi git
 3  pi is aliased to `ls -la | grep'
 4  drwxr-xr-x  9 sa sa 4096 Apr 10 15:33 .git
 5  -rw-r--r--  1 sa sa   29 Apr  7 23:59 .gitignore
 6  (master *) sa@wks:~/0/0$ cd

In line 1 we change into a directory under version control with GIT — note how the prompt gets prefixed with (master *). This tells us that we are inside a GIT repository, that we are currently working on the master branch and that we have some uncommitted changes (*). Now, here is what we do in order set up our enhanced Bash prompt so it shows us this additional GIT information:

 7  sa@wks:~$ grep GIT_PS .bashrc
 8          GIT_PS1_SHOWDIRTYSTATE=1
 9          GIT_PS1_SHOWSTASHSTATE=1
10          GIT_PS1_SHOWUNTRACKEDFILES=1
11          GIT_PS1_SHOWUPSTREAM="auto verbose"
12  sa@wks:~$ grep 'set some prompts' -A3 .my_bash_settings
13  ###_. set some prompts
14  if [ $CONN = local -a $USR = notroot ] ; then
15    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;34m\]$(__git_ps1 "(%s) ")\[\033[00m\]\u@\h:\w\$ '
16  elif [ $CONN = local -a $USR = root ] ; then

The important settings are with line 8 to 11 and with \[\033[01;34m\]$(__git_ps1 "(%s) ")\[\033[00m\] from line 15 as the rest of line 15 was in place before already. As mentioned, an explanation of what those settings do can be found in /etc/bash_completion.d/git which is part of the bash-completion Debian package (or whatever other Linux distribution one might use).

GIT + Virtual Environment

For the Pythoneers amongst us, who probably use virtual environments on a daily basis, there is good news too as we can combine both:

Blue is GIT related information, yellow is virtual environment related information. Line 15 from a above adds the GIT part, the thing below adds the virtual environment part to our Bash prompt:

sa@wks:~$ grep PS1 $VIRTUALENVWRAPPER_HOOK_DIR/postactivate
   PS1="\[\033[01;33m\]($(basename $VIRTUAL_ENV))\[\033[00m\] $_OLD_VIRTUAL_PS1"
sa@wks:~$

Some .bashrc customization I did

sa@wks:~$ cat .bashrc
# common settings to all interactive terminal types
if [ "$PS1" ]; then

  # udate window size
  shopt -s checkwinsize

  # make extended globs work
  shopt -s extglob

  # source external files
  source $HOME/.sec/user_name_and_host_name_pair2password
  source $HOME/.my_bash_history_settings
  source $HOME/.my_bash_settings

  # interactive, non-dumb terminal
  if [ "$TERM" != "dumb" ]; then

    # gnupg
    GPG_TTY=`tty`
    export GPG_TTY
    if test -f $HOME/.gpg-agent-info &&    kill -0 `cut -d: -f 2 $HOME/.gpg-agent-info` 2>/dev/null; then
         GPG_AGENT_INFO=`cat $HOME/.gpg-agent-info`
         export GPG_AGENT_INFO
    else
         eval `gpg-agent --daemon`
         echo $GPG_AGENT_INFO >$HOME/.gpg-agent-info
    fi

    # workdir
    export WORKDIR=$HOME/0
    alias cdwork='cd $WORKDIR'

    # mongodb
    export MONGODB=$WORKDIR/mongodb
    export MONGODB_DBPATH=/var/lib/mongodb
    alias cdmongodb='cd $MONGODB'
    alias lsmongodbpath='ls -lah $MONGODB_DBPATH'

    # python
    export PYTHONSTARTUP=$HOME/.pythonrc

    function man () {
    (/usr/bin/man "$@" ||

        python -c "
    try:
        help($1)
    except NameError:
         locals()['$1']=__import__('$1')
         help('$1')" ||

        echo "No manual entry or python module for $1") 2>/dev/null
    }

    # virtualenv/virtualenvwrapper
    export WORKON_HOME=$WORKDIR           #root dir for all virtualenvs
    alias mkvirtualenv='deactivate >& /dev/null; cdwork && mkvirtualenv'
    alias rmvirtualenv='deactivate >& /dev/null; cdwork && rmvirtualenv'

    # pip
    export PIP_DOWNLOAD_CACHE=$HOME/.pip/cache
    export PIP_SOURCE_DIR=$HOME/.pip/source
    export PIP_BUILD_DIR=$HOME/.pip/build
    export PIP_VIRTUALENV_BASE=$WORKON_HOME
    export PIP_REQUIRE_VIRTUALENV=true
    alias cdpipsource='cd $PIP_SOURCE_DIR'
    alias mypipcleaner='rm $PIP_DOWNLOAD_CACHE/* && rm $PIP_SOURCE_DIR/* && rm $PIP_BUILD_DIR/*'

    # pip bash completion start
    _pip_completion()
    {
        COMPREPLY=( $( COMP_WORDS="${COMP_WORDS[*]}" \
                       COMP_CWORD=$COMP_CWORD \
                       PIP_AUTO_COMPLETE=1 $1 ) )
    }
    complete -o default -F _pip_completion pip


    # bpython
    alias bp='/usr/bin/env bpython'
    alias bp3='$(which python3) -m bpython.cli'

    # django
    export DJANGO_COLORS="light"
    export DJANGO=$WORKDIR/django
    export DJANGO_PROJECT_TEMPLATE=$WORKDIR/1/dj-proj-templ
    alias cddjango='cd $DJANGO'


    # bootstrap django project
    alias createdjangoproject="mkdir pip gems &&\
                               cp $DJANGO_PROJECT_TEMPLATE/pip/*.txt pip &&\
                               pip install -r pip/django.txt &&\
                               cp $DJANGO_PROJECT_TEMPLATE/gems/Gemfile gems &&\
                               django-admin.py startproject pr &&\
                               mv pr temp &&\
                               mv temp/* . &&\
                               rm -r temp &&\
                               cp -ain $DJANGO_PROJECT_TEMPLATE/* . &&\
                               cp $DJANGO_PROJECT_TEMPLATE/.gitignore . &&\
                               cp $DJANGO_PROJECT_TEMPLATE/pr/settings.py pr &&\
                               cp $DJANGO_PROJECT_TEMPLATE/pr/urls.py pr &&\
                               gem install bundler &&\
                               cd gems/ && bundle install && cd .. &&\
                               gitflowinit"

    alias cdsr='cd $DJANGO_PROJECT_TEMPLATE'
    alias cdpr='cd $DJANGO_PROJECT_TEMPLATE/pr'


    # git
    GIT_PS1_SHOWDIRTYSTATE=1
    GIT_PS1_SHOWSTASHSTATE=1
    GIT_PS1_SHOWUNTRACKEDFILES=1
    GIT_PS1_SHOWUPSTREAM="auto verbose"

    function gitflowinit () {
        if [ -x $(which git) ]; then
            #echo -e "\n\n\tCreating a GIT repository including the initial commit\n\n"
            # cp $WORKON_HOME/django_env/config/gitignore.txt $PROJECT_ROOT/.gitignore
            # cd $PROJECT_ROOT
            git flow init -d >& /dev/null
            git add .
            git commit -a -s -m 'Initial git add .' >& /dev/null
            echo -e "\n\tResults of initial commit below:\n"
            git whatchanged -n1
        fi
    }


    # aliases
    alias ta='tree    --charset ascii -a        -I \.git*\|*\.\~*\|*\.pyc'
    alias tA='tree    --charset ascii -a'
    alias tap='tree   --charset ascii -ap       -I \.git*\|*\.\~*\|*\.pyc'
    alias td='tree    --charset ascii -d        -I \.git*\|*\.\~*\|*\.pyc'
    alias tad='tree   --charset ascii -ad       -I \.git*\|*\.\~*\|*\.pyc'
    alias tad2='tree  --charset ascii -ad -L 2  -I \.git*\|*\.\~*\|*\.pyc'
    alias tad3='tree  --charset ascii -ad -L 3  -I \.git*\|*\.\~*\|*\.pyc'
    alias tas='tree   --charset ascii -ash      -I \.git*\|*\.\~*\|*\.pyc'
    alias tug='tree   --charset ascii -aug      -I \.git*\|*\.\~*\|*\.pyc'
    alias tan='ta /media/usb0/mm/di'

    eval $(dircolors -b)
    alias ls='ls --color=auto'
    alias l='ls -1 -I "*\.pyc"'           #short
    alias ll='ls -lh -I "*\.pyc"'         #long
    alias la='ls -la'                           #long   show hidden
    alias lss='ls -1sSh'                        #short  show size     sort by size
    alias lsn='ls -1sh | grep -v \~'            #short  show size     do not show file with `~' in their names
    alias lssn='ls -1sSh | grep -v \~'          #short  show size     sort by size   do not show file with `~' in their names
    alias lssa='ls -1sah'                       #short  show size     show hidden
    alias lsssa='ls -1sSah'                     #short  show size     sort by size   show hidden
    alias lt='ls -lrth'                                #long   sort by mtime
    alias lat='ls -larth'                              #long   show hidden   sort by mtime
    alias lst='ls -1rt'                                #short  sort by mtime
    alias lsat='ls -1art'                              #short  show hidden   sort by mtime

    alias sync="rsync    -aSHAXhq --delete"
    alias syncv="rsync   -aSHAXh  --delete --progress --stats"
    alias syncnv="rsync  -aSHAXh  --delete --progress --stats --numeric-ids"
    alias syncsec="rsync -aSHAXh  --delete --progress --stats --numeric-ids --rsh=ssh"
    alias syncaudio="syncsec                                          $HOME/mm/audio/   e98dcca0-1a62-012e-f4bc-32287f7402ab:$HOME/backup$HOME/mm/audio"
    alias syncdocu="syncsec   --exclude-from $HOME/.rsync/exlude_file $HOME/mm/di/docu/ e98dcca0-1a62-012e-f4bc-32287f7402ab:$HOME/backup$HOME/mm/di/docu"
    alias syncmisc="syncsec   --exclude-from $HOME/.rsync/exlude_file $HOME/misc/       e98dcca0-1a62-012e-f4bc-32287f7402ab:$HOME/backup$HOME/misc"
    alias syncsi="syncsec     --exclude-from $HOME/.rsync/exlude_file $HOME/mm/si/      e98dcca0-1a62-012e-f4bc-32287f7402ab:$HOME/backup$HOME/mm/si"
    alias syncwork="syncsec   --exclude-from $HOME/.rsync/exlude_file $WORKDIR/         e98dcca0-1a62-012e-f4bc-32287f7402ab:$HOME/backup$WORKDIR"
    alias syncall="syncaudio && syncdocu && syncmisc && syncsi && syncwork"

    alias wcs="find .  -maxdepth 1 -type f -exec wc -l {} \; | sort -k1,1h | column -t"
    alias wcs2="find . -maxdepth 2 -type f -exec wc -l {} \; | sort -k1,1h | column -t"
    alias wcs3="find . -maxdepth 3 -type f -exec wc -l {} \; | sort -k1,1h | column -t"

    alias cdtmp='cd /tmp'
    alias cdhi='cd $HOME/misc/hi/'
    alias cddocs='cd /usr/share/doc'
    alias cdandroid='cd /media/usb0'

    alias cd='pushd >& /dev/null'    # add directory at DIRSTACK[0] and change into it
    alias dp='popd >& /dev/null'     # remove directory at DIRSTACK[0] and change into it
    alias ds='dirs -v'               # show DIRSTACK with array index
    alias dr='pushd +1 >& /dev/null' # rotate DIRSTACK

    alias dus="du -sh * .* | sort -k1,1h"
    alias strace="strace -rft"
    alias cp="cp -a"
    alias rm="rm -rf"
    alias mkdir='mkdir -p'

    alias gr='grep -rni --color'
    alias pi='ls -la | grep'
    alias psa='ps aux | grep'
    alias pst='pstree -hAcpul'

    alias aps='aptitude search'

    alias acs='apt-cache search'
    alias acsn='apt-cache search --names-only'
    alias acsh='apt-cache show'

    alias afl='apt-file list'
    alias afs='apt-file search'

    alias dpl='dpkg -l'
    alias dps='dpkg -s'
    alias dpp='dpkg -p'
    alias dll='dlocate -l'
    alias dlc='dlocate -lsconf'

    alias tst='scrot -b -d 3 -q 85 -t 25 /tmp/screenshot_$(date +%s).png'
    alias tsw='scrot -b -d 3 -q 85 -t 25 $WORKDIR/0/misc/mm/si/content/$(date +%s).png'

    alias ufs='unison full_sync.prf'
    alias ups='unison part_sync.prf'
    alias ue98dcca0-1a62-012e-f4bc-32287f7402ab="syncsec $WORKDIR/0/ e98dcca0-1a62-012e-f4bc-32287f7402ab:$HOME/backup$WORKDIR/0"
    alias u2fa128b0-1a56-012e-f4be-32287f7402ab="syncsec $WORKDIR/0/ 2fa128b0-1a56-012e-f4be-32287f7402ab:$HOME/backup$WORKDIR/0"
    alias uws='ue98dcca0-1a62-012e-f4bc-32287f7402ab && u2fa128b0-1a56-012e-f4be-32287f7402ab'
    alias ush='unison stage_hn.prf'
    alias uwp='ups && uws'

    alias em='emacs --debug-init &'
    alias evo='nice -n10 evolution >& /dev/null &'
    alias ht='htop'
    alias ia='iceape >& /dev/null &'
    alias iw='iceweasel >& /dev/null &'
    alias cro='chromium >& /dev/null &'
    alias mle='kill `cat ~/.mldonkey/mlnet.pid`'
    alias mls='nice -n15 mlnet &'
    alias pig='pidgin >& /dev/null &'
    alias pin='ping -vc 3 google.com'
    alias pwg='pwgen -sncB 55 1'
    alias san='cd /usr/local/bin/sancho/sancho-0.9.4-59-linux-gtk/ && nice -n15 ./sancho >& /dev/null &'
    alias serveme='python3 -m http.server 12345'
    alias sk='skype >& /dev/null &'
    alias spo='spotify >& /dev/null &'
    alias vlcl='ls -1t | head -n13 | xargs vlc >& /dev/null &'
    alias waf='nice -n19 w3af >& /dev/null &'
    alias wr='workrave &'

    export CONTEXT_PROJECTS=$WORKDIR/context/projects
    alias cdcontext_projects='cd $CONTEXT_PROJECTS'
    alias cmp='$WORKDIR/context/scripts/makeproject.pl'

    export JAVA_HOME="/usr/lib/jvm/java-6-sun/jre"
    alias ccp='compass create --syntax sass --sass-dir "css" --css-dir "css" --javascripts-dir "js" --images-dir "img" --require susy --using susy'
    export LESSOPEN="| /usr/share/source-highlight/src-hilite-lesspipe.sh %s"
    export LESS=' -R '


    function ,, () {
       cd ..
    }


    function xd () {
        cd $(/usr/bin/xd $*)
    }

    mkcd () {
        mkdir -p "$*"
        cd "$*"
    }

  fi

  # xterm
    case $TERM in
    xterm*)
        PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME}: ${PWD}\007"'
        ;;
    *)
        ;;
    esac
fi
Creative Commons License
The content of this site is licensed under Creative Commons Attribution-Share Alike 3.0 License.