Bash Scripting - Best Practices

1 - Readability

1.1 - Indentation

There are 3 commonly used indentation practices for Bash (I prefer the first method, however all 3 are "accepted"):

  • 2 spaces
  • 4 spaces
  • tabs (usually 8 spaces)

All examples will be shown using the first indentation method, however for reference here's a comparison between all 3.

Example:

## 2 spaces
if ...
  command
else ...
  command
fi
## 4 spaces
if ...
    command
else ...
    command
fi
## Tabs
if ...
        command
else ...
        command
fi

Indentation for if conditional statements

if [ test ] ; then
  command
elif [ test ] ; then
  command
else
  command
fi

Indentation for for statements

for a b c in $A ; do
  command
done

Indentation for while and until loops

while [ true ] ; do
  command
done

Indentation for case statements

case $VAR in
  true) command ;;
  false)
    command1
    command2
    ;;
  *)
    if [ test ] ; then
      command
    fi
    ;;
esac

Indentation for functions

_funct_do_var() {
  commands
}

1.2 - Comments

Code Comments

You should always comment your code. This will make it easier for others to understand, as well as for yourself should you need to change the script a few months (or years) later.

Make sure that the comments make sense. Do not try to save on typing as additional comments can save you (or someone else) a lot of time in the future.

Example: Commonly used comments

startTMATE() {
  ## Starts tmate handling
  # Launch tmate in a detached state
  $TMATE_BIN -S $TMT_SOCKET new-session -d
  # Blocks until the SSH connection is established
  $TMATE_BIN -S $TMT_SOCKET wait tmate-ready
  # Prints the SSH connection string
  $TMATE_BIN -S $TMT_SOCKET display -p '#{tmate_ssh}' > $LOG
  # Prints the read-only SSH connection string
  $TMATE_BIN -S $TMT_SOCKET display -p '#{tmate_ssh_ro}' >> $LOG
}

Example: In-line comments

## Sets up package manager aliases based on distro
# Settings for Ubuntu
if [[ "$DISTRO" =~ (Ubuntu|LinuxMint) ]] ; then
  alias aptdate='sudo apt-get update'         # Updates package list
  alias aptgrade='sudo apt-get upgrade'       # Updates all packages
  alias apts='apt-cache search'               # Search for package
  alias aptrm='sudo apt-get remove'           # Removes package
  aptinst() {                                # Install package
    if [ -d "${HOME}/bin/var/log" ] ; then
      LOG=${HOME}/bin/var/log/$(hostname)-package-install.log
    fi
    echo -e "$(date)\t-\tInstalling packages: $*" >> "$LOG"
    sudo apt-get install "$*"
  }
fi

Block Comment

Here Document

You can use a here document (EOF) to "trick" Bash in creating block comments.

Example:

alt text

You can read a bit on my other post here.

Bash Built-in Command

You can also use Bash's bultin :, which does nothing.

Here's the definition from Bash's man page:

: [arguments]
    No effect; the command does nothing beyond expanding arguments and
    performing any specified redirections. A zero exit code is returned.

Example:

: '
 your comments here
'

1.3 - Lines

Breaking Line of Code

When breaking long lines of code with \, indent the new lines.

Example:

[ -f "/etc/cups/cupsd.conf" ] && echo \
  "The CUPS configuration file exists" 

Using Empty Lines

You can use empty lines to keep your code clean, even inside code blocks.

Example:

# Downloads this weeks photos
if [ "$THIS_WEEKS_PHOTOS_URL" ] ; then
  cd ${UNSPLASH_DIR}
  for PHOTO_URL in $THIS_WEEKS_PHOTOS_URL ; do
    # Checks if jpg is in the URL
    if [[ $(echo $PHOTO_URL | grep -qi jpg ; echo $?) -eq 0 ]] ; then
      PHOTO_FILE_NAME=$(echo $PHOTO_URL | awk -F/ '{print $4}')
    # else, let's add it
    else
      PHOTO_FILE_NAME=$(echo $PHOTO_URL | awk -F/ '{print $4}').jpg
    fi
    # Get the photo and save as file name
    wget --progress=bar $PHOTO_URL -O $PHOTO_FILE_NAME
  done
fi

1.4 - Misc

  • Avoid the use of back-ticks ` for command substitution. Use $(...) for better readability

2 - Headers and Sections

2.1 - Script Header

Add descriptive headers to the script outlining it's name, what it does, usage and possibly version.

#!/bin/bash
################################################################################
################################################################################
# Name:          tmate.sh
# Usage:
# Description:   Runs tmate and outputs remote string to file
# Created:       2014-10-31
# Last Modified:
# Copyright 2014, Victor Mendonca - http://wazem.org
# License: Released under the terms of the GNU GPL license
################################################################################
################################################################################

2.2 - Sections

Divide your script into sections.

Put all global script variables in one section

#-------------------------------------------------------------------------------
# Sets variables
#-------------------------------------------------------------------------------

Put all functions in one section

#-------------------------------------------------------------------------------
# Functions
#-------------------------------------------------------------------------------

Put all major work in one section (where functions get called)

#-------------------------------------------------------------------------------
# Starts script
#-------------------------------------------------------------------------------

3 - Variables

3.1 - Naming

  • Give meaningful names to variables
  • When using uppercase, make sure the variable is not already being used
  • Avoid starting variables with _

3.2 - Calling

Double quote variables to prevent globbing or word splitting.

4 - Functions

Try to define your functions at the beginning of the script (see sections).

4.1 - Naming

  • Should start with lower case
  • It's good to start with _
  • Upper and lower case are also good

Example:

_lowerIt () {
  BASH_MAIN_VERSION=$(echo $BASH_VERSION | awk -F. '{print $1}')
  if [[ $BASH_MAIN_VERSION -lt 4 ]] ; then
    echo "Not supported with your version of bash"
    return
  fi
  if [ "$1" ] ; then
    echo ${1,,}
  fi
}

4.2 - Local Variables

Keep your function variables local if they are not being used outside of the function.

Example: This script will output "$2400" only once

_myFunction() {
  local extract
  extract='$2400'
  echo "$extract"
}
_myFunction
echo "$extract"

5- Code Smart

New and Deprecated Features

Get to know your version of Bash and any features it may have.

Example: Old and new arithmetic expansion

# New way for arithmetic expansion
$ echo $((4*25/2))
50
# Old way for arithmetic expansion, which will be deprecated
$ echo $[4*25/2]
50

New features

Bash 4 packs new features (like parameter expansion and output redirects). Get familiar with them as they can save you a lot of time.

Examples: Parameter expansion

$ title="bash best practices"
## Case substitution
$ echo ${title^^}
BASH BEST PRACTICES
## String substitution
$ echo ${title/best/worst}
bash worst practices

Examples: stdout and stderr output redirect

# New
ls 123 &> /dev/null
# Old
ls 123 > /dev/null 2>&1

6 - Software

You can also use software to help with your scripts.

6.1 - Syntax Highlighting

Most of today's editors include syntax highlighting, even VIM. Syntax highlighting will help you catch mistakes like a unclosed bracket or quote.

6.2 - Lint

Some editors, like Sublime Text include plugins for lint software. They can be of great help for you coding.

6.3 - Online Tools

There are also sites that can help checking your script (similar to lint), or a small set of commands.

  • explainshell.com - match command-line arguments to their help text
  • shellcheck.net - automatically detects problems in sh/bash scripts and commands

References

How to Send Pushbullet Notifications from Shell

If you use Pushbullet, you know how great the app is for notification. However you may not know that you can also use Pushbullet's API on your computer to send notifications. This comes pretty handy if you are running a script and wants to get some type of confirmation from it.

For example, I have a Raspberry Pi at home that I use for motion detection with IP cameras (with Motioneye). This server monitors Wifi clients and automatically disables the alerts when I'm home, and re-enables them when I leave. I use Pushbullet to get notifications when this happens.

To start, login to your account on Pushbullet and go to Settings.

pushbullet

You should see your "Access Token" on the bottom right (like the image below). Copy that as we will need it.

pushbullet2

Now head to GitHub repo for pushbullet-bash and download the script.

Set your PushBullet API key by creating the file $HOME/.config/pushbullet and adding the linePB_API_KEY= with your "Access Token".

The script has the following options:

Actions:
list - List all devices and contacts in your PushBullet account. (does not require
       additional parameters)
push - Push data to a device or contact. (the device name can simply be
       a unique part of the name that "list" returns)
pushes active - List your 'active' pushes (pushes that haven't been deleted).
delete $iden - Delete a specific push.
delete except $number - Delete all pushes except the last $number.
delete all - Delete all pushes.


To view your registered devices use the list option:

pushbullet3

And to send a notification to a device, use the push note option. For example, here's how I would send a notification to my cellphone:

$ pushbullet.sh push "Note 4" note "$HOSTNAME" "Your script is complete"
Success

 

pushbullet4

How to Multiline Comments in Bash

One of the things I always missed in Bash scripting was the lack of multiline comments. Well, not anymore. I found the other day that you can trick Bash by using a here file as multiline comment.

<<MY_COMMENT
  Anything in here is a comment.
MY_COMMENT

Just make sure you don’t mention MY_COMMENT anywhere else in the code.

Screenshot from 2015-07-31 171435

How to Disable Unity's Global Menu - 14.04

When it comes to disabling Unity's global menu, you have two options:

Disable the global menu for specific apps 

 

Change the global menu to show in the windows title bar (all apps)

 

 

1- Disable the global menu for specific apps

 

Make sure you have dconf-editor installed. You can install it with:

sudo apt-get install dconf-editor


Open dconf-editor and browse to com.canonical.unity-gtk-module.

Enter the name of each application under the blacklist key. They should be in the following format:

['app1', 'app2', 'app3']

 


Restart the app and you should see it the new menu's.

2- Change the global menu to show in the windows title bar (all apps)

Go to 'System => Appearance' and click on the 'Behaviour' tab. Select the radio box for 'In the window's title bar'

Android Media Player

Love Android? How about running Android as a 4k TV media player? Minix came up with a new hardware version of their Android media player this summer. The little device is packing a 4 core processor with 2GB of DDR3 and a 16GB of storage.

Check out the video below for an in depth review:

Read more...

Ubuntu Mate 14.04

Are you not happy with the new look and feel of Unity and Gnome? Do you miss the way Gnome used to look? Then you need to check out Ubuntu Mate. The LTS version (14.04) was just released, and it looks great.

https://ubuntu-mate.org/

Read more...

How to Quickly Secure CentOS 6.5

This is a quick tutorial on how to secure CentOS 6.5. It does not go into details and should only be used as quick solution.

1. Enable Auto Updates

Install “yum-cron”

yum -y install yum-cron

Review the config file

vim /etc/sysconfig/yum-cron

Start the service

/etc/init.d/yum-cron start

Set the service to auto start

chkconfig yum-cron on

2. Securing root

You should not be logging in as root. So first lets create a user

# useradd [user] 

Now let's give sudo access to that user by calling visudo and adding your user (last line)

## Allow root to run any commands anywhere 
root    ALL=(ALL)   ALL
[user]  ALL=(ALL)   ALL

Restrict root login to tty1

echo "tty1" > /etc/securetty

Remove read access from /root

chmod 700 /root

3. Securing SSH

Let's setup your SSH keys. See this tutorial on how to. Make sure you create the keys as the user and not root.

Make sure you can login as the user with the new keys and without being prompted for a password.

Now let's edit /etc/ssh/sshd_config as root and make the following changes:

  • Enable IPv4 only
  • Root cannot login via SSH
  • Users cannot login with password (we will be using the SSH keys)
  • Only allow a specific user to login
AddressFamily inet
PermitRootLogin no
PasswordAuthentication no
AllowUsers [user]

4. Disable IPv6

Disable IPv6 if you are not using it. Edit /etc/sysctl.conf by adding the lines below to the end of the file

# Disable IPv6
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1

5. Setup TCP Wrappers

This section will only be used if your server is accessed by a known list people from specific locations. If your server needs to be open to the public, skip this part.

Find out what IPs your ISP uses (you can use a site like http://ipchicken.com/ to find out your current IP). Do this on all the devices you will use to connect to this server (either via SSH, telnet, web browser, sftp, anything), including your phone.

Edit /etc/hosts.allow by adding a ALL: followed by the first set of digits of the IP you will use. For example, my file allows me to access from my ISP at home, by VPN and from my work.

# (Rogers)
ALL: 99.
# VPN 
ALL: 192.50.
# Work
ALL: 226.

You can also limit it to a service, like SSH and httpd

httpd: 99.
sshd: 192.50

Now add a ALL: ALL to /etc/hosts.deny

ALL: ALL

How to Install tmate Server on CentOS 6.5

1- Install required packages for tmate and for compiling

 

Use group install for development tools. It will install all the following pakcages for you: autoconf automake binutils bison flex gcc gcc-c++ gettext libtool make patch pkgconfig redhat-rpm-config rpm-build rpm-sign

# yum groupinstall 'Development Tools'

Then install other required packages for tmate

# yum install git kernel-devel zlib-devel openssl-devel ncurses-devel cmake ruby libssh-devel

Check if libevent is installed and the version. We need libevent2

# rpm -qa | grep libevent
libevent-devel-1.4.13-4.el6.x86_64
libevent-doc-1.4.13-4.el6.noarch
libevent-headers-1.4.13-4.el6.noarch
libevent-1.4.13-4.el6.x86_64

If the version is older, remove it

yum remove libevent libevent-devel libevent-headers

Download libevent2

wget https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
tar xf libevent-2.0.21-stable.tar.gz
cd libevent-2.0.21-stable

Compile it and install

./configure
make
make install

Create a symbolic link if you are on a 64-bit server

ln -s /usr/local/lib/libevent-2.0.so.5 /usr/lib64/libevent-2.0.so.5

2- Installing tmate-slave

 

Now download tmate-slave

# git clone https://github.com/nviennot/tmate-slave.git
Initialized empty Git repository in /root/Downloads/tmate-slave/.git/
remote: Counting objects: 23464, done.
remote: Total 23464 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (23464/23464), 18.50 MiB | 17.14 MiB/s, done.
Resolving deltas: 100% (13397/13397), done.

Change into the new directory and create the SSH keys

# cd tmate-slave/ $$ ./create_keys.sh 
Generating public/private dsa key pair.
Your identification has been saved in keys/ssh_host_dsa_key.
Your public key has been saved in keys/ssh_host_dsa_key.pub.

Save the 3x keys (dsa, rsa and ecdsa)

The key fingerprint is:

Install tmate-slave

# ./autogen.sh
# ./configure
# make

3- Starting tmate-slave

 

Move your tmate folder if you want. I have mine under $HOME/bin

[root@localhost bin]# mv ../Downloads/tmate-slave/keys/ .

If SSH is running, select another port and start tmate

# ./tmate-slave -k keys -l ../var/log/tmate.log -p 224 &

4- Configuring the client to use your server

 

Create $HOME/.tmate.conf with the following

set -g tmate-server-host "[your server FQDN]"
set -g tmate-server-port [server port]
set -g tmate-server-dsa-fingerprint   "dsa fingerprint"
set -g tmate-server-rsa-fingerprint   "rsa fingerprint"
set -g tmate-server-ecdsa-fingerprint "ecdsa fingerprint"
#set -g tmate-identity ""              # Can be specified to use a different SSH key

How to Make SSH Tunnel Available on the Network

Let's say you created a reverse tunnel to a remote computer/server and you want other computers on the same network to be able to connect to that tunnel. Sometimes this will work out of the box, but other times it requires additional changes.

On this example, I have created a reverse tunnel on port 8080 to my desktop compute with the command below:

ssh -R 8080:127.0.0.1:8080 [user]@desktop

However other computers on the same network cannot connect to it. When I check my desktop, I see that it's listening on that port, however it does not seem to be open (0.0.0.0)

# netstat -an | grep 8080
tcp        0      0 127.0.0.1:8080              0.0.0.0:*                   LISTEN

1- Check your /etc/ssh/sshd_config and make sure you have GatewayPorts uncommented and enabled:

GatewayPorts yes

Test your connection and see if that works. If it didn't, create the SSH tunnel again with one of the following options:

ssh -R \*:8080:127.0.0.1:8080 [user]@desktop

or

ssh -R 0.0.0.0:8080:127.0.0.1:8080 [user]@desktop

Most Commonly Used IRC Commands

User Commands

Joining

Join a channel

/join <#channel>

Leaving

Leave chat/channel

Makes you leave the specified channel.

/part <#channel> <message>

Closes inactive, chat, fserve, get, message or send windows.

/close

Closes the query window you have open to the specified nick.

/closemsg <nickname> 

Leave network

Disconnect you from IRC with the optional byebye message.

/quit [reason] 

Sending Messages

Direct message to user

<nick> <message>

Send a private message to user without opening a query window.

/msg <#channel|nickname> <message> 

Open a query window with user and send them a private message

/query <nickname> <message> 

Send the specified message to all ops on a channel.

/omsg [#channel] <message> 

User Status

Sets away message

/away <msg>

Sends the specified action to the active channel or query window.

/me <action text> 
/me shakes fist in the air in frustration about learning IRC.

User Identification

Register a nick

a. Register your nick

/nick <user1>
/msg NickServ REGISTER <password> <email-address>

b. Hide your email

/msg NickServ SET HIDEMAIL ON

c. Force server to ask password for your nick

/msg NickServ SET ENFORCE ON

d. It's recommended to setup a second nick

/nick <user2>

e. Make sure you are still logged in as user1

/msg NickServ IDENTIFY <user1> password

f. Group the two nicks

/msg NickServ GROUP 

Login with your registered used ID

/msg NickServ IDENTIFY <nick> <password>

Disconnects an old session

/msg NickServ REGAIN <nick> [password]
/msg NickServ GHOST <nick> [password]

View yours or other user's info

/msg NickServ info #shows your info
/msg NickServ info <nick>

Channel Commands

Register a channel

Note: make sure you are logged in

a. Find if channel is available

/msg ChanServ info <channel>

b. Create and join your new channel

/join <channel>

c. Register the channel to your nick

/msg ChanServ register <channel>

Setup a channel greeting message

/msg ChanServ SET <channel> ENTRYMSG <message>

Change channel mode

/msg ChanServ SET <channel> MLOCK [+|-]<mode>
i       invite-only
k       channel password
l       join limit
m       moderated, only OPs can post
p       private
s       secret channel
t       only ops can change topic

Lock topic

/msg ChanServ SET <channel> TOPICLOCK ON

ChanServ help commands

/msg ChanServ help commands
-ChanServ-  
-ChanServ- The following commands are available:
-ChanServ- FLAGS           Manipulates specific permissions on a channel.
-ChanServ- INVITE          Invites you to a channel.
-ChanServ- OP              Gives channel ops to a user.
-ChanServ- RECOVER         Regain control of your channel.
-ChanServ- REGISTER        Registers a channel.
-ChanServ- SET             Sets various control flags.
-ChanServ- UNBAN           Unbans you on a channel.
-ChanServ-  
-ChanServ- Other commands: ACCESS, AKICK, CLEAR, COUNT, DEOP, DEVOICE, 
-ChanServ-                 DROP, GETKEY, HELP, INFO, QUIET, STATUS, 
-ChanServ-                 SYNC, TAXONOMY, TEMPLATE, TOPIC, TOPICAPPEND, 
-ChanServ-                 TOPICPREPEND, UNQUIET, VOICE, WHY
Thursday the 27th.
Copyright 2012

©