Adding program shortcut to user profile (.bash_profile)

To get some background read an aritcle about .bash_profile vs .bashrc by Josh Staiger.

Following example will create an alias for pycharm.sh to be run using pchm.

# cd ~
# vi .bash_profile

Add alias pchm=’cd /home/muneel/Downloads/pycharm-community-2016.3.1/bin;./pycharm.sh’ to the end of file.

# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs

PATH=$PATH:$HOME/bin:/home/muneel/Downloads/pycharm-community-2016.3.1/bin

export PATH

alias pchm='cd /home/muneel/Downloads/pycharm-community-2016.3.1/bin;./pycharm.sh'
# bash .bash_profile
# pchm

Creating symbolic link (soft link) in Linux

119

To create symbolic link navigate to the file directory for which symbolic link will be created.

In following example, symbolic link is created for LiClipse in /usr/bin

# cd /home/muneel/Downloads/liclipse
# ln -s `pwd`/liclipse/LiClipse /usr/bin/liclipse

Note: If `pwd` is not written then full path to file is required in order to create working symbolic link

Verify link by

# ls -l /usr/bin/liclipse 
lrwxrwxrwx. 1 root root 49 Dec 27 01:13 /usr/bin/liclipse -> /home/muneel/Downloads/liclipse/liclipse

Access Linux command cheat sheet from command line

mary-kay-cheating-257x300Yes, Man pages are the best source for reference but quick short reference is always efficient then going through multiple pages.

Cheat is a command line tool by Chris Allen Lane  allowing you to access, create or update cheat sheets from the command line.

Installing Cheat

# yum install python-pip

# pip install docopt pygments

# git clone https://github.com/chrisallenlane/cheat.git
# cd cheat
# sudo python setup.py install

Example

# cheat mount
 To mount / partition as read-write in repair mode:
 mount -o remount,rw /

 Bind mount path to a second location
 mount --bind /origin/path /destination/path

 To mount Usb disk as user writable:
 mount -o uid=username,gid=usergroup /dev/sdx /mnt/xxx

 To mount a remote NFS directory
 mount -t nfs example.com:/remote/example/dir /local/example/dir

 To mount an ISO
 mount -o loop disk1.iso /mnt/disk

Auto Completion

Let’s make it more useful  by enabling command-line auto completion

# wget https://github.com/chrisallenlane/cheat/raw/master/cheat/autocompletion/cheat.bash
# sudo cp cheat.bash /etc/bash_completion.d/

Replay network captures in Linux using ‘tcpreplay’

tcpreplay_logo

Rewrite any source IP address and MAC address in traffic capture to 10.0.10.1 and 00:AB:DD:BB:58:1B. Source IP and MAC address should match with the interface on which traffic needs to be played.

# tcprewrite --srcipmap=192.168.1.10:10.0.10.1 --enet-smac=00:AB:DD:BB:58:1B --infile=capture.pcap --outfile=temp.pcap

if any source IP address needs to be re-written in the traffic capture then use

--srcipmap=0.0.0.0/0:10.0.10.1

Rewrite any destination IP address and MAC address in traffic dump to 10.0.100.20 and 00:AB:DD:BB:50:1A

# tcprewrite --dstipmap=0.0.0.0/0:10.0.10.1 --enet-dmac=00:AB:DD:BB:50:1A --infile=temp.pcap --outfile=final.pcap

After packet rewrite is done, traffic can be played using tcpreplay

# tcpreplay --intf1=eth0 final.pcap

To loop through a pcap file 10 number times, (use 0 for infinite loops or until CTRL-C is pressed)

# tcpreplay --intf1=eth0 --loop=10 final.pcap

If the pcap file(s) you are looping are small enough to fit in available RAM, consider using the – – enable-file-cache option. This option caches each packet in RAM so that subsequent reads don’t have to hit the slower disk.

Another useful option is – – quiet. This suppresses printing out to the screen each time tcpreplay starts a new iteration.

Options

 -q, --quiet                Quiet mode
   -T, --timer=str            Select packet timing mode: select, ioport, rdtsc, gtod, nano, abstime
       --sleep-accel=num      Reduce the amount of time to sleep by specified usec
       --rdtsc-clicks=num     Specify the RDTSC clicks/usec
   -v, --verbose              Print decoded packets via tcpdump to STDOUT
   -A, --decode=str           Arguments passed to tcpdump decoder
   -K, --enable-file-cache    Enable caching of packets to internal memory
       --preload-pcap         Preloads packets into RAM before sending
   -c, --cachefile=str        Split traffic via a tcpprep cache file
   -i, --intf1=str            Server/primary traffic output interface
   -I, --intf2=str            Client/secondary traffic output interface
       --listnics             List available network interfaces and exit
   -l, --loop=num             Loop through the capture file X times
       --pktlen               Override the snaplen and use the actual packet len
   -L, --limit=num            Limit the number of packets to send
   -x, --multiplier=str       Modify replay speed to a given multiple
   -p, --pps=num              Replay packets at a given packets/sec
   -M, --mbps=str             Replay packets at a given Mbps
   -t, --topspeed             Replay packets as fast as possible
   -o, --oneatatime           Replay one packet at a time for each user input
       --pps-multi=num        Number of packets to send for each time interval
   -P, --pid                  Print the PID of tcpreplay at startup
       --stats=num            Print statistics every X seconds
   -V, --version              Print version information
   -h, --less-help            Display less usage information and exit
   -H, --help                 Display usage information and exit
   -!, --more-help            Extended usage information passed thru pager
       --save-opts[=arg]      Save the option state to a config file
       --load-opts=str        Load options from a config file

Simple python logging module wrapper

Simple python logging module wrapper that can be inherited to the class or can be used as a standalone. This logging wrapper supports logging level of INFO, ERROR and DEBUG with option to print to CONSOLE, FILE , BOTH and NONE.  git pull from mlogger repo.

Following is an example

# Inherited to Class
class tester(MLOGGER):
    def __init__(self):
        MLOGGER.__init__(self,'test',logtype='NONE', filename = 'mylogg.log')
    def testmethod(self):
        self.error('error test')
        self.debug('debug test')
        self.warn('warn test')
        self.info('info test')

t = tester()
t.testmethod()

# As Standalone
logger= MLOGGER('Test', level=INFO, logtype = 'CONSOLE', filename = 'mylog.log')
logger.debug('debug test')
logger.info('info test')
logger.warn('warn test')
logger.error('error test')

Creating MPEG-TS (Transport Stream) Segments using FFmpeg

Apple HLS and MPEG-DASH both uses Transport Stream segments.

stream_playlists_2x

Transport Stream can be easily segmented by using ffmpeg.

Here’s the simple way of creating chunks of transport stream for a given segment duration.

$. /ffmpeg -ignore_unknown -i RAImpegCBS.ts -map 0 -codec copy -f segment -segment_time 2 -segment_list out.list segment%3d.ts

ignore_unknown : This is used to ignore any unknown types (SCTE-35, DATA Types). This option may not be available for all the versions of ffmpeg. Check “ffmpeg –help” for list.

i : Input file

map 0 : Maps ALL streams from the first input file to output

codec copy : No conversion/trans-coding of the format. For more codec options check “ffmpeg -codec”

segment_time : Duration of segment

out.list segment%3d.ts : Specifies name of the file listing all the segment and the naming of the segment to be used. In this case segment+3 digit(starting from 0).ts

These segments created by ffmpeg can now be used with either HLS or MPEG-DASH. See Apple’s Documentation on how to create HLS playlist.

Another way to segment is by using ffmpeg hls muxer, for that see hlsbook.net post.

Python Natural Sort (Alphanumeric Sort)

generic-sorting

Natural sort order is an ordering of strings in alphabetical order, except that multi-digit numbers are ordered as a single character. Natural sort order has been promoted as being more human-friendly (“natural”) than the machine-oriented pure alphabetical order.

def atoi(text):
    return int(text) if text.isdigit() else text

def natural_keys(text):
    '''
    alist.sort(key=natural_keys) sorts in human order
    http://nedbatchelder.com/blog/200712/human_sorting.html
    '''
    return [ atoi(c) for c in re.split('(\d+)', text) ]

# MAIN
alist.sort(key=natural_keys)

Simple Linux Watchdog

watchdog

vodolaz095 has a script on GitHub, where he mentions “watchdog.sh script for checking server running not mine, i stole it

I have been using it on my cloud server which runs CentOs 7.