Graduate Program KB

CLI navigation

  • pwd Displays current path location

  • ls Displays folders and files within the current directory

    • .. Performs ls on parent directory
    • . References current directory, no changes
    • ~ Perform ls on home directory
    • / Perform ls on root of the path
    • -l Display more information, such as permissions and size
    • -a Display hidden files (prefixed with .)
  • cd Change current working directory

    • .. Change to parent directory
    • . References current directory, no changes
    • ~ Change to home directory
    • / Change to root path directory

Tips

  • Up/Down arrow keys to revisit old commands
  • Tab for autocomplete
  • Ctrl + R for reverse searching commands, Left/Right arrow keys to return to CLI with selected command
  • ~/.bash_history is a file appending commands from terminal, beware of sensitive information
  • !! Runs previous command
  • clear Moves cursor to top of terminal to reduce clutter
  • Ctrl + Shift + C and Ctrl + Shift + V Copy/paste in terminal, standard shortcut is already assigned

Shortcuts

  • Ctrl + A Moves cursor to beginning of line
  • Ctrl + E Moves cursor to end of line
  • Ctrl + K Deletes everything after cursor, the contents are 'yanked'
  • Ctrl + U Deletes everything before cursor, the contents are 'yanked'
  • Ctrl + Y The 'yanked' contents are 'pasted'
  • Ctrl + L Clear the screen

Signals

  • Signals are notifications sent to programs, program decides what to do with it but generally abides
  • Ctrl + C Sends SIGINT to interrupt a program
  • Ctrl + D Sends SIGQUIT, not used as much but can end sessions on bash servers
  • SIGTERM No shortcut but ued by kill command to terminate programs
  • SIGKILL No shortcut but used by kill -9 or kill -SIGKILL to stop a program's runtime

Text editors

  • nano {file}

    • Commands listed at bottom of interface, ^ represents Ctrl and paired with a letter to form a shortcut
  • vim {file}

    • Esc Enter command mode (default mode)
    • i Enter insert mode from command mode
    • :w Save file
    • :q Quit file
    • :wq Save and quit file
    • :q! Quit file without saving

Interacting with files

  • less Enter read mode for a file

  • man Displays manual, most programs have one

  • cat Reads file and outputs to terminal

  • head or tail Read first 10 or last 10 lines of a files

    • -n={number} to specify number of lines
  • mkdir Creates a new folder.

  • mkdir -p Creates a nest of folders

  • touch Creates a new empty file

    • If file exists then it manipulates the 'last modified' and 'last access' times
  • rm Remove files

    • -r Remove empty folders
    • -rf Remove all types of files without needing confirmation
    • WARNING: Do not run rm -rf /, deletes everything from root of the system
  • cp Copy

    • Copy contents from file to file
    • Copy file to directory
    • -R Recursively copy directory to another directory
  • mv Move file or folders to another location

  • tar Group files into a single file (tarball, similar to zip file)

    • -z Compress the files
    • **-zcf name.tar.gz {file} ... {folder} Compress and combine files/folders
    • **-xzf name.tar.gz -C {dest-folder} Extract

Wildcards and expansions

  • Manipulate multiple files at once

  • * Represents the remainder of a file name

  • ? Represents exactly one character

  • [ ] Represents limited number of characters

    • Ex. ls file[1-3].txt or use ^ (not) ls file^[1-3].txt
  • Some expansions:

    • {a..z} a to z
    • {z..a} z to a, reverse order
    • {0..100..2} Every even number between 0 to 100 inclusive
    • {100..0..5} Every 5th number fromm 100 to 0 inclusive
    • {a..z}{1..5} a1, ..., a5, b1, ..., b5, c1, ..., c5, etc

Streams and pipes

  • Standard output - stdout

    • 1> Redirect output and overwrite the file
    • 1>> Redirect output and append to the file
  • Standard error - stderr

    • 2> Redirect error output instead of standard output
    • 2>> Redirect error output and append to file
Both the standard output and error streams can be redirected at the same time, can be useful if you want to separate error messages and normal output
  • /dev/null - Basically the void

  • Any output redirected to /dev/null is discarded

  • Standard input - stdin

    • < Redirect contents of a file to a program
  • Pipes can redirect the output of one program as input to another

    • | Program on the left of pipe will direct its output to the program on the right
    • Ex. ps aux | grep 'ps aux'

Users, groups and permissions

  • Users are stored in /etc/passwd

  • whoami Displays current user of the system

  • Superuser

    • sudo Switch user and do, perform command as root user then switch back to normal user
    • sudo su Switch to user
    • sudo useradd Creates new user
    • sudo passwd Adds password to user
  • Permissions of files and folders follow the format:

    • d rwx rwx rwx

    • The first character is either 'd' or '-' for directory/file

    • The groups represent permissions for users that own the file, groups that own the file and everyone else respectively

    • 'r' - Read

    • 'w' - Write

    • 'x' - Execute

    • '-' - No permissions

  • Can directly assign permissions:

    • Non-binary method:

      • Ex. sudo chmod u=rwx, g=rwx, o=rwx {file}
    • Binary method:

      • Ex. sudo chmod 777 {file}
      • Add 4, 2 or 1 for adding read, write or executable permissions respectively, else just set to 0
      • Can also use + or - to apply/remove permissions from all groups
      • Ex. chmod +x {file} Adds executable permissions for all groups

Environments

  • printenv Displays all environment variables, variables can be accessed with $

  • Set variables:

    • Variables will revert after session ends
    • {variable-name} = {value}
  • Set permanently:

    • Configure ~/.bash_profile
    if [ -f ~/.bashrc ]; then
    	source ~/.bashrc
    fi
    
    • ~/.bashrc will be run whenever a new session is created which loads all environment variables
    • Set variables in ~/.bashrc
    • export {variable-name}={value}

Processes

  • ps Display running processes
  • ps aux Display all running processes on the system
  • {command} & Append & to run a command in the background
  • jobs List status of background processes currently running, use with -l to obtain process ID
    • Listed jobs are assigned a number
    • bg {number} Set to run in the background
    • fg {number} Set to run in the foreground, will have to interrupt if you want to run any other commands

Exit codes

  • Any other code than 0 means fail, it can range from 0 to 256
  • Common exit codes:
    • 0 : Successful
    • 1 : General catch-all highlighting an error
    • 2 : Bash internal error, you or program used bash incorrectly
    • 126 : No permissions or file not executable
    • 127 : Command not found
    • 128 : Exit command had a problem, usually provided non-integer exit code
    • 130 : Interrupted program with Ctrl + C
    • 137 : Ended program with SIGKILL
    • 255 : Out-of-bounds, tried to exit with code larger than 255

SSH

  • ssh-keygen -t rsa Create public and private key files stored in ~/.ssh, set a passphrase then confirm
  • ssh user@ip-address
  • ifconfig Displays network information

wget and curl

  • wget {url}

  • curl {url}

  • curl -I {url} Only checks if server is ready to respond

  • curl -X {verb} {url} HTTP verbs like POST, PUT, DELETE and PATCH

  • curl -d {contents} {url} Send a POST body

  • curl -b {cookies} {url} Send cookies with request

  • curl -c {cookies-file} {url} Put lots of cookies in a file and send with request

  • curl -L {url} Follows redirect links such as bit.ly or other URL shorteners

  • curl -H {header} {url} Include headers

Package management

  • apt is newer than apt-get, generally ues apt if possible

  • apt install {package}

  • apt autoremove Removed unused dependencies

  • apt update Update available packages apt users

  • apt list List of packages installed

  • apt list --upgradable List packages installed with an update available

  • apt upgrade Update all installed packages to the latest version

  • apt full-upgrade Autoremove and upgrade functionalities combined

  • Snaps are a new way to package apps

    • Update automatically based off version difference
    • Sandboxed from rest of system
    • No need to review
  • snap install {package}

  • snap info {package} Display version details of package

Scripts

  • Shell scripts can run many commands at a time

  • . {bash-file} Runs script

  • Hashbang

    • Execute without bash, run using ./{bash-file}
    • Add to first line of file:
    #! /bin/bash
    
  • PATH is a series of paths where programs are stored

  • Add programs to PATH:

    cd ~
    mkdir bin
    mv <bash file> bin/<bash file>
    PATH=~/bin:$PATH
    
  • To permanently keep custom PATH, add to ~/.bashrc similar to environment variables

    PATH=~/bin:$PATH
    
  • In bash, you can assign values to variables like anything else:

    DESTINATION=~/temp
    mkdir -p $DESTINATION
    
  • Use read to retrieve user input and store into a variable. Can append -p to prompt user with a text for more description

    read -p 'Enter your name: ' name
    echo $name
    
  • Arguments can be passed into the program and stored in $1 for a single argument

    • $0 will be the file, and any subsequent $2, $3, etc will store any other arguments if two or more were provided
    ./<bash file> <name>
    
    name=$1
    echo $name
    
  • Use special notation [ ] to test commands, used to evaluate conditional statements

    • If:
    read -p 'Enter your name: ' name
    
    if [ -z $name ]; then
    	echo 'Empty name'
    fi
    
    • If, else if, else:
    read -p 'Enter your name: ' name
    
    if [ -z $name ]; then
    	echo 'Empty name'
    elif [ $name = 'Brian' ]; then
    	echo 'Name is Brian'
    else
    	echo $name
    fi
    
    • Case:
    read -p 'Enter your name: ' name
    
    case $name in
    	'Brian')
    		echo ':)'
    		;;
    	'Bob')
    		echo ':('
    		;;
    	'John')
    		echo ':|'
    		;;
    	*)
    		echo $name
    		;;
    esac
    
    • For loops:
    #! /bin/bash
    
    names=(Brian Bob 'John Smith')
    
    for name in ${names[*]}
    do
    	echo $name
    done
    
    echo "Number of names: ${#names[*]}"
    
    • While loops:
    NUM_TO_GUESS=$(( $RANDOM % 10 + 1 ))
    GUESSED_NUM=0
    
    echo "Guess a number between 1 and 10"
    
    while [ $NUM_TO_GUESS -ne $GUESSED_NUM ]
    do
    	read -p "Your guess: " GUESSED_NUM
    done
    
    echo "Congratulations!"
    
  • Some command operators:

    • -eq Equals
    • -ne Not equals
    • = Returns 0 if true
    • != Returns 1 if false
    • -gt Greater than
    • -le Less than
    • -e {file} Checks file exists
    • -w Checks file exists and writable
    • -z Checks if variable is empty