Shell Scripting Fundamentals: Your Complete Guide to Bash Automation

Shell scripting is the backbone of automation in Unix-like systems. Whether you’re a system administrator, DevOps engineer, or developer, mastering shell scripting will significantly boost your productivity and automation capabilities.

Table of Contents

Getting Started

Your First Script

Every shell script starts with a shebang (#!) that tells the system which interpreter to use:

#!/bin/bash
# This is your first shell script

echo "Hello, World!"
echo "Today's date is: $(date)"

Save this as hello.sh, make it executable, and run it:

chmod +x hello.sh
./hello.sh

Making Scripts Portable

For maximum compatibility, use the env command:

#!/usr/bin/env bash
# This finds bash wherever it's installed

Variables and Data Types

Basic Variables

#!/bin/bash

# String variables
name="John Doe"
project="Shell Scripting Tutorial"

# Numeric variables
count=42
version=1.5

# Using variables
echo "Author: $name"
echo "Project: ${project}"
echo "Count: $count"

Environment Variables

#!/bin/bash

# Reading environment variables
echo "Current user: $USER"
echo "Home directory: $HOME"
echo "Current path: $PWD"

# Setting environment variables
export MY_VAR="Hello World"

Command Substitution

#!/bin/bash

# Old style (backticks)
current_date=`date`

# New style (preferred)
current_date=$(date)
files_count=$(ls -1 | wc -l)

echo "Current date: $current_date"
echo "Files in directory: $files_count"

Control Structures

Conditional Statements

#!/bin/bash

# Basic if-else
if [ "$1" = "start" ]; then
    echo "Starting service..."
elif [ "$1" = "stop" ]; then
    echo "Stopping service..."
else
    echo "Usage: $0 {start|stop}"
    exit 1
fi

# File testing
file="$1"
if [ -f "$file" ]; then
    echo "$file exists and is a regular file"
elif [ -d "$file" ]; then
    echo "$file exists and is a directory"
else
    echo "$file does not exist"
fi

Loops

#!/bin/bash

# For loop with range
echo "Counting from 1 to 5:"
for i in {1..5}; do
    echo "Number: $i"
done

# For loop with array
fruits=("apple" "banana" "orange")
for fruit in "${fruits[@]}"; do
    echo "I like $fruit"
done

# While loop
counter=1
while [ $counter -le 3 ]; do
    echo "Iteration: $counter"
    ((counter++))
done

# Reading file line by line
while IFS= read -r line; do
    echo "Line: $line"
done < "input.txt"

Case Statements

#!/bin/bash

action="$1"
case $action in
    start)
        echo "Starting application..."
        ;;
    stop)
        echo "Stopping application..."
        ;;
    restart)
        echo "Restarting application..."
        ;;
    status)
        echo "Checking status..."
        ;;
    *)
        echo "Unknown action: $action"
        echo "Usage: $0 {start|stop|restart|status}"
        exit 1
        ;;
esac

Functions

Basic Functions

#!/bin/bash

# Function definition
greet() {
    local name="$1"
    local time="$2"
    echo "Good $time, $name!"
}

# Function with return value
is_file_exist() {
    local file="$1"
    if [ -f "$file" ]; then
        return 0  # success
    else
        return 1  # failure
    fi
}

# Using functions
greet "Alice" "morning"

if is_file_exist "/etc/passwd"; then
    echo "Password file exists"
fi

Advanced Function Example

#!/bin/bash

# Backup function with error handling
backup_directory() {
    local source_dir="$1"
    local backup_dir="$2"
    local timestamp=$(date +"%Y%m%d_%H%M%S")
    
    # Validate parameters
    if [ $# -ne 2 ]; then
        echo "Usage: backup_directory <source> <destination>"
        return 1
    fi
    
    if [ ! -d "$source_dir" ]; then
        echo "Error: Source directory '$source_dir' does not exist"
        return 1
    fi
    
    # Create backup
    local backup_name="backup_${timestamp}.tar.gz"
    local full_backup_path="${backup_dir}/${backup_name}"
    
    echo "Creating backup: $full_backup_path"
    tar -czf "$full_backup_path" -C "$(dirname "$source_dir")" "$(basename "$source_dir")"
    
    if [ $? -eq 0 ]; then
        echo "Backup completed successfully: $full_backup_path"
        return 0
    else
        echo "Backup failed"
        return 1
    fi
}

File Operations

Reading and Writing Files

#!/bin/bash

# Writing to files
echo "This is line 1" > output.txt
echo "This is line 2" >> output.txt

# Reading files
content=$(cat output.txt)
echo "File content: $content"

# Processing CSV files
process_csv() {
    local csv_file="$1"
    local line_number=0
    
    while IFS=',' read -r col1 col2 col3; do
        ((line_number++))
        if [ $line_number -eq 1 ]; then
            echo "Headers: $col1 | $col2 | $col3"
        else
            echo "Row $((line_number-1)): $col1 | $col2 | $col3"
        fi
    done < "$csv_file"
}

File Manipulation

#!/bin/bash

# File information
get_file_info() {
    local file="$1"
    
    if [ -f "$file" ]; then
        echo "File: $file"
        echo "Size: $(stat -f%z "$file" 2>/dev/null || stat -c%s "$file" 2>/dev/null) bytes"
        echo "Modified: $(stat -f%Sm "$file" 2>/dev/null || stat -c%y "$file" 2>/dev/null)"
        echo "Permissions: $(stat -f%A "$file" 2>/dev/null || stat -c%a "$file" 2>/dev/null)"
    else
        echo "File '$file' not found"
    fi
}

# Directory operations
organize_files() {
    local source_dir="$1"
    
    for file in "$source_dir"/*; do
        if [ -f "$file" ]; then
            extension="${file##*.}"
            extension_dir="$source_dir/$extension"
            
            mkdir -p "$extension_dir"
            mv "$file" "$extension_dir/"
            echo "Moved $(basename "$file") to $extension_dir/"
        fi
    done
}

Error Handling

Basic Error Handling

#!/bin/bash

# Exit on any error
set -e

# Exit on undefined variable
set -u

# Show commands being executed (for debugging)
# set -x

# Error handling function
handle_error() {
    local exit_code=$?
    local line_number=$1
    echo "Error occurred in script at line $line_number: exit code $exit_code"
    exit $exit_code
}

# Trap errors
trap 'handle_error $LINENO' ERR

Advanced Error Handling

#!/bin/bash

# Logging function
log() {
    local level="$1"
    shift
    local message="$*"
    local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
    echo "[$timestamp] [$level] $message" | tee -a script.log
}

# Safe command execution
safe_execute() {
    local command="$*"
    log "INFO" "Executing: $command"
    
    if eval "$command"; then
        log "SUCCESS" "Command completed successfully"
        return 0
    else
        local exit_code=$?
        log "ERROR" "Command failed with exit code: $exit_code"
        return $exit_code
    fi
}

# Usage example
safe_execute "ls -la /nonexistent" || {
    log "WARNING" "Directory listing failed, continuing..."
}

Best Practices

Script Template

#!/usr/bin/env bash

# Script: template.sh
# Description: Template for shell scripts
# Author: Your Name
# Date: $(date)
# Version: 1.0

set -euo pipefail  # Exit on error, undefined vars, pipe failures

# Global variables
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_NAME="$(basename "$0")"
readonly LOG_FILE="${SCRIPT_DIR}/${SCRIPT_NAME%.sh}.log"

# Colors for output
readonly RED='\033[0;31m'
readonly GREEN='\033[0;32m'
readonly YELLOW='\033[1;33m'
readonly NC='\033[0m' # No Color

# Logging functions
log_info() {
    echo -e "${GREEN}[INFO]${NC} $*" | tee -a "$LOG_FILE"
}

log_warn() {
    echo -e "${YELLOW}[WARN]${NC} $*" | tee -a "$LOG_FILE"
}

log_error() {
    echo -e "${RED}[ERROR]${NC} $*" | tee -a "$LOG_FILE"
}

# Cleanup function
cleanup() {
    log_info "Cleaning up..."
    # Add cleanup code here
}

# Trap cleanup function
trap cleanup EXIT

# Usage function
usage() {
    cat << EOF
Usage: $SCRIPT_NAME [OPTIONS]

OPTIONS:
    -h, --help      Show this help message
    -v, --verbose   Enable verbose output
    -d, --debug     Enable debug mode

EXAMPLES:
    $SCRIPT_NAME --verbose
    $SCRIPT_NAME --debug

EOF
}

# Main function
main() {
    local verbose=false
    local debug=false
    
    # Parse command line arguments
    while [[ $# -gt 0 ]]; do
        case $1 in
            -h|--help)
                usage
                exit 0
                ;;
            -v|--verbose)
                verbose=true
                shift
                ;;
            -d|--debug)
                debug=true
                set -x
                shift
                ;;
            *)
                log_error "Unknown option: $1"
                usage
                exit 1
                ;;
        esac
    done
    
    log_info "Script started"
    
    # Your main logic here
    
    log_info "Script completed successfully"
}

# Run main function with all arguments
main "$@"

Real-World Examples

System Health Check Script

#!/bin/bash

# System health monitoring script
system_health_check() {
    echo "=== System Health Check Report ==="
    echo "Date: $(date)"
    echo "Hostname: $(hostname)"
    echo
    
    # CPU Usage
    echo "=== CPU Usage ==="
    top -bn1 | grep "Cpu(s)" | sed "s/.*, *\([0-9.]*\)%* id.*/\1/" | awk '{print "CPU Usage: " 100-$1 "%"}'
    echo
    
    # Memory Usage
    echo "=== Memory Usage ==="
    free -h | awk '/^Mem:/ {printf "Memory Usage: %s/%s (%.2f%%)\n", $3,$2,$3/$2*100}'
    echo
    
    # Disk Usage
    echo "=== Disk Usage ==="
    df -h | awk '$NF=="/"{printf "Root Disk Usage: %s/%s (%s)\n", $3,$2,$5}'
    echo
    
    # Load Average
    echo "=== Load Average ==="
    uptime | awk -F'load average:' '{print "Load Average:" $2}'
    echo
    
    # Network Connectivity
    echo "=== Network Check ==="
    if ping -c 1 google.com &> /dev/null; then
        echo "Internet: Connected"
    else
        echo "Internet: Disconnected"
    fi
    echo
    
    # Service Status
    echo "=== Critical Services ==="
    for service in ssh nginx mysql; do
        if systemctl is-active --quiet $service 2>/dev/null; then
            echo "$service: Running"
        else
            echo "$service: Not running"
        fi
    done
}

system_health_check

Automated Backup Script

#!/bin/bash

# Automated backup script with rotation
BACKUP_SOURCE="/home/user/important_data"
BACKUP_DEST="/backup"
RETENTION_DAYS=7
LOG_FILE="/var/log/backup.log"

log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

create_backup() {
    local timestamp=$(date +"%Y%m%d_%H%M%S")
    local backup_name="backup_${timestamp}.tar.gz"
    local backup_path="${BACKUP_DEST}/${backup_name}"
    
    log_message "Starting backup of $BACKUP_SOURCE"
    
    # Create backup directory if it doesn't exist
    mkdir -p "$BACKUP_DEST"
    
    # Create compressed backup
    if tar -czf "$backup_path" -C "$(dirname "$BACKUP_SOURCE")" "$(basename "$BACKUP_SOURCE")"; then
        log_message "Backup created successfully: $backup_path"
        
        # Calculate backup size
        local size=$(du -h "$backup_path" | cut -f1)
        log_message "Backup size: $size"
        
        return 0
    else
        log_message "ERROR: Backup creation failed"
        return 1
    fi
}

cleanup_old_backups() {
    log_message "Cleaning up backups older than $RETENTION_DAYS days"
    
    find "$BACKUP_DEST" -name "backup_*.tar.gz" -type f -mtime +$RETENTION_DAYS -delete
    
    local remaining=$(find "$BACKUP_DEST" -name "backup_*.tar.gz" -type f | wc -l)
    log_message "Remaining backups: $remaining"
}

# Main execution
if create_backup; then
    cleanup_old_backups
    log_message "Backup process completed successfully"
else
    log_message "ERROR: Backup process failed"
    exit 1
fi

Additional Resources

Whether you’re just starting with shell scripting or looking to advance your skills, these resources will help you on your journey:

🎮 Online Playgrounds & Sandboxes

Practice shell scripting directly in your browser without installing anything:

Browser-Based Terminals

  • JSLinux - Full Linux system running in browser with multiple distros available
  • Copy.sh - v86 - x86 virtualization in JavaScript with various OS options including Linux
  • WebMinal - Free online Linux terminal for learning and practicing
  • Tutorialspoint Online Terminal - Simple browser-based Linux terminal

Cloud IDEs with Terminal Access

  • Repl.it (Replit) - Full-featured online IDE with Bash support and file system
  • CodePen - While primarily for web development, supports basic shell commands
  • GitPod - Cloud development environment with full Linux terminal (free tier available)
  • GitHub Codespaces - VS Code in the browser with terminal access (free tier)
  • StackBlitz - Instant dev environments with terminal support

Interactive Learning Platforms

  • Katacoda - Interactive Linux and shell scripting scenarios
  • Play with Docker - Free Docker playground that includes Linux terminals
  • Killercoda - Interactive learning platform with hands-on Linux labs
  • Instruqt - Interactive learning tracks with virtual environments

Shell-Specific Playgrounds

  • Bash Online - Simple online Bash script executor
  • OneCompiler - Bash - Online Bash compiler and runner with syntax highlighting
  • JDoodle - Bash - Execute Bash scripts online with input/output support
  • Paiza.io - Online coding environment supporting multiple languages including Bash

Virtual Lab Environments

Mobile-Friendly Options

Quick Script Testing

  • ShellCheck Online - Not a playground but essential for validating shell scripts
  • Explain Shell - Understand what shell commands do before running them
  • Regex101 - Test regular expressions used in shell scripts

Getting Started Tips for Playgrounds:

  1. Start Simple: Begin with basic commands like ls, pwd, echo
  2. Create Test Files: Use touch and echo to create files for practice
  3. Practice File Operations: Try moving, copying, and editing files
  4. Test Your Scripts: Copy-paste the examples from this blog and run them
  5. Experiment Safely: These environments are isolated, so feel free to experiment
  1. Read the tutorial section in this blog
  2. Try examples in an online playground like Replit or JSLinux
  3. Modify and experiment with the code
  4. Create your own scripts based on what you learned
  5. Test edge cases and error handling
  6. Share your creations with the community

Remember: Online playgrounds are perfect for learning and quick testing, but for serious development work, consider setting up a local Linux environment or using a more robust cloud IDE.

📚 Beginner Tutorials

🎓 Interactive Learning Platforms

📖 Advanced Resources

🔧 Reference & Documentation

🛠️ Tools & Utilities

  • Shellharden - Tool to help make your shell scripts more robust
  • Bats - Bash Automated Testing System for testing shell scripts
  • Shfmt - Shell script formatter and parser
  • Fish Shell - User-friendly command line shell with great scripting capabilities

📺 Video Tutorials

🏆 Practice Platforms

📱 Mobile Apps

🌐 Communities & Forums

📚 Books (Free Online)

🎯 Specialized Topics

Remember: The best way to learn shell scripting is by practicing! Start with simple scripts and gradually work your way up to more complex automation tasks.

Conclusion

Shell scripting is a powerful tool for automation and system administration. Start with simple scripts and gradually incorporate more advanced features as you become comfortable with the basics.

Key Takeaways:

  1. Always use proper error handling - Don’t let your scripts fail silently
  2. Make your scripts readable - Use meaningful variable names and comments
  3. Test thoroughly - Test your scripts in different environments
  4. Follow conventions - Use consistent coding style and structure
  5. Log everything - Keep track of what your scripts are doing

Next Steps:

  • Practice with the examples provided
  • Create your own automation scripts for daily tasks
  • Learn about advanced topics like signal handling and process management
  • Explore tools like shellcheck for script validation

Happy scripting! 🚀

Found this helpful? Share your own shell scripting tips and tricks in the comments below!