Unix Bash for High Performance Computing (HPC)

🖥️ What is Unix/Linux?

Unix is a powerful multiuser operating system. Linux is a Unix-like system commonly used in HPC. You interact with it using a command-line interface (CLI) or shell (bash). Welcome to the beginner’s guide to Unix and the command line interface for HPC users.

🧭 Navigating the Unix Filesystem

Show your current directory

pwd

List files in the current directory

ls -lh

Change directory

cd /path/to/folder

Go up one directory

cd ..

📁 Working with Files and Directories

Create a directory

mkdir myfolder

Create a file

touch file.txt

Copy a file

cp file1.txt file2.txt

Move (or rename) a file

mv file1.txt folder/

Delete a file or folder

rm file.txt
rm -r folder/

📄 Viewing and Editing Files

Show file contents

cat file.txt

Scroll through a file

less file.txt

Show first/last lines

head file.txt
tail file.txt

🔍 Searching and Finding Files

Search inside files

grep "search_term" file.txt

Find files by name

find . -name "*.txt"

🔄 Permissions and Ownership

Check permissions

ls -l

Change permissions

chmod 755 script.sh

Change ownership (admin only)

sudo chown user:group file.txt

🛠️ Useful Commands

🧪 Practice Exercises

1. Create a folder named unix_practice
2. Inside it, create three .txt files
3. Copy one file to a subfolder
4. Use grep to search a keyword in a file
5. Change file permissions using chmod
6. Use find to locate all .txt files in your home directory

🛠️ Bash Scripting Basics

A script is a file of commands executed line by line. Example: my_script.sh

#!/bin/bash
echo "Starting job..."
cd /home/user/myproject
python analysis.py

Run it with:

bash my_script.sh

📝 Writing & Submitting Jobs to a Scheduler (SLURM)

Basic SLURM Script: job.slurm

#!/bin/bash
#SBATCH --job-name=test_job
#SBATCH --output=out.txt
#SBATCH --time=00:10:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=2

module load python
python3 analysis.py

Submit and manage jobs

sbatch job.slurm      # Submit job
squeue -u $USER       # Check job status
scancel <jobid>       # Cancel job

🔍 Monitoring Jobs

top                  # Real-time CPU usage
htop                 # Enhanced top (if available)
squeue -u $USER      # Job queue
sacct -j <jobid>     # Job accounting info

✅ Best Practices for Bash in HPC

🚀 Quick Reference: Bash Cheat Sheet