# Linux Intermediate Part2 **which:** This command will **locate the executable files** associated with a given command by searching through the directories listed in the **PATH** environment variable. Below example is to locate the **gcc compiler.** The output of that show that the gcc compiler is located in **usr/bin/gcc** location. ```bash labex:project/ $ ls labex:project/ $ which gcc /usr/bin/gcc labex:project/ $ which python /usr/local/bin/python ``` **Note:** The `which` command searches through the directories listed in your **PATH environment variable** and returns the **first match** it finds. **which -a:** This is to find all location for python. The below example shows there are multiple python installations in the system. ```bash labex:project/ $ which -a python /usr/local/bin/python /usr/bin/python /bin/python /usr/local/bin/python /usr/local/bin/python ## The PATH variable ## The order matters: directories listed earlier have higher priority. ## This means if there are two executables with the same name in different directories, the one in the directory that appears first in PATH will be used. labex:project/ $ echo $PATH /usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/labex/.local/bin:/usr/local/go/bin:/home/labex/golang/bin:/usr/share/maven/bin:/usr/local/bin:/usr/sbin/nodejs/bin:/usr/sbin/yarn/bin:/home/labex/.local/bin:/usr/local/go/bin:/home/labex/golang/bin:/usr/share/maven/bin:/usr/local/bin:/usr/sbin/nodejs/bin:/usr/sbin/yarn/bin ##Example labex:project/ $ echo '#!/bin/bash echo "Hello from TechCorp!"' > ~/hello chmod +x ~/hello labex:project/ $ hello zsh: command not found: hello labex:project/ $ labex:project/ $ export PATH=$PATH:$HOME labex:project/ $ hello Hello from TechCorp! labex:project/ $ which hello /home/labex/hello labex:project/ $ ``` **whereis:** This command will help us to **locate binary, source and manual page files** for various commands in Linux system. See the below example and the output. 1. `/usr/bin/ls`: This is the location of the `ls` binary executable. When you type `ls` in the terminal, this is the actual program that runs. 2. `/usr/share/man/man1/ls.1.gz`: This is the location of the manual page for `ls`. Manual pages contain detailed information about how to use commands. ```bash labex:project/ $ whereis ls ls: /usr/bin/ls /usr/share/man/man1/ls.1.gz ``` * `/usr/bin` is a **common location for user commands.** * `/usr/share/man` is where manual pages are **typically stored.** * `man1` indicates this is a **user command manual** (as opposed to system calls or library functions). * The `.gz` extension means the file is **compressed** to save space. ```bash ## whereis -b is to check if the executable file exists. labex:project/ $ whereis -b grep grep: /usr/bin/grep labex:project/ $ whereis -b nonexistent nonexistent: ## whereis -m is to locate the manual page labex:project/ $ whereis -m ssh ssh: /usr/share/man/man1/ssh.1.gz ## To comine both binary and manual pages to show the output labex:project/ $ whereis -bm python3 python3: /usr/bin/python3 /usr/lib/python3 /etc/python3 /usr/share/python3 /usr/share/man/man1/python3.1.gz ``` **whereis -s:** This is to seach the **source file.** ```bash labex:project/ $ whereis -s bash bash: labex:project/ $ whereis bash bash: /usr/bin/bash /usr/share/man/man1/bash.1.gz ``` **find:** As the name suggest, it’s used for finding a file. See the below example. ```bash labex:project/ $ find . -name "clue.txt" ./evidence/clue.txt ##Example of finding all .txt and all .log files labex:project/ $ find . -name "*.txt" -o -name "*.log" ./evidence/clue.txt ./evidence/new_lead.txt ./logs/system.log ./logs/recent_activity.log ./notes/interview.txt ``` * `find` is the command we're using * `.` tells `find` to start searching from the **current directory** * `-name "clue.txt"` tells `find` to look for a file **exactly named** "clue.txt" ```bash ## finding all files greater than 1 MB in size. labex:project/ $ find . -type f -size +1M ./evidence/large_file.dat ``` * `find .` is our familiar starting point * `-type f` tells `find` to look only for regular files (not directories or other special types) * `-size +1M` specifies that we want files larger than 1 megabyte ```bash ## -mtime -1 is new. It tells find to look for files modified less than 1 day ago. labex:project/ $ find . -type f -mtime -1 ./evidence/clue.txt ./evidence/large_file.dat ./evidence/new_lead.txt ./logs/system.log ./logs/recent_activity.log ./notes/interview.txt ``` ```bash labex:project/ $ find . -name "*.txt" -exec cat {} \; The suspect was last seen wearing a red hat. New lead: Check the security camera footage from the back alley. Witness reported hearing a loud noise at approximately 10 PM. ``` * `find . -name "*.txt"` we've seen before - it finds all .txt files * `-exec cat {} \;` is new. It tells `find` to execute the `cat` command on each file it finds * `cat` is a command that displays the **contents** of a file * `{}` is a **placeholder** that `find` replaces with each filename it finds * `\;` marks the **end** of the `-exec` command * Additional `find` command options not covered in this lab include: * `-user`: Find files owned by a specific user * `-group`: Find files belonging to a specific group * `-perm`: Find files with specific permissions * `-maxdepth`: Limit the depth of directory traversal * `-mindepth`: Start searching from a minimum depth * `-empty`: Find empty files or directories * `-newer`: Find files newer than a specified file **grep:** This command is use to **search** for **specific pattern** in the file. See the below example to search for **‘ERROR’** in the server.log file. **grep is case sensitive.** Note: ***grep stands for ‘Global Regular Expression Print’.*** ```bash labex:project/ $ grep "ERROR" logs/server.log [2023-11-23 00:00:00] ERROR: Database connection failed [2023-10-23 00:00:00] ERROR: Database connection failed [2023-03-04 00:00:00] ERROR: Database connection failed [2023-08-19 00:00:00] ERROR: Database connection failed ## Get the count of the pattern labex:project/ $ grep -c "ERROR" logs/server.log 146 ## to make the search pattern case insensitive, use -i labex:project/ $ grep -i "error" logs/server.log [2023-11-23 00:00:00] ERROR: Database connection failed [2023-01-18 00:00:00] error: File not found [2023-12-09 00:00:00] error: File not found [2023-01-20 00:00:00] error: File not found ## Searching multiple files labex:project/ $ grep "database connection failed" logs/* logs/error.log:[2023-07-02 11:25:40] WARNING: database connection failed, retrying... logs/server.log:[2023-07-01 10:20:30] ERROR: database connection failed ## Use regular expression in grep search grep "2023-[0-9][0-9]-[0-9][0-9]" logs/server.log ``` **grep -v**: This is used for **not showing** result with that pattern. **\-v mean ‘Not this‘** ```bash labex:project/ $ grep -v "ERROR" logs/server.log [2023-01-18 00:00:00] error: File not found [2023-12-09 00:00:00] error: File not found [2023-01-20 00:00:00] error: File not found [2023-02-22 00:00:00] WARNING: High CPU usage detected ``` **wc:** This is used for counting the words. **wc** stands for **word count.** ```bash labex:project/ $ ls project_overview.md README.md requirements.txt src technical_specs.txt ## -l is for line. The below command will show number of lines. labex:project/ $ wc -l requirements.txt 51 requirements.txt labex:project/ $ wc -l src/main.py 801 src/main.py ## Let's count the number of words using -w labex:project/ $ wc -w project_overview.md 2305 project_overview.md labex:project/ $ wc -w technical_specs.txt 461 technical_specs.txt ##count the character using -m labex:project/ $ wc -m src/utils.py 10103 src/utils.py labex:project/ $ wc -m src/database.py 10106 src/database.py ##combining multiple commands labex:project/ $ wc -l -w -m README.md 121 278 8388 README.md ``` **cut:** This is to **extract and analyse** the data from a text file. See the below example for customer.txt file where the data is is comma separated format. ```bash labex:project/ $ cat customers.txt ID,Name,Age,Email 1,John Doe,25,john.doe@email.com 2,Jane Smith,35,jane.smith@email.com 3,Lily Chen,30,lily.chen@email.com 4,Andy Brown,22,andy.brown@email.com labex:project/ $ cut -d ',' -f 2 customers.txt Name John Doe Jane Smith Lily Chen Andy Brown ## This to starting from the 2nd line, basically to skip header. labex:project/ $ cut -d ',' -f 2 /home/labex/project/customers.txt | tail -n +2 John Doe Jane Smith Lily Chen Andy Brown ``` * `cut`: The name of the command we're using * `-d ','`: Specifies that we're using a comma as the delimiter between fields * `-f 2`: Tells `cut` to extract the second field * `/home/labex/project/customers.txt`: The path to our input file ```bash ## Examples ## The below example is to show the 2nd and 3rd column labex:project/ $ cut -d ',' -f 2,3 /home/labex/project/customers.txt Name,Age John Doe,25 Jane Smith,35 Lily Chen,30 Andy Brown,22 ## Extracting a range of columns. In the below example it is from 1st to 3rd col. ## We can also combine 1-3,5 to exatract 1st,2nd, 3rd and 5th col. labex:project/ $ cut -d ',' -f 1-3 /home/labex/project/customers.txt ID,Name,Age 1,John Doe,25 2,Jane Smith,35 3,Lily Chen,30 4,Andy Brown,22 ## -c 11-25: This tells cut to extract characters 11 through 25 from each line labex:project/ $ cat inventory.txt ISBN Title Quantity 1234567890The Great Adv 100 2345678901Mystery in th 75 3456789012Cooking Basi 50 4567890123Science Exp 125 labex:project/ $ cut -c 11-25 inventory.txt itle Q The Great Adv Mystery in th Cooking Basi Science Exp ## Mix -grep with cut labex:project/ $ cat books.txt ISBN,Title,Author,Price 978-1234567890,The Great Adventure,Alice Writer,19.99 978-2345678901,Mystery in the Woods,Bob Author,24.99 978-3456789012,Cooking Basics,Carol Chef,15.99 978-4567890123,Science Explained,David Scientist,29.99 labex:project/ $ grep -E ',[2-9][0-9]\.[0-9]{2}$' /home/labex/project/books.txt | cut -d ',' -f 2 Mystery in the Woods Science Explained ``` **sort:** This is for arranging and organizing the data. See the below example ```bash labex:project/ $ cat students.txt David Lee Alice Johnson Charlie Brown Bob Smith Eve Wilson labex:project/ $ sort students.txt Alice Johnson Bob Smith Charlie Brown David Lee Eve Wilson ``` ```bash labex:project/ $ cat student_ages.txt David Lee:21 Alice Johnson:18 Charlie Brown:19 Bob Smith:20 Eve Wilson:18 labex:project/ $ sort -n -t: -k2 student_ages.txt Alice Johnson:18 Eve Wilson:18 Charlie Brown:19 Bob Smith:20 David Lee:21 ## -nr is for reverser sort for number labex:project/ $ sort -nr -t: -k2 student_grades.txt Charlie Brown:95 Alice Johnson:92 Eve Wilson:91 Bob Smith:88 David Lee:87 ## -u is to remove duplicate line in the list. ## See in the below example, It sorted the lines and remove the duplicate one ## which is 'Alice Johnson:Chess Club' labex:project/ $ cat student_clubs.txt Alice Johnson:Chess Club Bob Smith:Debate Team Charlie Brown:Chess Club David Lee:Science Club Eve Wilson:Debate Team Alice Johnson:Chess Club Bob Smith:Science Club labex:project/ $ sort -u student_clubs.txt Alice Johnson:Chess Club Bob Smith:Debate Team Bob Smith:Science Club Charlie Brown:Chess Club David Lee:Science Club Eve Wilson:Debate Team ``` * `-n`: This option tells `sort` to perform a numeric sort. * `-t:`: This specifies that fields are separated by colons. * `-k2`: This tells `sort` to use the second field (the age) as the sorting key. **uniq:** This is to identifying and filtering **duplicate lines**. ```bash labex:project/ $ sort customer_purchases.txt > sorted_purchase.txt labex:project/ $ cat sorted_purchase.txt Alice,Electronics Alice,Electronics Alice,Electronics Bob,Books Bob,Books Charlie,Clothing Charlie,Clothing David,Home Goods Eve,Toys Frank,Sports labex:project/ $ uniq sorted_purchase.txt unique_purchases.txt labex:project/ $ cat unique_purchases.txt Alice,Electronics Bob,Books Charlie,Clothing David,Home Goods Eve,Toys Frank,Sports ## Some more example with -c for count labex:project/ $ uniq -c sorted_purchase.txt purchase_counts.txt labex:project/ $ cat purchase_counts.txt 3 Alice,Electronics 2 Bob,Books 2 Charlie,Clothing 1 David,Home Goods 1 Eve,Toys 1 Frank,Sports labex:project/ $ uniq -d sorted_purchase.txt repeat_customers.txt labex:project/ $ cat repeat_customers.txt Alice,Electronics Bob,Books Charlie,Clothing ```