...

Linux Shell Commands


System Info:

ps:
$ ps aux 
server processes, with user
$ ps -AlFH
long form, with arguments to commands, and threads
$ ps -ejH $ ps axjf $ pstree $ ps -ef
show processes
$ ps aux | grep -i tomcat
check if tomcat is running
top:
interactive commands:
  h 
show help commands
z or b
highlights running processes
c
display full command line
n 5
show only top 5 tasks
l t m
toggle headers (load avg, cpu tasks, memory))
d
change refresh rate from default 3 secs
1
toggle individual cpu / cpu total mode
O
choose column to sort by
  • see http://superuser.com/questions/575202/understanding-top-command-in-unix
  • see https://www.linux.com/learn/tutorials/42048-uncover-the-meaning-of-tops-statistics
command execution by the shell:
shell searches for command to execute in the following order:
  • see if builtin
  • then check in hashtable
  • then check in PATH variable
$ type -a ls 
tells you if command is builtin, hashed or where it is located
$ which pwd
tells you where the binary is located
$ echo $PATH
shows path(s) where commands are searched for
$ hash
displays the hashtable cache used for quick command lookups
$ hash -r
clears the hashtable cache
$ export PATH=$PATH:~/bin
to temporarily add ~/bin to the path variable
$ alias ll='ls -l'
sets ll to be an alias for ls -l
$ unalias ll
unalias (use \ll to bypass alias)
$ history
then can do !123, to run the command listed by history
proc:
$ cat /proc/cpuinfo
$ cat /proc/meminfo | grep MemTotal 
should show total memory in the system
$ cat /proc/mounts
misc:
$ lscpu 
shows info about CPU(s)
$ free -tg
memory, including totals, in GB
$ mpstat -P ALL
cpu utilization on multi cpu machine
$ ls -l /proc/1138/exe
gives the process associated with the pid 1138
$ pwdx 23340
gives the current working directory of the process of pid 1138
$ pkill -9 -u jayz
to kill all sessions of the user jayz

File Management and Navigation:

cd:
$ cd ~ 
go to your home directory
$ cd -
go to prev direcotory that we cd’ed from
list:
$ ls -l 
can use ll as alias for this
  • 1st Character – File Type: First character specifies the type of the file.
    • -
      normal file
    • d
      directory
    • s
      socket file
    • l
      link file
  • Field 1 – File Permissions: 9 character specifies the files permission, rwx for user, rwx for group and rwx for others.
  • Field 2 – Number of links: Second field specifies the number of links for that file.
  • Field 3 – Owner: Third field specifies owner of the file.
  • Field 4 – Group: Fourth field specifies the group of the file.
  • Field 5 – Size: Fifth field specifies the size of file.
  • Field 6 – Last modified date & time: Sixth field specifies the date and time of the last modification of the file.
  • Field 7 – File name: The last field is the name of the file.
$ ls -1 
list with one line per file name
$ ls -lh
long listing of files with human readable sizes
$ ls -t
display files by last modified first
$ ls -1lr
reverse
$ ls -a
show all ifles, including hidden files
$ ll
alias for ls -a
$ ls -R
show files recursively
$ ls -F * /
directory.
* nothing
normal file.
* @
link file.
* *
Executable file
$ ls -i
show files with inode number
$ ln targetFile linkToBeCreatedToTargetFile
creates hard links (so they'll have the same inode)
$ ls -l
should show the number of hard links the files have
(rm only removes one link, the file is not deleted until all the links are deleted)
$ ln -s targetFile linkToBeCreatedToTargetFile
creates a soft link (symlink) the 2 files have different inodes
$ ls -l
will also show the linking between the files
misc:
$ ant start-icnow | tee op.txt 
sends output to both stdout and a file
$ watch command
you can see the output of the command updated every two seconds
tail:
$ tail -f /var/log/syslog -f /var/log/auth.log 
multi tail
$ tail -f filename1 filename2
try this one too
$ tail -c30 /var/log/syslog
display last 30 bytes from syslog
$ head -c40 /var/log/syslog
display first 40 bytes from syslog
find:
also see grep

$ find .  
list all files in current and subdirectories
$ find . *.txt -type f | xargs grep -i pattern
feed all files with ext .txt from the current and sub dirs to grep, which does a case-insen search for files containing pattern.
$ find / -name filename
search starting from /
$ find . -name *handle*
case sensitive names containing handle
$ find . -iname *handle*
case insensitive
$ find . -inum 433580
find all the hard links with given inode
$ find . -type d -name AMA
find all directories named AMA
$ find . -maxdepth 1 -not -iname "MyCProgram.c" $ find ~ -empty $ find . -type d
find all directories
$ find . -type f
find only the normal files
$ find . -type f -name ".*"
find all the hidden files
$ find -type d -name ".*"
find all the hidden directories
$ find -newer FILE
find files which are modified after modification of a particular FILE
$ find -anewer FILE
find files which are accessed after modification of a specific FILE
$ find -cnewer FILE
find files whose status got changed after the modification of a specific FILE
$ find ~ -size +100M
find files bigger than the given size
$ find ~ -size -100M
find files smaller than the given size
$ find ~ -size 100M
find files that matches the exact given size
$ find . -mmin -60
find files in current and sub dirs, updated in the last 60 minutes
$ find / -mtime -1
find all the files (under /) that got updated within the last 1 day
$ find -amin -60
find files in current and sub dirs, accessed within last 60 minutes
$ find / -atime -1
finds all the files (under /) taccessed within the last 1 day
$ find . -cmin -60
find files in current and sub dirs, changed within last 60 minutes
$ find / -ctime -1
finds all the files (under /) changed within the last 1 day
less:
$ less file1 
starts off by displaying the head of file1
ESC should take you to command mode, indicated by a :
h inline help
= status, position and statistics
Navigation[Arrows]/[Page Up]/[Page Down]/[Home]/[End][Space bar]
ESC < Go to first line in file (or line N).
ESC > Go to last line in file (or line N).
ngJump to line number n. Default is the start of the file.
nGJump to line number n. Default is the end of the file.
j scroll forward 1 line (10j scroll forward 10 lines)
k scroll backward 1 line (5 scroll backward 5 lines)
d Forward one half-window (d5 sets half-window to 5 lines).
u Backward one half-window (u2 set half-window to 2 lines).
mletter Mark the current position with letter.
'letter Return to position letter. [' = single quote]
/patternSearch forward for (N-th) matching line. can use regular expressions.
?patternSearch backward for (N-th) matching line.
nGo to next match (after a successful search).
NGo to previous match.
ESC-u Undo (toggle) search highlighting.
&patternDisplay only matching lines (& again resets)
^N or ! Search for NON-matching lines.
!command Execute a shell command
Ftail current file. Ctrl+c to exit this mode.
-i toggle Case-insensitive searches.
-M Shows more detailed prompt, including file position.
-N toggle show line numbers
-S toggle line wrap ("chop long lines")
-s toggle squeeze blank lines
qQuit.
grep:
$ grep -r -i -n 'hello world' . 
case insensitive recursive search for 'hello world' starting from current dir, also print line numbers
$ grep -r -i 'pattern' .
recursive case-insensitive search for 'pattern', starting from current dir
$ grep -rin 'pattern' .
recursive case-insensitive search for 'pattern', starting from current dir, show line number
$ grep -ril 'pattern' .
recursive case-insensitive search for 'pattern', starting from current dir show, only occuring file name
$ grep -i 'hello world' menu.h main.c
case insentive search for 'hello world' in files menu.h and main.c
-i ignore case distinctions
-v select non-matching lines
-n print line number with output lines
-H print the file name for each match
-o show only the part of a line matching PATTERN
-r recursive
--include=FILE_PATTERN search only files that match FILE_PATTERN
--exclude=FILE_PATTERN skip files and directories matching FILE_PATTERN
--exclude-dir=PATTERN directories that match PATTERN will be skipped.
-L print only names of FILEs containing no match
-l print only names of FILEs containing matches
-c print only a count of matching lines per FILE
-B print NUM lines of leading context
-A print NUM lines of trailing context
-C print NUM lines of output context
vi:
ESC switches from edit mode to command mode
edit mode commands
i Insert before current cursor position
I Insert at beginning of current line
a Insert (append) after current cursor position
A Append to end of line
r Replace 1 character
R Replace mode
x Delete single character
dd Delete current line and put in buffer
5dd Delete 5 lines and put them in buffer
J Attaches the next line to the end of the current line (deletes carriage return).
u Undo last command
yy (:y) Yank current line into buffer
7yy Yank 7 lines into buffer
p Put the contents of the buffer after the current line
P Put the contents of the buffer before the current line
^d Page down
^u Page up
^g Display current line number
command mode commands
:13 Position cursor at line 13
:$ Position cursor at end of file
:w Write the current file.
:w new.file Write the file to the name 'new.file'.
:w! existing.file Overwrite an existing file with the file currently being edited.
:wq Write the file and quit.
:q Quit.
:q! Quit with no changes.
:set number Turns on line numbering
:set nonumber Turns off line numbering
h,j,k,l or arrow keysNavigation - Left,Down,Up,Right
/string Search forward for string
?string Search back for string
n Search for next instance of string
N Search for previous instance of string
:5,13:s/string1/string2/[g] Substitute string1 with string2 on lines5 to 13. [g] for global replace, else only 1st occurance is replaced per line.
  • ^ matches start of line
  • . matches any single character
  • $ matches end of line
  • (escape special characters with \)
Examples:
:1,$:s/dog/cat/g Substitute 'cat' for 'dog', every instance for the entire file - lines 1 to $ (end of file)/td>
:23,25:/frog/bird/Substitute 'bird' for 'frog' on lines 23 through 25. Only the first instance on each line is substituted.

Also see http://www.lagmonster.org/docs/vi.html


Disks related:

Applications - > Accessories - Disk Usage Analyzer (baobab) graphical

$ df -h 
disk usage, in GB
$ du -shx */
disk usage, for all directories from current directory
$ du -shx *
disk usage, for all directories and files from current directory
$ sudo fdisk -l
will list all (mounted) filesystems and partitions on the hard disks
$ sudo hdparm -I /dev/sda
details on disk
$ sudo mount
shows all currently mounted partitions
$ mkdir /mnt/backup
create mount point
$ mount /mnt/backup
mount the drive (add entry to /etc/fstab to mount automatically everytime on boot)
$ sudo rm -r /home/JLseagull/.local/share/Trash/files/
empty trash

Networking related:

wget, curl:
$ wget -q -O - checkip.dyndns.org|sed -e 's/.*Current IP Address: //' -e 's/<.*$//' 
command line whatismyip
$ wget
download entire website
--recursive
download the entire site
--no-clobber
don't overwrite any existing files (if the download is interrupted and resumed)
--page-requisites
get all the page resources (images, css, etc.)
--html-extension
save files with the .html extension
--convert-links
convert links so that they work locally, off-line
--restrict-file-names=windows
modify filenames so that they will work in Windows as well
--domains web.site
don't follow links outside web.site
--no-parent
don't follow links outside the directory root/
www.web.site/root/
start downloading site from this context
netstat:
$ netstat -ie 
shows network interface information
$ netstat -ap
shows all the ports the processes are connected to
$ netstat -ant | grep LISTEN
show all listening tcp ports
$ netstat -nlp
lists ports in use
$ netstat -ct
continuously shows all tcp connections
$ netstat --listen
show open ports
nmap:

To scan a network, for eg. to find the IP of a machine running ssh (port 22) on the 192.168.1.* network, listing only open ports (--open) and listing service/version (-sV)

$ nmap -p 22 --open -sV 192.168.1.*
to be able to login from local to remote using passwordless ssh
local$ ssh-keygen -t dsa 
enter passphrase, or leave blank
local$ ssh-copy-id -i ~/.ssh/id_dsa.pub username@remote
automatically create the necessary folders and permissions correctly on remote
(or, do this) ssh-copy-id -i ~/.ssh/id_dsa.pub srvc01.chi # OR, if you want to do it manually local$ scp ~/.ssh/id_dsa.pub username@remote local$ ssh username@remote remote$ cat ~/id_dsa.pub >> ~/.ssh/authorized_keys remote$ chmod 644 ~/.ssh/authorized_keys
ssh needs this file to be NOT world writable
remote# chmod 700 ~/.ssh # ssh needs .ssh to be NOT world writable remote$ exit local$ ssh username@remote
will ask for ssh passphrase (or not, if passhphrase was blank, useful for automating
logins, etc)
Note: if passphrase was entered, can add to local ssh keyring? or ssh-agent ? for extra security? if using ssh2, authorized_keys file is actually authorized_keys2 ? to restart sshd on the remote machine
/etc/init.d/sshd reload 
do not do /etc/init.d/sshd restart, that will kick you out of the session ?
certs related:
$ keytool -printcert -v -file cert1.crt
$ keytool -list -v -storetype JCEKS -keystore keystore1.jceks 
$ keytool -list -v -keystore keystore1.jks
$ keytool -list -v -keystore keystore1.jks -alias alias1
misc:
$ lsof -i :8080 
shows all machines using port 8080
$ fuser 7000/tcp
gives the pid of the process using the tcp port 7000
clean up the foll:
/etc/sshd/sshd_config on remote should have
RSAAuthentication yes
PubkeyAuthentication yes
AllowUsers username 
to restrict which users can ssh
AuthorizedKeysFile %h/.ssh/SOME_ORIGINAL_NAME
to use non-default authorised keys file
check /var/log/auth for error messages http://www.debian-administration.org/articles/87 good site for "Keeping SSH access secure"

64 bit support:

Check if a CPU supports 64-bit by entering the command below. In the output of the flags section, you will see many entries. Look for a 2 character entry (surrounded by spaces - not part of another word) called "lm".
$ cat /proc/cpuinfo | grep lm
If you're unsure whether the Linux OS that you are currently running is 64-bit, you can enter the following command and look for the value x86_64 in the output as opposed to something like i686, i586, and so on:
$ uname -m
x86_64
Similarly to above, you can also check if a given Linux executable is 64-bit. In the case below we examine the /bin/ls executable and look for the string "ELF 64-bit LSB executable" as opposed to "ELF 32-bit LSB executable":
$ file /bin/ls
/bin/ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses
shared libs), for GNU/Linux 2.6.24

sudo and account related:

$ su -l # get a root shell
$ sudo !! # repeat the previous command which you were not able to run because it required sudo
$ last # shows last few logins
$ lastb # shows last few failed logins
$ cat /etc/password # this file has list of all users
$ id username # shows userid, and groups that user belongs to
$ sudo users-admin # fires off users admin gui
$ sudo usermod -a -G dba JLseagull # add user JLseagull to group dba
$ chown greatGull file1 # change owner of file1 to greatGull
$ chgrp gullFlock file1 # change group of file1 to gullFlock
To enable the root account in Ubuntu, enter the command sudo passwd root. When you see the phrase "Enter new UNIX password" this is to define the password for the root account: To enable the root account
user@ubuntu:~$ sudo passwd root
[sudo] password for user:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Encryption related:

Use "Disk Utility" to graphically show volume information, including encryption status
To check from the command line
root$ cat /etc/crypttab
sda5_crypt UUID=fa5a6f0a-f8ce-1ea9-c1a1-23231a98b65e none luks

root$ cryptsetup status sda5_crypt

root$ pvdisplay -m

root$ lvdisplay -m

gpg related:

$ gpg --import key.asc # to import the private keys into gpg
$ gpg FILENAME.pgp # and provide passphrase above to decrypt

# script to process all the files in a folder
#!/bin/sh
for currentfile in *.pgp
do
 echo .................................... processing ${currentfile}
 gpg --passphrase substPassphraseHere --quiet --no-verbose --no-secmem-warning --no-greeting $currentfile
done
echo finished

Software management:

Can use "Synaptic" for apt management, or can do command line apt related commands.

To remove
$ sudo apt-get remove avahi-daemon

Very good resources: