Linux Essentials

regular expressions
...describe search patterns

similar to wildcards, more sophisticated

denote different variation patterns of strings

basic and extended form

alphanumeric or standard strings

advanced matching rules

special chars
[] bracket expression
match any one character of the given
b[aeiou]g bug bag big beg bog

[-] range expression
a[2-4]z a2z a3z a4z

.
a.z any letter or number can be the dot

^ start of line
$ end of line

. repetition of the matched item

escaping characters in regular expressions
filename\.txt

file archives

collect groups of files to move them around

tar, zip

tar - tape archive

compressed as "tarball"

zip gzip bzip2 xz
zip programs

gzips gunzip
bzip2 bunzip2
xz unxz

.tgz tarball compressed with gzip

tbz tarball compressed with bzip

txz

grep -E '[aeiou]{2,}' fruitstand.txt
find ocurrences of two neighboring vowels

grep -E '2.+' fruitstand.txt
find 2 and something following, but not as last character

grep -E '2$' fruitstand.txt
looking for 2 at the end of the line

grep -E 'is|go|or' fruitstand.txt
search for any of these substrings / letter combinations in the file

slice it and dice it to find things in logfiles

grep -E '^[a-l]' fruitstand.txt
select lines beginning with lowercase a to l

grep -E '^[^flt]' fruitstand.txt
select everything that does NOT begin with f, l or t

kernel = low level computer program, bridge between user and computer resources
memory, input, output

rolling releases ... system is constantly updated

usecase embedded systems for linux
arduino, raspberry pi

small footprint, highly customizable

android is sort of a linux
and it's fucking everywhere

LTS long term support version

distros are installed as .iso files
or write your .iso to a bootable usb drive

free virtualization software virtualbox - use it to play with the install procedure

keep your system up to date to have it secured

type in the virtual system into terminal:
sudo apt install linux-headers-$(uname -r) build-essential dkms
then have guest addons installed via script

linux less than 2% end user market share... why?
available free, it's secure, runs on any hardware ....

mac os runs only on mac hardware.

windows has ms dos terminal

mac has terminal

linux has shell

open source software is a great example of open collaboration

android is an embedded operating system variant of linux that is used in smartphones, tablets and tv's

hcl hardware compatibility list

preemptive multitasking is a task in which a computer operating system uses some criteria to decide how long to allocate to any one task before giving another task a turn to use the operating system. the act of taking control of the operating system from one task and giving it to another task is called preempting.

open source software advantages
scalability, security, affordability, transparency, interoperability

source code - human readable form of machine process

in open source, cost is distributed differently

nevertheless, you can donate or pay for certain specific implementations.

wireshark network analyzer

gparted partition editor

timeshift backup tool

atom text editor with version control system integration

putty ssh gui

javascript files
index.html - > script.js
referenced via
!!!script

python files
interactive environment takes python commands
also executed via shell files

php files
php

java
compiled language
classes are compiled into executable binaries
java "ExecutableClass"
executes the file

stylesheets
css styles html pages
change look and feel of a web site
link rel="stylesheet"

c++
compiled language
lower level programming language
g++ compiler
g++ -o hello helloworld.cpp
./hello

go
newer object oriented language
compiled language

c
low level programming language
predecessor of c++
gcc helloworld.c -o hello
./hello

c#

ruby
hello_world.rb
interpreted or scripted

interactive ruby environment
irb

repositories contains specifically compiled packages for distro

command line can be quicker for some use cases

desktop environments are interchangeable

ubuntu is so slick and eyecandy )))

linux desktops are highly configurable

flexible support for word processor formats, cross compatibility

anything that works on windows or mac, also works on linux

bash acts as a shell program in the command line

terminal emulator / terminal lets user interact

there are quite a bunch of emulators...

and tons of commands they accept

ls list
cp copy
cd change directory
mv move file
rm delete / remove file / dir
man manual
mkdir make directory
rmdir remove directory
touch create a file
locate find a file
clear clear screen buffer

software packages have dependencies, often ship with instructions

.deb .rpm .tgz

debian derived distros

red hat linux system

tarball universal linux format

package managers try to resolve dependencies automatically
and updates

dpkg apt rpm yum dnf

apt yum pacman dnf

each distro has its own package structure

windows and mac os desktop environments are unseparable,
while linux makes it possible to choose one

the kernel directly controls hardware and translates the commands given from a piece of software into something the hardware can understand and act upon

terminal programs are used to access the shell

the application packaging is different for each distro family

telnet or ssh allows remote login to a shell

each user can define default shell
and default terminal gui program

terminal shortcut ctrl alt t

computer waits
user enters command
computer executes

command - char sequence
inspired by english language

syntax is crucial

command parameters switches

options switches start with a dash

-a -c is the same as -ac

long options start with two dashes --help

commands don't have leading dashes

command options arguments

shell automates things with shell scripts

shell script - file containing a series of commands

variables must start with a letter
must not contain whitespaces
punctuation marks are not allowed

printenv lists all available variables

environmental vars are all UPPERCASE

meta-characters are e.g. blanks and spaces

dollar
star
semicolon
ampersand

meta-characters must be quoted to prevent the shell from interpreting them as such

double quotes, single quotes, backslash are ways to quote
or turn off special char capabilities

echo Hello\; World

echoes

Hello; World

echo "I have \$1200"

echoes

I have $1200

pwd print working directory
ls list directory contents
touch create empty file

touch text1.txt

ls can show additional info

ls -la

echo Hello; ls
does both commands.

\ escapes one character
"" escapes a whole string

man pages explain usage of commands
what arguments and syntax they accept

keyword search on the man pages:
man -k keyword

ls -a
ls --all

shorthand and longhand version of the command

both do the same thing

info are like man pages but much more detailed

info
p previous
n next
q quit

quoting is the generic name given to the action of protecting shell meta characters from being treated specially by the shell

root of file system - topmost directory
something like the c: drive

root user
super user with full system access

windows uses backslashes
linux uses slashes

/home/username/letters
c:\my documents\letters

windows detects drives during the boot process

linux mounts and unmounts devices

on bootup drives can be mounted automatically

filenames are case sensitive on linux
on windows, filenames are not case sensitive

files in linux do not need to have a file extension

windows and linux has normal and hidden files

hidden files are not shown by default

on linux, only the owner of a file or dir can grant access

access permissions are read write execute

directory layout is somewhat standardized
fhs filesystem hierarchy standard

shareable unshareable static variable files

linux uses forward slash as dir separator

pwd print working directory

absolute references /
home dir references ~/
relative references filename/path/to/somewhere

hidden subdirs . ..
. this dir
.. parent dir

cp copies files
cp source dest
cp [OPTION] source dest

mv move and rename files

linux can link one file as multiple links
hard link
symbolic link

hard link
duplicate dir entry for the same file
ln origname linkname

symlink
refers to another file by name
ln -s origname linkname

in windows the only thing like this is "create desktop shortcut"

symlink is a new name for a file that already exists

wildcard - a symbol that stands for other characters
"globbing"

b??l ... bowl, ball, bool, ....

asterisk - will match any character

b**l ...

bracketed value [ ]
b[ao][lw]l ... bowl, ball, NOT bull!

careful with wildcards!!!

linux is case sensitive!
therefore grep -ri exists

in windows, they don't treat things with case sensitivity

linux filesystems are case sensitive
win filesystems under linux are case insensitive

FAT NTFS ... cross platform compatibility can depend on case sensitivity

rmdir only removes empty dirs

rm -r removes everything, recursive delete

directories are just special files
a file that holds other files

touch used with a dir updates dir's time stamp

mv uses dirs or files

ln -s can link to a dir

hidden files have a dot as first character

ls ba??.txt
lists everything starting with "ba" and two more chars
and the ending ".txt"

globbing is using for certain things,
while you slice and dice the information
using ?? ** [] expressions

pipe |

output of first command into stdin of second command

grep searches for keywords ....
find specified strings and return filename

grep takes regular expressions

input / output redirection :
xargs generate command line options from programs outputs

> creates new file containing stdout
>> concatenates existing files, adds at the end
2> creates new file with std errors
2>> appends std errors to file
&> creates new file with stdout and stderr
< uses as stdin
<< looks up the following as stdin
<> specified file as stdin and stout

stdout
stderr
stdin

/dev/null is a trashcan device

ls > myoutput
cat myoutput

wc word count
wc -l counts lines

< emulates input from a file as if it came from the keyboard

ls video.mpeg blah.foo > myoutput 2>&1

2>&1 what is &1 ? stdout

head -3 give me the first three things!

ls | head -4 give me the first four listings!

ls | tail -2 last 2!

<< operator allows the user to accept text on the following lines as standard input

basic and extended regular expressions

the ^ represents the start of a line,
the $ represents the end of a line

conf and log files are text files

vi nano emacs

nano is a clone of pico,
more lightweight than emacs

vi commands
^ get me to the start of the line
w jumps word forward
3w jumps 3 words forward
3b jumps 3 words back
set nu adds line numbers
: enters command mode

shell script must be executable

scripts are able to run commands

& ampersand allows multiple commands to be launched

otherwise the script waits for programs to finish

find locates files

echo displays messages

$0 name of script
$1 first param
$2 second param ....

vars can also be assigned inside the script

$variable

errors are bugs

valuable system commands:
hostname date uptime

use logic in shell scripts to handle errors

conditional expressions
if

if [-f file]

if [-s file]
if file is greater than zero

if [string1==string2]

&& and || operators

logical and , logical or

and : both sides of condition must be true
or : one side of condition must be true

if [ -s /tmp/tempstuff ]

then
echo "/tmp/tempstuff found; aborting :-("
exit
fi

if [condition]
then
bla
else
otherbla
fi

case word in
pattern1) commands;
pattern2) commands;

for loops
for d in '/bin/ls .wav';
do aplay $d

done

seq loops
seq 1 10
seq 1 2 10
increments by 2

while loop
as long as condition is true, do thins

until
executes until some condition is met

functions
myfn() {
commands
}

keyword function may optionally precede the function name

function myfn() {
commands
}

exit causes immediate termination of script

scripts have exit values
numeric values between 0 - 255

$termcause = 0
exit $termcause

exit with errorcode to the user or the follow up script

by convention, environment variable names are all uppercase.

a running program is called a process

packages vary between distros

each package is a single file

packages are somehow like compressed folders

packages contain version info and can have dependencies

architecture information

packages are not necessarily binaries, can also contain sourcecode

package managers resolve dependencies, so you install
whole dependency trees

command to install
dependency location
user approval
download and installation

package management requires root access

deb package manager apt

dnf is modernized yum, fedora

yum still used by centos

it's good to keep the package manager updated weekly...

package managers can search and install from the command line

apt search chrome

apt remove package

sudo apt autoremove

security: uninstall unused programs

apt show package
shows info about package

you don't need to build ten packages from source before using your target package
the package manager does it

on servers, there is no gui - so you must use package managers from the command line

kernel manages memory, hardware, devices....

kernel calls init process
init calls logins ... etc
children of init
parent process that launched a process
child process

processes form a tree of parents and their children

processes have PID s .... process id's
and PPID parent process's id

ps lists processes
top process table with active processes at the top

free shows mem usage

ps ax | grep gedit
use pipe to look for specific processes

top
dynamic realtime view of a running system

ps
information for a single moment in time

top is interactive ps

load average shows hung up processes
if necessary kill or stop processes using pid number

sorting by memory shows memory leaks

free shows overall memory use

mem - total ram statistics
used - free - total memory
swap - is used as addition to physical ram if not enough memory available

logfiles daemons

daemons write logfiles

/var/log
common dir for logfiles

boot.log
summarizes start of services late in the boot process
cups/
print services
gdm/

syslog general purpose logfile
secure security related log

Xorg.0.log
X window system logfile

logs are frequently rotated
old ones are deleted, new ones created

syslog / syslogd
started during boot process

kernel logging
klog / klogd

the kernel ring buffer
stored in memory
its contents are continuing to change
while the computer runs
dmesg shows kernel ring buffer messages

diagnosing hardware and driver problems

can be redirected to log file to avoid missing things that fall out of the buffer

the package system is installed as part of the distro by default

what to do if networking is not automatically setup!

domain name system
"the internet phonebook"

example.com -> ip number

dhcp
gain config info from another computer automatically

ethernet
wired network hardware

wireless uses ethernet technique over the are

hostnames ... make computer names memorable
you don't have to memorize numbers.

internet - net of nets
tcp - ip

ip address - phone number for a computer

ip internet protocol

computers communicate via ip addresses

IPv4 four byte addresses
IPv6 six byte addresses

netmask
network portion - computer portion of address

subnet mask

router - connects two ore more networks together
"gateway"

single device for switch router modem in one ...

tcp/ip is a set of standards
transmission control protocol / internet protocol

wi-fi wireless networking

creating a network connection
assigning ip address and netmask

netmask / subnet mask denotes the same

dhcp server sends automatic configuration
ip addresses fixed / dynamic
your computer gets the same ip every time
your computer gets a different ip every time

configure wireless with gui
easy

iwlist
iwconfig
wireless network tools

wired network tools
ifconfig
route
/etc/resolv.conf
dhcp client
dhclient, dhcpd

distro specific net scripts

ping sends packets and test connection

traceroute

dns domain name servers

host dig nslookup

netstat swissknife net tool
lists open ports

linux risk is in unused server programs running

enable a firewall for security
allows or denies traffic on specific criteria

use good passwords!

networking is a threat, you can be tricked

keep your software up to date for the latest patches

on a server you might want to set a static ip address
address netmask gateway

ip address show
shows ip address!

ip link show
only gives the first two lines of
ip address show

sudo ip link set enp0s3 down
turns off the specified wired connection
interface down

sudo ip link set enp0s3 up
turn it back on
interface up

dhcp server tend to remember ip addresses, so there is a chance you get the same ip again

sudo ip addr add 192.168.1.10/24 dev enp0s3
set interface to specified ip address with subnet mask 255.255.255.0

man ip
all needed info for networking on the commandline

dhcp dynamic host configuration protocol

network config needs
ip address, subnet mask, gateway, dns server

a firewall is a program that polices network transactions, allowing or disallowing them based on programmed criteria.

username password uid gid
user - id group - id

/home directory is set for user
default shell is set for user

passwords as "salted hash"

/etc/password contains uid gid and stuff
/etc/shadow contains passwords

passwords can have a time period for which they are valid

an account expires - and yet still may exist.
deactivation comes last.

an expired account still has its password.
deactivated accounts lose their password.

groups are collections of accounts

group name, gid
groups can have passwords

user list - specifies the users in the group

newgrp group2

who --- who is currently using the computer
whoami --- display current user id

id --- show current uid and gid
and all groups of the user

login date and time, remote host

w is similar to who but more verbose
session's idle time , jcpu

create new user
adduser jason
asks for new unix password:

skel is the skeleton for what a normal new user's home dir would be...

sudo useradd -s /bin/bash -d /home/jason -m G jason jason
ubuntu tool

doesn't set passwd, call
sudo passwd jason

user forgot password !=!?!
sudo passwd jason
enter new unix password: __

sudo passwd -S jason
prints status of account
checks if there is a valid good password
gives date of last password change

sudo chage -l jason
same as passwd -S user, but better readable

id jason gives id and groups of jason
usermod -l janson jason
modifies the user

changes name of user "jason" to "janson"
but leaves groups unmodified

try
sudo groupmod -n janson jason

janson got fired
sudo deluser --remove-home janson

again the group stays...
grep '^ja' -nu /etc/passwd
sudo userdel -r jasondion
deletes everything

/var/log/auth.log
logs all user and group activities

cyber security, hello there!
grep your way through auth.log :-=)=

sudo useradd -m tim -p P@$$w0rd
id tim

by default ubuntu creates groups that match the user

sudo groupadd students
sudo groupadd teachers
sudo groupadd admin

sudo usermod -a -G admin tim
add group admin to tim

sudo usermod -a -G teachers diontraining

who's in the admin group?
grep admin /etc/group

grep teachers /etc/group

change group admin to staff
sudo groupmod -n Staff admin

some distros don't let you login as root

su command
lets you switch to root on the shell

sudo issues one command as root

don't run
rm .*
as root

anyone with root password has godlike permissions on that machine

the what column of the output of w
tells what program the session is running


files have owner and permission
linux as multiuser os needs it

everything is treated as a file inside linux
processes, directories

files have uid and gid

root user can do what he wants with any file

regular users can only manipulate their files

changing the ownership
chown

chown jason file
makes file owned by jason

chgrp change group

chgrp jason file
makes file belong to group

directory's owner can manipulate dir

chown tim:Staff foo.txt
makes foo.txt belong to user tim and group Staff


ls -l test
lists permissions, links (hardlinks), owner, group, size, timestamp, filename

permission string
file type code
- normal file
d directory
| symbolic link
p named pipe
s socket
b block device
c character device

owner permissions
group permissions
world permissions
rwx
read write execute

normal data file
-rwxr-xr-x

read write
4 6 2
7
3 5
1
execute

rwxrwxrwx 777
rwxr-x--- 750
rw-r--r-- 644
r-------- 400

777 read write execute for everyone
750 owner has full permissions, group can read and execute
644 owner can read / write, group and world can read
400 owner can read, group and world have no permissions

symlinks always have 777

whether you can create files depends on the directory permissions you're in

root even writes 000 files ))
muhaha

umask determines default permissions for new files
666 .... the number of the beast
-rw-rw-rw-

special files

directories are special filetypes
create or delete a file depends on directory permissions

sticky bit set on directory
user may only delete own files or files stored in own directory

octal code for sticky bit
1755
sets sticky bit
0755
removes sticky bit

chmod o+t
set sticky bit on subdir
chmod o-t
remove sticky bit on subdir

suid
set user id
run the program with permissions of file owner
rather than with the permissions of user who runs the program

sgid
set group id
set the group of the running program to the group of the file

chmod stuff

4 sets suid bit
2 sets sgid
6 sets both bits

u+s sets suid
g+s sets sgid
ug sets both

hidden files on linux have filenames prepended with a .
dotfiles

configuration files

ls -a
shows hidden files and directories

.
current directory
..
parent directory

permissions
r = 4
w = 2
x = 1

chmod 777 foo1.txt
everyone can do everything

chmod 1644 foo1.txt
set sticky bit of foo1.txt

see the "T"
-rw-r--r-T 1 ptero ptero 0 Jan 10 16:28 foo1.txt

chmod 0644 foo1.txt
unset the sticky bit

octal code 644
symbolic code rw-r--r--

dotfiles are hidden from the ls command, unless option -la is used

hard link contains the data in the target file
soft link does not

man page sections can be
NAME
SYNOPSIS
DESCRIPTION
EXAMPLES
SEE ALSO
OPTIONS
EXIT STATUS
RETURN VALUE
ENVIRONMENT
BUGS
FILES
AUTHOR
REPORTING BUGS
HISTORY
COPYRIGHT

mkdir "dir1 dir1"
creates a directory named dir1 dir1
...useless as it may seem ...

single and double quotes are supported

/proc is a virtual or pseudo filesystem used to interface with the kernel and processes

tty is a name for a terminal connection

tail -c NUM
output the last NUM bytes of file

useradd -M
--no-create-home

which option of the command history can be used to clear the user's history?
history -c

what is a requirement of the gpl license but not the bsd license?
users who modify and distribute the software under the gpl license must make the modifications they made available to the recipients under the same license.

which command can be used to resolve a dns name to an ip address?
host

what is gpl, must stay gpl
in order to stay open source

which file on a linux system is modified to set the maximum number of days before a password must be changed?
/etc/shadow