Files
less
To read a file quite easily you can use less.
It is really good for long files (scrolling) and you can exit with q (similar to vim and man):
less <filename>
Best used for reading long documents due to its fast scrolling. It has very similar keys to vim in terms of use.
man
You can get the manual for programs with man.
man ls
Above would show the manual for how to use ls. It is essentially an extended version of --help
cat
Now if you just want the contents of the file spat out (usually used for small files) you can use:
cat file.txt
This outputs the contents of file.txt as stdout.
tail and head
Similar to cat but:
Tail gives the last 10 lines of the file unless otherwise specified.
This will output the last 4 lines of file.txt
tail -n 4 file.txt
Similarly head gets the first 10 lines unless another number is specified:
This will produce first 10 lines of file.txt and then the first 6 lines of file.txt
head file.txt
head -n 6 file.txt
Below, the -f will keep running and wait for the file to get stuff added to it and will show you as the stuff gets added.
Commonly used for watching logs and other files of similar nature (history, etc.).
tail -f log
Making a directory
Make a single directory
mkdir myDirectory
-p allows you to make multiple nested folders all at once
mkdir -p folder1/folder2/folder3
Creating a file
Create a file (any type of file)
touch file.json
Removing files
To remove a file or empty directory
rm file
If you want to remove multiple files cautiously you can use -i flag.
This will ask you for every python file if you do in fact want to delete it.
rm -i *.py
To remove a directory that isn't empty
rm -r directory
-r refers to recursively deleting all the files and sub-directories in this folder. Use this with caution.
-f refers to force, useful when you know you want to get rid of a lof of files and don't want to be asked about it.
CAUTION using rm permanently deletes items... cannot be recovered... rm -rf should be used with caution
Copying files
Copies contents from one file to another
cp file1.txt file2.txt
To copy an entire folder and everything in it use the -R flag.
cp -R folder1 folder2
Moving files
Moving files is useful for not only moving files but renaming them too.
Moving them:
mv file1.txt my/path/
Renaming them:
mv file1.txt file2.txt
Essentially if the second argument is a directory it will move it and if the second argument is a file, it will rename it.
This can be useful for changing the extension on a file aswell.
Converting a text file to a python file:
mv file.txt file.py
tar
Is essentially a glob of files all stuck together, it is different from a zip file.
tar -cf archive.tar file1.txt folder1
This will place file1.txt and folder1 into a non-compressed tar file
To compress the tar file you can do the following:
tar -zcf archive.tar.gz file1.txt folder1
Notice that -z has to be first in the flag and notice the .gz file extension
To extract a tar file you can do it like so, keep in mind the order of the flags and the destination must exist.
tar -xzf archive.tar.gz -C destination
Wildcards
To create multiple files with one command: (file1.js, file2.js, file3.js, file4.js)
touch file{1,2,3,4}.js
touch file{1..10}.json
*
Will look for everything with what is surrounding the *. Some examples to demonstrate this:
ls file*.txt
ls *.py
ls data*
First command will find every file called file with a .txt extension (file_more.txt, file91934.txt, file.txt, file1.txt)
Second command will find every file with .py extension (data.py, index.py, etc.)
Third command will find every file or directory called data. This includes contents in the directory:
Say you have data.txt and data/, within the data directory you have file1.txt.
dempsey$ ls data*
data.txt
data:
file1.txt
?
Will look for a single character at the position of the ?. Below will find every file with a character after "file", for example: file1.txt filea.txt file7.txt Won't pick up file10.txt, file_data.txt
ls file?.txt
[ ]
You can limit the files you want aswell, for example below will get files with a number 1-5 on the end of it (file1.txt, file4.txt, etc.):
ls file[1-5].txt
ls file[^1-5].txt
The second command there will exclude the first 5.
Keep in mind that this is for one character, so if you have file10.txt, the exclude command won't find it because it will only look for for file[6-9].txt
Useful advanced Wildcards
Best just shown with examples:
dempsey$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z
dempsey$ echo {z..a}
z y x w v u t s r q p o n m l k j i h g f e d c b a
dempsey$ echo {0..100..2}
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
dempsey$ echo {0..100..3}
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 99
dempsey$ echo {0..190..7}
0 7 14 21 28 35 42 49 56 63 70 77 84 91 98 105 112 119 126 133 140 147 154 161 168 175 182 189
dempsey$ echo {a..z}{1..3}
a1 a2 a3 b1 b2 b3 c1 c2 c3 d1 d2 d3 e1 e2 e3 f1 f2 f3 g1 g2 g3 h1 h2 h3 i1 i2 i3 j1 j2 j3 k1 k2 k3 l1 l2 l3 m1 m2 m3 n1 n2 n3 o1 o2 o3 p1 p2 p3 q1 q2 q3 r1 r2 r3 s1 s2 s3 t1 t2 t3 u1 u2 u3 v1 v2 v3 w1 w2 w3 x1 x2 x3 y1 y2 y3 z1 z2 z3
Streams & Pipes
Output Streams
Standard output (stdout) is essentially a stream and we can redirect where this stream goes. A good example is we redirect the output stream into a text file.
echo 'file text is so cool' 1> my-file.txt
This won't print to the console like echo usually does because we redirected the stream which contained the stdout (string) and instead of letting the stream "flow" to the console, we redirect it with 1> to go to a .txt file.
- 1> redirects stdout, replaces what's in the destination
- 1>> redirects stdout, appends to the destination
- 2> redirects stderr, replaces what's in the destination
- 2>> redirects stderr, appends to the destination
- > replaces whats in the destination with both streams (stdout and stderr)
- >> appends to destination location with both streams (stdout and stderr)
Essentially you use the numbers to seperate streams wich is handy if you want output and errors to be seperate (maybe for logging purposes, error logs, etc.). If you don't care to seperate streams, don't use the numbers.
Input Streams
Used to feed programs input. A easy example is feeding cat contents of a file. The < isn't really needed obviously it'd work without the input symbol, but this demonstrates the point
cat < input.txt
A useful example is that if you want to search for something in a file. In this example we are searching for "ls -lash" commands previously used in our bash_history.
grep "ls -lash" < ~/.bash_history.txt
grep is a very useful program that essentially just searches for stuff.
A more complicated example: Passes ls.txt into grep which searches for "ls-error.txt". The standard output stream is passed into a grep.txt file and the standard error stream is ignored.
grep "ls-error.txt" < ls.txt 1> grep.txt 2> /dev/null
NOTE programs such as grep can recognise when you are piping or not and will emit the color if in a pipe to make it easier for the next program to parse it.
Piping
Passing stuff from program to program, rather than file to file or program to file is what we call piping. This is achieved with the | operator. Below we pass the output from cat into the grep program to search for something.
cat ls.txt | grep "ls-error.txt"
You can use a & to run processes in the background, just place it at the end of a command
yes > /dev/null &
Permissions
Users
- Switch users with
su - View users with
cat /etc/passwd
A computer comes with lots of users, many may seem very random. The idea is Principle of Power.
You don't want a user to have power if it doesn't need it to fulfill its role.
So if someone gets access to one of the random users, it won't be able to do any real damage because it wont have access.
Root User / Superuser
Is the user will all the privileges. If someone gets access to this they can do whatever they want.
In order to do things as the superuser you must use the sudo program as a prefix to the rest of your command. This is to ensure safe access to those privileges.
Not every user can run the sudo program because they don't have permission.
Not running commands as the root user can save you as a developer from harmful mistakes
Groups
Rather than giving every user specific permissions, you can give a group permissions and then add users to the group.
- Add a user to a group
sudo usermod --append --groups sudo <user>. (--append --groups === -aG)
What the code means
-rwxrw-r--
Notice the 10 chars above. - The first one represents what it is. Commonly either a dash for a flat file (.txt, .jpg, .py, etc.) or a d for directory. - The next 3 are for the user rwx -> read, write and execute. If it's a dash then the user doesn't have permission to do that - The next 3 are for the group, same as above. - The last 3 are for everyone else not in the group, same as above.
Change Ownership - chown
We can change ownerships of things with chown. Below is an example of changing a file with ownerhip "root root" to "root dempsey". Giving the dempsey user group access.
sudo chown root:dempsey file.txt
Change Permissions - chmod
We can do this as follows. Say we want to give everyone read and write persmissions.
We can use the change mode program, chmod.
sudo chmod u=rw,g=rw,o=rw file.txt
sudo chmod 666 file.txt
Shorthand
- 777 everyone can read, write and execute
- 666 everyone can read and write
- 444 everyone can read
- 760 user can read, write and execute. Group can read and write. Others can't do anything.
- The first digit represents user (u), second represents group (g) and third represents others (o).
- just using + after chmod will add whatever you put for all 3 groups. E.g.
chmod +xwill add execution permissions for all groups.
Random Takeaways
- echo doesn't read from stdin, below doesn't do anything.
cat file.txt | echo
- cat was designed to take input from stdin. If you replace above echo with cat, it will work as expected.