Linux Turtorial
In this tutorial we will cover the basics of using a Linux workstation or server to write code.
CS Machines and Accounts
First, check out accounts for information on CS accounts.
All CS students have access to our public computers on the 3rd floor of Trottier and servers accessible by ssh.
Remote Access with SSH
All our machines are accessible from anywhere in the world using ssh. Ssh is installed by default on MacOS, Linux, BSD, and Windows since April 2018 (so make sure you have installed updates.) If you have a Chromebook, you can install an extension to use ssh. If you're using an old version of windows, you'll have to install a program called PuTTY which will let you use ssh. But honestly, it's way better just to update and use ssh natively.
To use ssh, you will first have to open a terminal. On MacOS and Linux, you can open the Terminal App, on Windows it's called CMD.
On all systems (except PuTTY and ChromeOS) you can the enter the command.
ssh username@mimi.cs.mcgill.ca
Where username is the one generated by newuser.cs.mcgill.ca or the firstname.lastname component of your McGill email. You will then be asked to enter your password which is either the one generated with your cs username, or your McGill email password, depending on which account you're using. Note, ssh won't show you little dots or stars when you enter your password, so you just have to try to get it right. If you think you messed it up, just hold delete for a second or two to clear the text you entered.
If you logged in successfully, you are now logged in to a linux machine and can continue on as if you were sitting in from of one of the machines in trottier.
firstname.lastname@mail.mcgill.ca $ ssh firstname.lastname@mimi.cs.mcgill.ca
Note that if you use your mcgill credentials, your files will not be saved after you logout.
See Remote Access for more information.
Prompt
Unlike other operating systems, the primary way of using Linux is via the command line. Instead of clicking, dragging, and scrolling, you enter text commands into an interactive terminal. These commands are then run, doing some task that you may or may not see, and the output is shown to you. Everything is in the form of text, and all tools can interact with each other.
Opening the terminal we see something like
you@lab1-1:~$
Or, more generally:
username@hostname:location$
- username: The account we are using on the machine. This is the same on all McGill CS machines
- hostname: The name of the machine we are using
- location: The directory or folder we are in. this could be something like
Documents, which you have seen in the file browser
This prompt tells us some information we care about, and the $ indicates that we can enter a command. You can customize this prompt to be as useful or useless as you like.
To run a command, type it in to the command prompt area then hit the enter key.
Files and Directories
In Linux, there is a saying that "everything is s a file" which is it's most defining characteristic compared to Windows. Files are exactly the same as on Windows or MacOS. They are a chunk of data on the hard drive, with a name that places it on the filesystem. The filesystem is the hierarchy of files and directories. Directories are like files, but the only data they contain is the locations of other files that are inside them.
A basic outline of the Linux filesystem, ignoring some parts we don't care about:
/
├── bin
├── boot
├── dev
├── etc
├── home
├── 2013
├── 2014
├── you
├── file.py
├── Documents
├── cool-stuff.py
├── Doownloads
├── Music
├── 2015
Don't worry about the meanings of the names for now, the point is that we'll be working in a tree of files and directories contained in other directories.
This is exactly the same is in MacOS or Windows, except here we call the directories instead of folders.
Looking Around
The first thing we should do is figure out where we are. We can use the pwd command. This stands for Present Working Directory,
$ pwd /home/2014/you
This is the home directory. It is the only part of the computer that you can modify, and it is off limits to everyone else.
There is a special shorthand for the home directory, it is ~ and it means the exact same thing as the output of the previous command.
Now let's see what's around.
$ ls Desktop Documents Downloads
Notice these are the same as in the file browser.
We can get more information by using the -l flag (for long)
$ ls -l total 70 drwxr-xr-x 2 you nogroup 5 Sep 5 20:35 Desktop drwxr-xr-x 3 you nogroup 8 Aug 27 11:52 Documents
We can look inside a directory by passing it as an argument to ls
$ ls -l Documents/ total 35 -rw-r--r-- 1 you nogroup 32 Aug 27 11:34 fun.py -rw-r--r-- 1 you nogroup 3890 Aug 27 11:35 fun.txt -rw-r--r-- 1 you nogroup 1187 Aug 27 11:52 linux.html drwxr-xr-x 2 you nogroup 2 Aug 26 14:25 MATLAB -rw-r--r-- 1 you nogroup 813 Aug 27 11:42 'Untitled 1.html'
We can also reference files relative to home using that ~ shorthand.
$ ls ~/Documents/ a.out fun.c fun.py fun.txt linux.html MATLAB 'Untitled 1.html'
Hidden Files
Some files which we don't care about seeing all the time will have a period or "dot" at the front of their name. The are hidden by default both in ls and the file browser. They are exactly the same as other files, and you can use the -a flag to view them.
$ ls -la total 786 drwx--x--x 45 you nogroup 69 Sep 10 19:15 . drwxr-xr-x 3 root root 0 Sep 11 14:46 .. drwxr-xr-x 3 you nogroup 3 Aug 13 16:37 .astropy -rw------- 1 you nogroup 7932 Sep 10 19:15 .bash_history -rw-r--r-- 1 you nogroup 272 Aug 26 14:24 .bashrc drwx------ 31 you nogroup 48 Sep 10 14:32 .cache drwxr-xr-x 34 you nogroup 38 Aug 26 14:30 .config drwx------ 3 you nogroup 3 Mar 6 2019 .dbus drwxr-xr-x 2 you nogroup 5 Sep 5 20:35 Desktop drwxr-xr-x 3 you nogroup 10 Sep 10 16:31 Documents drwxr-xr-x 3 you nogroup 4 Aug 13 16:43 Downloads
. and ..
There are two sort-of special directories that exist in every directory on the system. The allow you to move around relative to the current directory, so you don't have to move everywhere using ~/directory/to/go/to.
$ ls -la total 786 drwx--x--x 45 you nogroup 69 Sep 10 19:15 . drwxr-xr-x 3 root root 0 Sep 11 14:46 ..
. is a reference to the current directory, so you can do things like
$ python ./fun.py $ ls ./ Desktop Documents Downloads
This is mostly useful when you compile code (which we will do later.) When you do that, you get a program out, and have to run it like this:
$ ls fun.c $ gcc fun.c $ ls fun.c a.out $ ./a.out
.. Is a reference to the directory that contains the current directory, so if we're in We don't always want to work in our home directory, sometimes we need to go elsewhere to make running commands easier. To do that we use the There are a bunch of different ways to write a location. Here are most of them. To create an empty file: This isn't super useful in most cases, usually you will just open a file in your editor and if it doesn't already exist, the editor will create it when you save it. This is super useful and you will do it all the time. It's the same as the "New Folder" action in file browsers. Manual (or man) pages are generally awesome. They usually contain all the information you need to use a grogram effectively, and if they don't, it's usually becuase the developer of the program is lazy and shouldn't be trusted. You should get in the habit out reading man pages. They are great. That is all. Let's write a c program that will jsut print some message about how we're feeling to the screen. Add the following code. Then press ctrl-x to exit, anwser Y to saving, and hit enter to save the file as fun.c you could enter a new file name here to create a new file. If you want to view the contents of a file, say fun.c, you can use two commands. This will print the file directly to the screen. It is great for small files. If you have a long file that you would like to scroll through, or search for text, you can use the We can delete files using the You can delete directories by passing the On Linux, renaming somethinng and moving it are the same operation. Use the The You could then modify fun.c and if your idea doesn't work, you can simply delete the modified fun.c and rename fun.c.backup back to fun.c, thus restoring it to the state it was in before you tried that ill-fated idea. Use the We write code in programming languages that are easy for humans to read and write. Computers don't know how to read these languages, they only know binary. So we need a program to turn our code into binary. This is called a compiler. For assignments, the two you will use mostly are Remember when we did The first column on letters and dashes gives us the permissions of the file or directory. The third column is the owner of the file and the fourth is the group of the file. Above, the owner is you and the group is nogroup. The permissions we can give a file are read, write, and execute. Reading is the ability to view the contents of a file, wrinting is the abaility to change the contents of a file, and executing is the ability to run the file as a program. For directories, executing is the ability to go into the directory. Each of those permissions is set for threee classes of users: the owner of the file, users in the group of the file, and everyone else. In the output of ls for b.out, the first group of rwx is for the user (you) the second is for the group (nogroup) and the third is for everyone else. In that case, all permissions are set so anyone can do anything they want. For a.out, you can read and write to it, but not execute it, and on one else can do anything to it. If we want to change the permissions on the files, we can use the Now everyone can execute this program. This is what you will have to do to the c and bash programs you write for your courses before you can test them. There are different ways to run different programs. When you write and compile a c program, you will get a binary execuatable out. Once you've set the right permissions on it, you can run it as follows: For Java, you don't usually have to mess with permissions. Python files don't need to be compiled before use. So you can just write a python file, say spam.py and run it. Normally, when using the ssh command, we have to enter our password every time we log in. This is annoying if you use ssh a lot. Fortunately, there is a really nice system to avoid this. Instead of your password, you can authenticate using an ssh key pair which are just two text files with very special data. On your own computer, you can generate them using the following command That command creates the files This command will place your public key into your CS account. When you try to log in, ssh will see the public key and then look for the private key on your laptop. If the private key is there, it will let you in. Another great ssh-based command is rsync. It allows you to copy files and directories between machines. So if you've been working on some code on your laptop, and would like to transfer it to mimi to test it, you can run the command Make sure to use the -a flag for directories. The general form is: You can search recursively through directories using the So far, we have been running commands and reading the output to the screen. But what if we want to save the output of the command to a file? Say we want to analyze the result of our program, or quickly write a small file. We can use redirection to write test to a file. The As a side note; the The The most interesting part of linux commands is when you use them together. Since they all take text as input, and return text as output, we can send the output of one program into another program to do some work with it. We do this using the One of the most useful patterns is to take a program which outputs a lot of information and pipe its output into grep to find the information you're looking for. The above command shows us only the directories that contain system libraries. Another great use for grep is to find files we care about in a big project. If I have downloaded a large python project and I want to see where all the files called "models.py" are, I could run the command: By itself, the find command just lists every file within the directory you pass to it. So On any linux machine, you can change your password using the On the CS machines, your home directory is on a file server. It is backed up, and will be the same on every machine you use. The downside is that we have limited space so we have to put a limit on how much space each student gets. You can check ihow much space you are using using the If you run out of disk space, you will not be able to log in to the machines in Trottier, ssh will still work fine. Unfortunately, there's no way for us to avoid this. If it happens, you can use ssh to delete some files, namely: The cache is mostly used by browser cookies and other data that is safe to delete. If you're sitting in front of a Trottier machine, and keep getting sent back to the login screen when you try to log in, it's probably because you are ut of space. You can press Some commands to check out what is running on your machine: An interactive list of the processes (programs) running on your machine A list of the processes you are running A list of all the processes running on the machine A list of the disks connected to the machine Information about how much RAM is being used Say you've accidentally written a program with an infinite loop. You can find it and stop it so that it doesn't hog all the system resources. If you want to know where a program lives on the ssystem, you can use the which command. If there are multiple versions of the program in your path, or other files associated with the program, you can see them all using whereis Like variables in programming languages, we can use environment variables to store bits of information. There are a whole bunch of them that get set automatically. You can list them all with the The most usefull ones (at least for now) are the ones relating to the "path." This PATH variable stores the different directories that the shell searches for programs. When you enter a command at the prompt, the shell has to find it in order to run it. If it were to search through the whole computer it would take way to long, so it keeps this small set of places to look. You can see which programs live in each of those directories: There you see bash, which is the shell we are using, and grep, which we have seen. I cut off the output here because it's very long. If you have some of your own programs and would like to be able to use them like any of the built in ones, you can create a directory in your home folder and add it to your path. Now, any programs you put in ~/Documents, ~/
$ pwd
/home/2014/you
$ ls ../
you
$ ls ../../
2000 2002 2004 2006 2008 2010 2012 2014 2016 2018
$ ls ../../../
bin dev home initrd.img.old lib32 lost+found mnt proc run snap sys usr vmlinuz
boot etc initrd.img lib lib64 media opt root sbin srv tmp var vmlinuz.old
Moving Around
cd or change directory command. We pass it the name of the directory we want to go to.
$ cd Documents
$ cd ..
$ cd /home/2014/you/Downloads
$ cd ~/Documents
$ cd ../Music
$ cd
Creating Files
$ touch fun.c
Creating Directories
$ mkdir new-dir
$ ls
fun.c new-dir/
Editing Files
nano is a simple editor for use in a terminal. You can use the arrow keys to move around, but you can't use the mouse. It has many more shortcuts, but we won't talk about them here. Check out the manual page for it, and any other command.
$ man nano
Detour to Talk About Manual Pages
$ man nano
$ apropos editor
atobm (1) - bitmap editor and converter utilities for the X Wind...
bitmap (1) - bitmap editor and converter utilities for the X Wind...
bmtoa (1) - bitmap editor and converter utilities for the X Wind...
dotty (1) - A Customizable Graph Editor
ed (1) - line-oriented text editor
editor (1) - Nano's ANOther editor, an enhanced free Pico clone
apropos searches man pages by keyword, so you would then look up, say editor, in manBack to the Program
$ nano fun.c
#include <stdio.h>
int main() {
printf("Wow, coding is fun!\n");
return 0;
}
Viewing Files
$ cat fun.c
less command, which will enter an interactive pager. Use the arrow, or j and k to scroll, and q to quit.$ less file
Deleting
rm command. Note that this isn't like moving files to the trash, when you use rm, you are removing the systems record of the file. It's like it never existed and there's (almost) no way to get it back.
$ touch bad-idea.c
$ ls
fun.c
bad-idea.c
$ rm bad-idea.c
$ ls
fun.c
-r flag, which is even more dangerous. For the ultimate danger, pass the -f flag which will not warn you of any possible problems.Renaming and Moving
mv command, with the original name and new name as arguments. You can also pass a directory as the last argument and the file will be moved to that directory woith the same name.
$ mv fun.c super-fun.c
$ mv super-fun.c new-dir/
$ mv new-dir/super-fun.c fun.c
Copying
cp command copies files. You pass the name of the file you want to copy, and the name of the new file as arguments and it makes an exact copy which you can them modify. A good use case for this is making backups of files if you want to try an new idea in your code.
$ cp fun.c fun.c.backup
-R flag to recursively copy entire directories.
$ cp -R ~/Documents ~/New
Building Code
gcc, and javac.
$ ls
fun.c wow.java
$ gcc fun.c
$ ls
a.out fun.c wow.java
$ javac fun.java
$ ls
a.out fun.c HelloWorld.class wow.java
Running Code
Detour about Permissions
ls -l and we got all that garbage before the file names? Turns out, it isn't actually garbage. It's actually garbage that will cause you endless headaches the second you forget about it.
$ ls -l
total 786
-rw------- 1 you nogroup 7932 Sep 10 19:15 a.out
-rwxrwxrwx 1 you nogroup 7932 Sep 10 19:15 b.out
chmod command. chmod has a somewhat complicated way of definiging permissions, so we will just us it to let everyone execute a.out for now. We do this by passing a+x, meaning "all add execute", as the first argument, and the file as the second.
$ chmod a+x a.out
$ ls -l
total 786
-rwx--x--x 1 you nogroup 7932 Sep 10 19:15 a.out
-rwxrwxrwx 1 you nogroup 7932 Sep 10 19:15 b.out
Back to Running Things
$ ./a.out
"What did i write earlier?"
$ java HelloWorld.class
Hello World!
$ cat spam.py
print("spam spam spam")
$ python spam.py
spam spam spam
SSH Keys
corey@my-laptop:~ $ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/corey/.ssh/id_rsa):
Created directory '/Users/corey/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/corey/.ssh/id_rsa.
Your public key has been saved in /Users/corey/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:zJ2SQwYgAS4x9kKXthtPetW+2De1GttGHfUACfYVTIU corey@corey-imac.CS.McGill.CA
The key's randomart image is:
+---[RSA 2048]----+
|++ooo.. o..=++.|
|+oo+ . . ...E .|
|.o... + . .o|
|. .o . * + . ..|
| * . S o . .|
| o o + o . |
| . o ..o . |
| . o o+o |
| .o+. |
+----[SHA256]-----+
~/.ssh/id_rsa which is your private key and should never be moved, and ~/.ssh/id_rsa.pub which is your public key and can be copied to any machine you like. These files are generated in such a way that there is no way you could have created one without the other. You can put your public key on a server using the follow command:
corey@my-laptop:~ $ ssh-copy-id you@mimi.cs.mcgill.ca
rsync
corey@my-laptop:~ $ rsync file you@mimi.cs.mcgill.ca:~/file
corey@my-laptop:~ $ rsync -a project/ you@mimi.cs.mcgill.ca:~/project
grep
grep is a program which allows us to search for text. Say you have a large file and you want to find all occurrences of a variable name within it. You could use grep to find them.grep [options] [pattern] [file]
grep -n class Code/socs/socsmain/models.py
15:class Area(models.Model):
19: class Meta:
32:class Lab(models.Model):
40: class Meta:
50:class Interest(models.Model):
54: class Meta:
64:class Page(models.Model):
75: class Meta:
79:class Professor(models.Model):
109: class Meta:
-R flag.grep -R [pattern] [directory]
Redirection to Files
> operator will write the output of a program on its left side to the file on its right side. If the file doesn't exist already, it will create it, and if it does exist, it will overwrite it.echo command just outputs whatever you input to it.
$ echo "echo is not very interesting"
echo is not very interesting
$ echo "for i in range(100): print(i)" > count.py
$ cat count.py
for i in range(100): print(i)
$ python count.py > output.txt
$ less output.txt
>> operator wil append the outut of the program on its left side to the file on its right side. It's the same as > except that if the file exists, it will append its output to the end of the file.
$ echo "wow this is cool\n" >> cool.txt
$ cat cool.txt
wow this is cool
$ echo "this is even cooler\n" >> cool.txt
$ cat cool.txt
wow this is cool
this is even cooler
Redirection Between Programs
| or "pipe" operator. The output of the program on the left will become the input to the program on the right.
$ ls -l /
total 108
drwxr-xr-x 2 root root 4096 Sep 11 06:37 bin
drwxr-xr-x 4 root root 4096 Sep 19 06:59 boot
drwxr-xr-x 19 root root 4000 Sep 4 15:07 dev
drwxr-xr-x 186 root root 12288 Sep 19 06:56 etc
drwxr-xr-x 43 root root 4096 Sep 4 15:33 home
lrwxrwxrwx 1 root root 33 Sep 19 06:58 initrd.img -> boot/initrd.img-4.15.0-64-generic
lrwxrwxrwx 1 root root 33 Sep 19 06:58 initrd.img.old -> boot/initrd.img-4.15.0-62-generic
drwxr-xr-x 25 root root 4096 Sep 4 16:09 lib
drwxr-xr-x 2 root root 4096 Sep 4 16:09 lib32
drwxr-xr-x 2 root root 4096 Jul 25 2018 lib64
drwx------ 2 root root 16384 Sep 4 13:21 lost+found
drwxr-xr-x 2 root root 4096 Jul 25 2018 media
drwxr-xr-x 2 root root 4096 Jul 25 2018 mnt
drwxr-xr-x 6 root root 4096 Sep 18 16:47 opt
dr-xr-xr-x 259 root root 0 Sep 4 14:13 proc
drwx------ 5 root root 4096 Sep 10 15:55 root
drwxr-xr-x 41 root root 2420 Sep 19 15:14 run
drwxr-xr-x 2 root root 12288 Sep 19 06:56 sbin
drwxr-xr-x 4 root root 4096 Sep 4 13:28 snap
drwxr-xr-x 2 root root 4096 Jul 25 2018 srv
dr-xr-xr-x 13 root root 0 Sep 4 20:10 sys
drwxrwxrwt 80 root root 12288 Sep 19 15:09 tmp
drwxr-xr-x 12 root root 4096 Sep 4 16:09 usr
drwxr-xr-x 19 root root 4096 Sep 4 15:05 var
lrwxrwxrwx 1 root root 30 Sep 19 06:58 vmlinuz -> boot/vmlinuz-4.15.0-64-generic
lrwxrwxrwx 1 root root 30 Sep 19 06:58 vmlinuz.old -> boot/vmlinuz-4.15.0-62-generic
$ ls -l / | grep lib
drwxr-xr-x 25 root root 4096 Sep 4 16:09 lib
drwxr-xr-x 2 root root 4096 Sep 4 16:09 lib32
drwxr-xr-x 2 root root 4096 Jul 25 2018 lib64
$ find ~/Code/socs | grep models.py
/home/2014/you/Code/socs/slides/models.py
/home/2014/you/Code/socs/socsmain/models.py
find . will show you every file under the current directory, find ~ will show you every file in your home directory, and find / will show you every file on the computer.Commands to Manage Your Account
passwd command.quota command.
$ quota
TYPE NAME USED QUOTA
POSIX User you 7.46G 10G
$ rm -rf ~/.cache
ctrl+alt+F3 to log in to a terminal. This terminal functions exactly like ssh or the terminal app. When you're done, pres ctrl+alt+F1 to return to the login screen.Commands to Check on the System
$ top
$ ps
PID TTY TIME CMD
6167 pts/10 00:00:00 bash
21678 pts/10 00:00:00 ps
$ ps aux | head
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.7 0.0 227792 10784 ? Ss Sep04 168:43 /lib/systemd/systemd --system --deserialize 38
root 2 0.0 0.0 0 0 ? S Sep04 0:00 [kthreadd]
root 6 0.0 0.0 0 0 ? I Sep04 0:00 [mm_percpu_wq]
root 7 0.0 0.0 0 0 ? S Sep04 0:25 [ksoftirqd/0]
root 8 0.0 0.0 0 0 ? I Sep04 12:05 [rcu_sched]
root 9 0.0 0.0 0 0 ? I Sep04 0:00 [rcu_bh]
root 10 0.0 0.0 0 0 ? S Sep04 0:04 [migration/0]
root 11 0.0 0.0 0 0 ? S Sep04 0:02 [watchdog/0]
root 12 0.0 0.0 0 0 ? S Sep04 0:00 [cpuhp/0]
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0 0 238.5G 0 disk
├─nvme0n1p1 259:1 0 512M 0 part /boot/efi
└─nvme0n1p2 259:2 0 238G 0 part
├─lab1--1--vg-root 253:0 0 237G 0 lvm /
└─lab1--1--vg-swap_1 253:1 0 980M 0 lvm [SWAP]
$ free -h
total used free shared buff/cache available
Mem: 15G 617M 13G 6.9M 1.3G 14G
Swap: 979M 0B 979M
Stopping Badly Behaving Programs
$ echo 'while True: print("baaaaad")' > bad-prorgam.py
$ python bad-program.py
# in another terminal, since the first is locked up
$ ps | grep bad-program.py
21680 pts/10 00:00:00 bad-program.py
$ pgrep bad-program
21680
$ kill 21680
which and whereis
$ which cat
/bin/cat
$ whereis passwd
passwd: /usr/bin/passwd /etc/passwd /usr/local/bin/passwd /usr/share/man/man5/passwd.5.gz /usr/share/man/man1/passwd.1ssl.gz /usr/share/man/man1/passwd.1.gz
Environment Variables
set command.
$ set
BASH=/bin/bash
BASHOPTS=checkwinsize:cmdhist:complete_fullquote:expand_aliases:extglob:extquote:force_fignore:interactive_comments:login_shell:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_COMPLETION_VERSINFO=([0]="2" [1]="8")
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="4" [1]="4" [2]="20" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu")
PATH=/var/bin:/usr/local/cuda/bin:/usr/local/pkgs/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/puppetlabs/bin
$ ls /bin
bash fuser nisdomainname stty
brltty fusermount ntfs-3g su
bunzip2 getfacl ntfs-3g.probe sync
busybox grep ntfscat systemctl
$ mkdir ~/bin
$ export PATH="~/bin:$PATH"
~/bin will be available without using the ./ method of calling them.
