File Management
Contents
This is a howto explaining how to manage files on a UNIX system. This is very important to get the most out of your shell account.
Basics
The very basics of file management is how to list, create, copy, move and delete files and directories. To create a file, you can open up a text editor, write some text, and then save it. To do this with nano, type:
nano myfile.txt
This will open up the text editor nano. Write some text and press ctrl-x to exit, nano will then ask if you want to save the file. You can also use ctrl-o to just save the file, without exiting nano.
You can also use this command to just create a new empty file:
touch newfile
To create a new directory, use this command:
mkdir mydir
To see your newly created files and directories, use the command ls:
ls
To copy a file, we use the command cp. You have to specify what file(s) to copy, and then their destination. Like this:
cp myfile.txt mydir/
This will copy your file myfile.txt into the directory mydir. To copy a whole directory tree, specify the flag -r like this:
cp -r mydir/ mydir-bak
Note: You should try to learn and use "tab-completion" when you move around files etc. You do this by writing the first couple of letters in a name, and then press Tab on your keyboard. The file/directory name should then be automatically completed. Try it out!
To move in to your newly created directory, use the command cd like this:
cd mydir/
You can move back by using cd .. (moves one level up in the directory tree), or just cd with no arguments to move back to your home directory.
To display what directory you're currently in (working directory), use this command:
pwd
To move a file or directory, use the command mv. Like this:
mv myfile.txt ../
Which would move myfile into the parent directory (one level up in the directory tree.)
You can also use the mv command to rename files, like this:
mv myfile.txt renamedfile.txt
Lastly, to delete files, use the command rm, like this:
rm renamedfile.txt
To remove a whole directory, you have to use the flag -r.
Understanding and listing permissions
In UNIX, all files belongs to a user and a group.
For example, your files on the shell will be owned by yourself, and belong to the group shelluser by default.
You can set permissions for your files based on this owner and group. There are three possible permissions, read, write and execute. And you can set these permissions on the owner, the group, or "others". For example, the owner (you) can read and write, while the group can only read, and "others" (those who are neither the owner or in the group) can't do anything with the file.
To view the permissions on a file, you can use the command ls with the flag -l to get a detailed output of files in a directory:
ls -l -rw-r--r-- 1 independence shelluser 0 2007-11-05 19:32 myfile.txt
Detailed Layout.
-rw-r--r-- 1 independence shelluser 0 2007-11-05 19:32 myfile.txt |\ /\ /\ / | | | | | | | | | | | | | | | | | \-File name | | | | | | | | | \-Time created/modified | | | | | | | | \-Date created/modified | | | | | | | \-Size of file | | | | | | \-Group the file is in | | | | | \-Owner's username | | | | \-Number of hard links (or files in directories) | | | \-Other permissions | | \-Group permissions | \-Owner permissions \-File type
The first 10 characters signify permissions on this file. You can also see the owner and group of the file, when it was last modified, and the actual filename.
The 10 characters are divided into 4 groups. The first group is only one character, and this position displays if this item is a directory, in this case it's not so there's just a - displayed (instead of a d.)
Then there are three groups of three caracters. The first group displays what permissions the owner has got, the second group the group permissions, and the last group permissions for others.
In this case, the first three letter group is "rw-". This means that the owner has read and write permissons on the file, but not execute permissions (which would have been displayed by an x.)
The second and third group are both "r--" which means that the group and others both have only read permissions on the file. This is the default of newly created files.
Changing permissions
To change the owner of a file, you can use the command chown. Like this:
chown independence myfile.txt
This will change the owner of myfile.txt to independence. You can also specify a new group like this:
chown independence:othergroup myfile.txt
You can only change ownership of files that you own yourself, not others' files (duh.)
To change the permissions of a file, we use the command chmod.
We can specify that the group should get write permissions for example:
chmod g+w myfile.txt
The syntax here is first who you want to change the permission for:
- u (user)
- g (group)
- o (others)
- a (all, ie user+group+others)
Then you specify either + or - to signify that you want to add or remove a permission.
Lastly, you specify what permission you want to add or remove:
- r (read)
- w (write)
- x (execute)
Some more examples:
Give the owner user execute permissions on myfile:
chmod u+x myfile.txt
Remove write permissions for others:
chmod o-w myfile.txt
Play around a bit yourself, and view the result with:
ls -l yourfile.txt
Changing Permissions (Numeric)
We can also change permission of user, group, others using numbers. This might be used in other howtos and similar on the Internet so it's good to know, but it's not really needed to manage files. We will get into this now. We will be changing permissions of files with numbers.
So lets first explain how to understand what numbers control what permission, it is rather simple actually, their are only 3 numbers you need to remember.
- read == 4
- write == 2
- execute == 1
Basicly we are just doing fundamental adding, to get the number you first decide what permissions you want to set, then add those numbers together, we need to do this for all 3 categories, user, group, and other. So if we want a permission of read/write but no execute then the number would be 6, and if we wanted read/execute only the number would be 5.
well now that we understand the basic idea, lets do some examples.
The following example will make myfile.txt user read/write/execute, group read/execute, and other read.
chmod 754 myfile.txt ls -l -rwxr-xr-- 1 bushblows shelluser 0 2007-11-05 19:32 myfile.txt
Now we will make myfile.txt user read/write/execute, group read/write, other read/write.
chmod 766 myfile.txt -rwxrw-rw- 1 bushblows shelluser 0 2007-11-05 19:32 myfile.txt
Disk usage
When you list files with ls -l you will get the filesize. However, since this filesize is displayed in only bytes (no prefixes) it can be hard to get a good sense of how large files are. To help us read the output we can use the flag -h which stands for human readable. This will make the filesizes display with prefixes like KB or MB which is much easier to read.
ls -lh -rw-r--r-- 1 independence shelluser 851K 2007-10-09 16:02 latest.tar.gz
You can see the filesize here is 815K (KB or KiB really), which is much easier to read than something like 870766.
To view the size of a whole directory tree, we can use the command du. This will display the sizes of all directories below and including the working directory. To get some nice prefixes again, use the flag -h:
du -h irclogs/BlinkenIRC 3.5M BlinkenIRC/
To view your total disk usage/quota use the command df:
df Filesystem Size Used Avail Use% Mounted on - 1018M 175M 843M 18% /home/independence
Compressing/Archives
Archives and compression are very handy in many different situations, and this is the topic we will talk about in this section. The most generally used archive/compression software in *nix is tar(archive), gzip(compression), and bzip(compression).
Tar is software used for archiving files/directories, but also supports flags to compress as well, we will touch on both a simple archive and an archive with compression.
To just archive a directory with files/directories in it, we would use this command:
tar cf my_archive.tar /destination/of/directory
And our archived file would be my_archive.tar in our current working directory.
So we archived a file, but now we need a file from the archive, we need to un-archive it, we would do so with this command.
tar xf my_archive.tar
Now to archive and compress a file, we will archive with tar and use gzip's compression. It's common to use the file ending .tar.gz (or less common: .tgz) to signify that a file is a gzip-compressed archive. This is how to make such an archive:
tar czf my_archive_compressed.tar.gz /destination/to/directory
So we archived and compressed our file but need a file from inside, so lets un-compress it and un-archive it.
We would do so with this command:
tar xzf my_archive_compressed.tar.gz
Remember to always put the flag f before the archive filename, and not in the middle of the other flags like cfz. This is because the flag f expects an argument which is the filename.
If you replace z with j, you get bzip compression instead of gzip. This is in my experience a better compression for text, but a bit more intense on the CPU.
More flags and usage for tar can be found with the command 'man tar'.
So lets just compress a file with no archiving, we will start with gzip. This command will compress the desired file.
gzip somefile.txt
Now if this file is a directory, then we need an extra flag unlike tar, so we would do so with this command.
gzip -r some_directory
Uncompressing is pretty simple, however we use a different command to uncompress, with this command we would uncompress our file.
gunzip some_file.gz
Finding files
Basic syntax for finding a file with a specific filename in a directory:
find <dir> -iname <filename>
<dir> is the directory where you want to look, ~ (your home directory) for example. <filename> is the name, or parts of a name, of a file you're looking for. Remember to use * if you don't have the entire filename.
Example, finding all files ending with .txt in your home directory:
find ~ -iname "*.txt"
In this example we had to enclose the expression in "" because bash will expand * to files in your current directory otherwise. Also note that we use -iname instead of just -name to match results case insensitive.
Find has many other flags which can be used other than -iname, you can find files with a specific owner, modified at a specific time, or a specific size or with certain permissions set for example. The man-page contains descriptions of all flags and several examples on how to use them.