One place for hosting & domains

      Filesystem

      How To Use the pathlib Module to Manipulate Filesystem Paths in Python 3


      The author selected the COVID-19 Relief Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Python 3 includes the pathlib module for manipulating filesystem paths agnostically whatever the operating system. pathlib is similar to the os.path module, but pathlib offers a higher level—and often times more convenient—interface than os.path.

      We can identify files on a computer with hierarchical paths. For example, we might identify the file wave.txt on a computer with this path: /Users/sammy/ocean/wave.txt. Operating systems represent paths slightly differently. Windows might represent the path to the wave.txt file like C:Userssammyoceanwave.txt.

      You might find the pathlib module useful if in your Python program you are creating or moving files on the filesystem, listing files on the filesystem that all match a given extension or pattern, or creating operating system appropriate file paths based on collections of raw strings. While you might be able to use other tools (like the os.path module) to accomplish many of these tasks, the pathlib module allows you to perform these operations with a high degree of readability and minimal amount of code.

      In this tutorial, we’ll go over some of the ways to use the pathlib module to represent and manipulate filesystem paths.

      Prerequisites

      To get the most out of this tutorial, it is recommended to have some familiarity with programming in Python 3. You can review these tutorials for the necessary background information:

      Constructing Path Instances

      The pathlib module provides several classes, but one of the most important is the Path class. Instances of the Path class represent a path to a file or directory on our computer’s filesystem.

      For example, the following code instantiates a Path instance that represents part of the path to a wave.txt file:

      from pathlib import Path
      
      wave = Path("ocean", "wave.txt")
      print(wave)
      

      If we run this code, we’ll receive output like the following:

      Output

      ocean/wave.txt

      from pathlib import Path makes the Path class available to our program. Then Path("ocean", "wave.txt") instantiates a new Path instance. Printing the output shows that Python has added the appropriate operating system separator of / between the two path components we gave it: "ocean" and "wave.txt".

      Note: Depending on your operating system, your output may vary slightly from the example outputs shown in this tutorial. If you are running Windows, for example, your output for this first example might look like oceanwave.txt.

      Currently, the Path object assigned to the wave variable contains a relative path. In other words, ocean/wave.txt might exist in several places on our filesystem. As an example, it may exist in /Users/user_1/ocean/wave.txt or /Users/user_2/research/ocean/wave.txt, but we haven’t specified exactly which one we are referring to. An absolute path, by contrast, unambiguously refers to one location on the filesystem.

      You can use Path.home() to get the absolute path to the home directory of the current user:

      home = Path.home()
      wave_absolute = Path(home, "ocean", "wave.txt")
      print(home)
      print(wave_absolute)
      

      If we run this code, we’ll receive output roughly like the following:

      Output

      /Users/sammy /Users/sammy/ocean/wave.txt

      Note: As mentioned earlier, your output will vary depending on your operating system. Your home directory, of course, will also be different than /Users/sammy.

      Path.home() returns a Path instance with an absolute path to the current user’s home directory. We then pass in this Path instance and the strings "ocean" and "wave.txt" into another Path constructor to create an absolute path to the wave.txt file. The output shows the first line is the home directory, and the second line is the home directory plus ocean/wave.txt.

      This example also illustrates an important feature of the Path class: the Path constructor accepts both strings and preexisting Path objects.

      Let’s look at the support of both strings and Path objects in the Path constructor a little more closely:

      shark = Path(Path.home(), "ocean", "animals", Path("fish", "shark.txt"))
      print(shark)
      

      If we run this Python code, we’ll receive output similar to the following:

      Output

      /Users/sammy/ocean/animals/fish/shark.txt

      shark is a Path to a file that we constructed using both Path objects (Path.home() and Path("fish", "shark.txt")) and strings ("ocean" and "animals"). The Path constructor intelligently handles both types of objects and cleanly joins them using the appropriate operating system separator, in this case /.

      Accessing File Attributes

      Now that we’ve learned how to construct Path instances, let’s review how you can use those instances to access information about a file.

      We can use the name and suffix attributes to access file names and file suffixes:

      wave = Path("ocean", "wave.txt")
      print(wave)
      print(wave.name)
      print(wave.suffix)
      

      Running this code, we’ll receive output similar to the following:

      Output

      /Users/sammy/ocean/wave.txt wave.txt .txt

      This output shows that the name of the file at the end of our path is wave.txt and the suffix of that file is .txt.

      Path instances also offer the with_name function that allow you to seamlessly create a new Path object with a different name:

      wave = Path("ocean", "wave.txt")
      tides = wave.with_name("tides.txt")
      print(wave)
      print(tides)
      

      If we run this, we’ll receive output like the following:

      ocean/wave.txt
      ocean/tides.txt
      

      The code first constructs a Path instance that points to a file named wave.txt. Then, we call the with_name method on wave to return a second Path instance that points to a new file named tides.txt. The ocean/ directory portion of the path remains unchanged, leaving the final path as ocean/tides.txt

      Accessing Ancestors

      Sometimes it is useful to access directories that contain a given path. Let’s consider an example:

      shark = Path("ocean", "animals", "fish", "shark.txt")
      print(shark)
      print(shark.parent)
      

      If we run this code, we’ll receive output that looks like the following:

      Output

      ocean/animals/fish/shark.txt ocean/animals/fish

      The parent attribute on a Path instance returns the most immediate ancestor of a given file path. In this case, it returns the directory that contains the shark.txt file: ocean/animals/fish.

      We can access the parent attribute multiple times in a row to traverse up the ancestry tree of a given file:

      shark = Path("ocean", "animals", "fish", "shark.txt")
      print(shark)
      print(shark.parent.parent)
      

      If we run this code, we’ll receive the following output:

      Output

      ocean/animals/fish/shark.txt ocean/animals

      The output is similar to the earlier output, but now we’ve traversed yet another level higher by accessing .parent a second time. Two directories up from shark.txt is the ocean/animals directory.

      Using Glob to List Files

      It’s also possible to use the Path class to list files using the glob method.

      Let’s say we had a directory structure that looked like this:

      └── ocean
          ├── animals
          │   └── fish
          │       └── shark.txt
          ├── tides.txt
          └── wave.txt
      

      An ocean directory contains the files tides.txt and wave.txt. We have a file named shark.txt nested under the ocean directory, an animals directory, and a fish directory: ocean/animals/fish.

      To list all the .txt files in the ocean directory, we could say:

      for txt_path in Path("ocean").glob("*.txt"):
          print(txt_path)
      

      This code would yield output like:

      Output

      ocean/wave.txt ocean/tides.txt

      The "*.txt" glob pattern finds all files ending in .txt. Since the code sample executes that glob in the ocean directory, it returns the two .txt files in the ocean directory: wave.txt and tides.txt.

      Note: If you would like to duplicate the outputs shown in this example, you’ll need to mimic the directory structure illustrated here on your computer.

      We can also use the glob method recursively. To list all the .txt files in the ocean directory and all its subdirectories, we could say:

      for txt_path in Path("ocean").glob("**/*.txt"):
          print(txt_path)
      

      If we run this code, we’d receive output like the following:

      Output

      ocean/wave.txt ocean/tides.txt ocean/animals/fish/shark.txt

      The ** part of the glob pattern will match this directory and all directories beneath it, recursively. So, not only do we have the wave.txt and tides.txt files in the output, but we also receive the shark.txt file that was nested under ocean/animals/fish.

      Computing Relative Paths

      We can use the Path.relative_to method to compute paths relative to one another. The relative_to method is useful when, for example, you want to retrieve a portion of a long file path.

      Consider the following code:

      shark = Path("ocean", "animals", "fish", "shark.txt")
      below_ocean = shark.relative_to(Path("ocean"))
      below_animals = shark.relative_to(Path("ocean", "animals"))
      print(shark)
      print(below_ocean)
      print(below_animals)
      

      If we run this, we’ll receive output like the following:

      Output

      ocean/animals/fish/shark.txt animals/fish/shark.txt fish/shark.txt

      The relative_to method returns a new Path object relative to the given argument. In our example, we compute the Path to shark.txt relative to the ocean directory, and then relative to both the ocean and animals directories.

      If relative_to can’t compute an answer because we give it an unrelated path, it raises a ValueError:

      shark = Path("ocean", "animals", "fish", "shark.txt")
      shark.relative_to(Path("unrelated", "path"))
      

      We’ll receive a ValueError exception raised from this code that will be something like this:

      Output

      Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/Python3.8/pathlib.py", line 899, in relative_to raise ValueError("{!r} does not start with {!r}" ValueError: 'ocean/animals/fish/shark.txt' does not start with 'unrelated/path'

      unrelated/path is not a part of ocean/animals/fish/shark.txt, so there’s no way for Python to compute a relative path for us.

      Conclusion

      The pathlib module is a powerful part of the Python Standard Library that lets us manipulate filesystem paths quickly on any operating system. In this tutorial, we have learned to use some of pathlib’s key utilities for accessing file attributes, listing files with glob patterns, and traversing parent files and directories.

      The pathlib module exposes additional classes and utilities that we did not cover in this tutorial. Now that you have a baseline, you can use the pathlib module’s documentation to learn more about other available classes and utilities.

      If you’re interested in using other Python libraries, check out the following tutorials:



      Source link

      How To Set Filesystem Quotas on Debian 10


      Introduction

      Quotas are used to limit the amount of disk space a user or group can use on a filesystem. Without such limits, a user could fill up the machine’s disk and cause problems for other users and services.

      In this tutorial we will install command line tools to create and inspect disk quotas, then set a quota for an example user.

      Prerequisites

      This tutorial assumes you are logged into a Debian 10 server, with a non-root, sudo-enabled user, as described in Initial Server Setup with Debian 10.

      The techniques in this tutorial should generally work on Linux distributions other than Debian, but may require some adaptation.

      To set and check quotas, we first need to install the quota command line tools using apt. Let’s update our package list, then install the package:

      sudo apt update
      sudo apt install quota
      

      You can verify that the tools are installed by running the quota command and asking for its version information:

      quota --version
      

      Output

      Quota utilities version 4.04. . . .

      It’s fine if your output shows a slightly different version number.

      Next we will update our filesystem’s mount options to enable quotas on our root filesystem.

      Step 2 – Updating Filesystem Mount Options

      To activate quotas on a particular filesystem, we need to mount it with a few quota-related options specified. We do this by updating the filesystem’s entry in the /etc/fstab configuration file. Open that file in your favorite text editor now:

      sudo nano /etc/fstab
      

      The file’s contents will be similar to the following:

      /etc/fstab

      # /etc/fstab: static file system information.
      UUID=06b2aae3-b525-4a4c-9549-0fc6045bd08e   / ext4    errors=remount-ro   0   1
      

      This fstab file is from a virtual server. A desktop or laptop computer will probably have a slightly different looking fstab, but in most cases you’ll have a / or root filesystem that represents all of your disk space.

      Update the line pointing to the root filesystem by adding options as follows:

      /etc/fstab

      # /etc/fstab: static file system information.
      UUID=06b2aae3-b525-4a4c-9549-0fc6045bd08e   /   ext4    errors=remount-ro,usrquota,grpquota   0   1
      

      You will add the new options to the end of any existing options, being sure to separate them all with a comma and no spaces. The above change will allow us to enable both user- (usrquota) and group-based (grpquota) quotas on the filesystem. If you only need one or the other, you may leave out the unused option.

      Remount the filesystem to make the new options take effect:

      sudo mount -o remount /
      

      Note: Be certain there are no spaces between the options listed in your /etc/fstab file. If you put a space after the , comma, you will see an error like the following:

      Output

      mount: /etc/fstab: parse error at line 2 -- ignored

      If you see this message after running the previous mount command, reopen the fstab file, correct any errors, and repeat the mount command before continuing.

      We can verify that the new options were used to mount the filesystem by looking at the /proc/mounts file. Here, we use grep to show only the root filesystem entry in that file:

      cat /proc/mounts | grep ' / '
      

      Output

      /dev/vda1 / ext4 rw,relatime,quota,usrquota,grpquota,errors=remount-ro,data=ordered 0 0

      Note the two options that we specified. Now that we’ve installed our tools and updated our filesystem options, we can turn on the quota system.

      Step 3 – Enabling Quotas

      Before finally turning on the quota system, we need to manually run the quotacheck command once:

      sudo quotacheck -ugm /
      

      This command creates the files /aquota.user and /aquota.group. These files contain information about the limits and usage of the filesystem, and they need to exist before we turn on quota monitoring. The quotacheck parameters we’ve used are:

      • u: specifies that a user-based quota file should be created
      • g: indicates that a group-based quota file should be created
      • m: disables remounting the filesystem as read-only while performing the initial tallying of quotas. Remounting the filesystem as read-only will give more accurate results in case a user is actively saving files during the process, but is not necessary during this initial setup.

      If you don’t need to enable user- or group-based quotas, you can leave off the corresponding quotacheck option.

      We can verify that the appropriate files were created by listing the root directory:

      ls /
      

      Output

      aquota.group bin dev home initrd.img.old lib64 media opt root sbin sys usr vmlinuz aquota.user boot etc initrd.img lib lost+found mnt proc run srv tmp var vmlinuz.old

      If you didn’t include the u or g options in the quotacheck command, the corresponding file will be missing. Now we’re ready to turn on the quota system:

      sudo quotaon -v /
      

      Output

      /dev/vda1 [/]: group quotas turned on /dev/vda1 [/]: user quotas turned on

      Our server is now monitoring and enforcing quotas, but we’ve not set any yet! Next we’ll set a disk quota for a single user.

      Step 4 – Configuring Quotas for a User

      There are a few ways we can set quotas for users or groups. Here, we’ll go over how to set quotas with both the edquota and setquota commands.

      Using edquota to Set a User Quota

      We use the edquota command to edit quotas. Let’s edit our example sammy user’s quota:

      sudo edquota -u sammy
      

      The -u option specifies that this is a user quota we’ll be editing. If you’d like to edit a group’s quota instead, use the -g option in its place.

      This will open up a file in the default text editor, similar to how crontab -e opens a temporary file for you to edit. The file will look similar to this:

      Disk quotas for user sammy (uid 1001):
        Filesystem                   blocks       soft       hard     inodes     soft     hard
        /dev/vda1                        24          0          0          7        0        0
      

      This lists the username and uid, the filesystems that have quotas enabled on them, and the block– and inode-based usage and limits. Setting an inode-based quota would limit how many files and directories a user can create, regardless of the amount of disk space they use. Most people will want block-based quotas, which specifically limit disk space usage. This is what we will configure.

      Note: The concept of a block is poorly specified and can change depending on many factors, including which command line tool is reporting them. In the context of setting quotas on Debian, it’s fairly safe to assume that 1 block equals 1 kilobyte of disk space.

      In the above listing, our user sammy is using 24 blocks, or 24KB of space on the /dev/vda1 drive. The soft and hard limits are both disabled with a 0 value.

      Each type of quota allows you to set both a soft limit and a hard limit. When a user exceeds the soft limit, they are over quota, but they are not immediately prevented from consuming more space or inodes. Instead, some leeway is given: the user has – by default – seven days to get their disk use back under the soft limit. At the end of the seven day grace period, if the user is still over the soft limit it will be treated as a hard limit. A hard limit is less forgiving: all creation of new blocks or inodes is immediately halted when you hit the specified hard limit. This behaves as if the disk is completely out of space: writes will fail, temporary files will fail to be created, and the user will start to see warnings and errors while performing common tasks.

      Let’s update our sammy user to have a block quota with a 100MB soft limit, and a 110MB hard limit:

      Disk quotas for user sammy (uid 1001):
        Filesystem                   blocks       soft       hard     inodes     soft     hard
        /dev/vda1                        24       100M       110M          7        0        0
      

      Save and close the file. To check the new quota we can use the quota command:

      sudo quota -vs sammy
      

      Output

      Disk quotas for user sammy (uid 1001): Filesystem space quota limit grace files quota limit grace /dev/vda1 24K 100M 110M 7 0 0

      The command outputs our current quota status, and shows that our quota is 100M while our limit is 110M. This corresponds to the soft and hard limits respectively.

      Note: If you want your users to be able to check their own quotas without having sudo access, you’ll need to give them permission to read the quota files we created in Step 4. One way to do this would be to make a users group, make those files readable by the users group, and then make sure all your users are also placed in the group.

      To learn more about Linux permissions, including user and group ownership, please read An Introduction to Linux Permissions

      Using setquota to Set a User Quota

      Unlike edquota, setquota will update our user’s quota information in a single command, without an interactive editing step. We will specify the username and the soft and hard limits for both block- and inode-based quotas, and finally the filesystem to apply the quota to:

      sudo setquota -u sammy 200M 220M 0 0 /
      

      The above command will double sammy‘s block-based quota limits to 200 megabytes and 220 megabytes. The 0 0 for inode-based soft and hard limits indicates that they remain unset. This is required even if we’re not setting any inode-based quotas.

      Once again, use the quota command to check our work:

      sudo quota -vs sammy
      

      Output

      Disk quotas for user sammy (uid 1001): Filesystem space quota limit grace files quota limit grace /dev/vda1 24K 200M 220M 7 0 0

      Now that we have set some quotas, let’s find out how to generate a quota report.

      Step 5 – Generating Quota Reports

      To generate a report on current quota usage for all users on a particular filesystem, use the repquota command:

      sudo repquota -s /
      

      Output

      *** Report for user quotas on device /dev/vda1 Block grace time: 7days; Inode grace time: 7days Space limits File limits User used soft hard grace used soft hard grace ---------------------------------------------------------------------- root -- 981M 0K 0K 35234 0 0 nobody -- 7664K 0K 0K 3 0 0 ntp -- 12K 0K 0K 3 0 0 _apt -- 8K 0K 0K 2 0 0 debian -- 16K 0K 0K 4 0 0 sammy -- 24K 200M 220M 7 0 0

      In this instance we’re generating a report for the / root filesystem. The -s command tells repquota to use human-readable numbers when possible. There are a few system users listed, which probably have no quotas set by default. Our user sammy is listed at the bottom, with the amounts used and soft and hard limits.

      Also note the Block grace time: 7days callout, and the grace column. If our user was over the soft limit, the grace column would show how much time they had left to get back under the limit.

      In the next step we’ll update the grace periods for our quota system.

      Step 6 – Configuring a Grace Period for Overages

      We can configure the period of time where a user is allowed to float above the soft limit. We use the setquota command to do so:

      sudo setquota -t 864000 864000 /
      

      The above command sets both the block and inode grace times to 864000 seconds, or 10 days. This setting applies to all users, and both values must be provided even if you don’t use both types of quota (block vs. inode).

      Note that the values must be specified in seconds.

      Run repquota again to check that the changes took effect:

      sudo repquota -s /
      

      Output

      Block grace time: 10days; Inode grace time: 10days . . .

      The changes should be reflected immediately in the repquota output.

      Conclusion

      In this tutorial we installed the quota command line tools, set up a block-based quota for one user, and generated a report on our filesystem’s quota usage.

      The following are some common errors you may see when setting up and manipulating filesystem quotas.

      quotaon Output

      quotaon: cannot find //aquota.group on /dev/vda1 [/] quotaon: cannot find //aquota.user on /dev/vda1 [/]

      This is an error you might see if you tried to turn on quotas (using quotaon) before running the initial quotacheck command. The quotacheck command creates the aquota or quota files needed to turn on the quota system. See Step 3 for more information.

      quota Output

      quota: Cannot open quotafile //aquota.user: Permission denied quota: Cannot open quotafile //aquota.user: Permission denied quota: Cannot open quotafile //quota.user: No such file or directory

      This is the error you’ll see if you run quota and your current user does not have permission to read the quota files for your filesystem. You (or your system administrator) will need to adjust the file permissions appropriately, or use sudo when running commands that require access to the quota file.

      To learn more about Linux permissions, including user and group ownership, please read An Introduction to Linux Permissions



      Source link

      How To Set Filesystem Quotas on Debian 9


      Introduction

      Quotas are used to limit the amount of disk space a user or group can use on a filesystem. Without such limits, a user could fill up the machine’s disk and cause problems for other users and services.

      In this tutorial we will install command line tools to create and inspect disk quotas, then set a quota for an example user.

      Prerequisites

      This tutorial assumes you are logged into a Debian 9 server, with a non-root, sudo-enabled user, as described in Initial Server Setup with Debian 9.

      The techniques in this tutorial should generally work on Linux distributions other than Debian, but may require some adaptation.

      To set and check quotas, we first need to install the quota command line tools using apt. Let’s update our package list, then install the package:

      sudo apt update
      sudo apt install quota
      

      You can verify that the tools are installed by running the quota command and asking for its version information:

      quota --version
      

      Output

      Quota utilities version 4.03. . . .

      It’s fine if your output shows a slightly different version number.

      Next we will update our filesystem’s mount options to enable quotas on our root filesystem.

      Step 2 – Updating Filesystem Mount Options

      To activate quotas on a particular filesystem, we need to mount it with a few quota-related options specified. We do this by updating the filesystem’s entry in the /etc/fstab configuration file. Open that file in your favorite text editor now:

      sudo nano /etc/fstab
      

      The file’s contents will be similar to the following:

      /etc/fstab

      # /etc/fstab: static file system information.
      UUID=06b2aae3-b525-4a4c-9549-0fc6045bd08e   / ext4    errors=remount-ro   0   1
      

      This fstab file is from a virtual server. A desktop or laptop computer will probably have a slightly different looking fstab, but in most cases you’ll have a / or root filesystem that represents all of your disk space.

      Update the line pointing to the root filesystem by adding options as follows:

      /etc/fstab

      # /etc/fstab: static file system information.
      UUID=06b2aae3-b525-4a4c-9549-0fc6045bd08e   /   ext4    errors=remount-ro,usrquota,grpquota   0   1
      

      You will add the new options to the end of any existing options, being sure to separate them all with a comma and no spaces. The above change will allow us to enable both user- (usrquota) and group-based (grpquota) quotas on the filesystem. If you only need one or the other, you may leave out the unused option.

      Remount the filesystem to make the new options take effect:

      sudo mount -o remount /
      

      Note: Be certain there are no spaces between the options listed in your /etc/fstab file. If you put a space after the , comma, you will see an error like the following:

      Output

      mount: /etc/fstab: parse error at line 2 -- ignored

      If you see this message after running the previous mount command, reopen the fstab file, correct any errors, and repeat the mount command before continuing.

      We can verify that the new options were used to mount the filesystem by looking at the /proc/mounts file. Here, we use grep to show only the root filesystem entry in that file:

      cat /proc/mounts | grep ' / '
      

      Output

      /dev/vda1 / ext4 rw,relatime,quota,usrquota,grpquota,errors=remount-ro,data=ordered 0 0

      Note the two options that we specified. Now that we’ve installed our tools and updated our filesystem options, we can turn on the quota system.

      Step 3 – Enabling Quotas

      Before finally turning on the quota system, we need to manually run the quotacheck command once:

      sudo quotacheck -ugm /
      

      This command creates the files /aquota.user and /aquota.group. These files contain information about the limits and usage of the filesystem, and they need to exist before we turn on quota monitoring. The quotacheck parameters we’ve used are:

      • u: specifies that a user-based quota file should be created
      • g: indicates that a group-based quota file should be created
      • m: disables remounting the filesystem as read-only while performing the initial tallying of quotas. Remounting the filesystem as read-only will give more accurate results in case a user is actively saving files during the process, but is not necessary during this initial setup.

      If you don’t need to enable user- or group-based quotas, you can leave off the corresponding quotacheck option.

      We can verify that the appropriate files were created by listing the root directory:

      ls /
      

      Output

      aquota.group bin dev home initrd.img.old lib64 media opt root sbin sys usr vmlinuz aquota.user boot etc initrd.img lib lost+found mnt proc run srv tmp var vmlinuz.old

      If you didn’t include the u or g options in the quotacheck command, the corresponding file will be missing. Now we’re ready to turn on the quota system:

      sudo quotaon -v /
      

      Output

      /dev/vda1 [/]: group quotas turned on /dev/vda1 [/]: user quotas turned on

      Our server is now monitoring and enforcing quotas, but we’ve not set any yet! Next we’ll set a disk quota for a single user.

      Step 4 – Configuring Quotas for a User

      There are a few ways we can set quotas for users or groups. Here, we’ll go over how to set quotas with both the edquota and setquota commands.

      Using edquota to Set a User Quota

      We use the edquota command to edit quotas. Let’s edit our example sammy user’s quota:

      sudo edquota -u sammy
      

      The -u option specifies that this is a user quota we’ll be editing. If you’d like to edit a group’s quota instead, use the -g option in its place.

      This will open up a file in the default text editor, similar to how crontab -e opens a temporary file for you to edit. The file will look similar to this:

      Disk quotas for user sammy (uid 1001):
        Filesystem                   blocks       soft       hard     inodes     soft     hard
        /dev/vda1                        24          0          0          7        0        0
      

      This lists the username and uid, the filesystems that have quotas enabled on them, and the block– and inode-based usage and limits. Setting an inode-based quota would limit how many files and directories a user can create, regardless of the amount of disk space they use. Most people will want block-based quotas, which specifically limit disk space usage. This is what we will configure.

      Note: The concept of a block is poorly specified and can change depending on many factors, including which command line tool is reporting them. In the context of setting quotas on Debian, it’s fairly safe to assume that 1 block equals 1 kilobyte of disk space.

      In the above listing, our user sammy is using 24 blocks, or 24KB of space on the /dev/vda1 drive. The soft and hard limits are both disabled with a 0 value.

      Each type of quota allows you to set both a soft limit and a hard limit. When a user exceeds the soft limit, they are over quota, but they are not immediately prevented from consuming more space or inodes. Instead, some leeway is given: the user has – by default – seven days to get their disk use back under the soft limit. At the end of the seven day grace period, if the user is still over the soft limit it will be treated as a hard limit. A hard limit is less forgiving: all creation of new blocks or inodes is immediately halted when you hit the specified hard limit. This behaves as if the disk is completely out of space: writes will fail, temporary files will fail to be created, and the user will start to see warnings and errors while performing common tasks.

      Let’s update our sammy user to have a block quota with a 100MB soft limit, and a 110MB hard limit:

      Disk quotas for user sammy (uid 1001):
        Filesystem                   blocks       soft       hard     inodes     soft     hard
        /dev/vda1                        24       100M       110M          7        0        0
      

      Save and close the file. To check the new quota we can use the quota command:

      sudo quota -vs sammy
      

      Output

      Disk quotas for user sammy (uid 1001): Filesystem space quota limit grace files quota limit grace /dev/vda1 24K 100M 110M 7 0 0

      The command outputs our current quota status, and shows that our quota is 100M while our limit is 110M. This corresponds to the soft and hard limits respectively.

      Note: If you want your users to be able to check their own quotas without having sudo access, you’ll need to give them permission to read the quota files we created in Step 4. One way to do this would be to make a users group, make those files readable by the users group, and then make sure all your users are also placed in the group.

      To learn more about Linux permissions, including user and group ownership, please read An Introduction to Linux Permissions

      Using setquota to Set a User Quota

      Unlike edquota, setquota will update our user’s quota information in a single command, without an interactive editing step. We will specify the username and the soft and hard limits for both block- and inode-based quotas, and finally the filesystem to apply the quota to:

      sudo setquota -u sammy 200M 220M 0 0 /
      

      The above command will double sammy‘s block-based quota limits to 200 megabytes and 220 megabytes. The 0 0 for inode-based soft and hard limits indicates that they remain unset. This is required even if we’re not setting any inode-based quotas.

      Once again, use the quota command to check our work:

      sudo quota -vs sammy
      

      Output

      Disk quotas for user sammy (uid 1001): Filesystem space quota limit grace files quota limit grace /dev/vda1 24K 200M 220M 7 0 0

      Now that we have set some quotas, let’s find out how to generate a quota report.

      Step 5 – Generating Quota Reports

      To generate a report on current quota usage for all users on a particular filesystem, use the repquota command:

      sudo repquota -s /
      

      Output

      *** Report for user quotas on device /dev/vda1 Block grace time: 7days; Inode grace time: 7days Space limits File limits User used soft hard grace used soft hard grace ---------------------------------------------------------------------- root -- 981M 0K 0K 35234 0 0 nobody -- 7664K 0K 0K 3 0 0 ntp -- 12K 0K 0K 3 0 0 _apt -- 8K 0K 0K 2 0 0 debian -- 16K 0K 0K 4 0 0 sammy -- 24K 200M 220M 7 0 0

      In this instance we’re generating a report for the / root filesystem. The -s command tells repquota to use human-readable numbers when possible. There are a few system users listed, which probably have no quotas set by default. Our user sammy is listed at the bottom, with the amounts used and soft and hard limits.

      Also note the Block grace time: 7days callout, and the grace column. If our user was over the soft limit, the grace column would show how much time they had left to get back under the limit.

      In the next step we’ll update the grace periods for our quota system.

      Step 6 – Configuring a Grace Period for Overages

      We can configure the period of time where a user is allowed to float above the soft limit. We use the setquota command to do so:

      sudo setquota -t 864000 864000 /
      

      The above command sets both the block and inode grace times to 864000 seconds, or 10 days. This setting applies to all users, and both values must be provided even if you don’t use both types of quota (block vs. inode).

      Note that the values must be specified in seconds.

      Run repquota again to check that the changes took effect:

      sudo repquota -s /
      

      Output

      Block grace time: 10days; Inode grace time: 10days . . .

      The changes should be reflected immediately in the repquota output.

      Conclusion

      In this tutorial we installed the quota command line tools, set up a block-based quota for one user, and generated a report on our filesystem’s quota usage.

      The following are some common errors you may see when setting up and manipulating filesystem quotas.

      quotaon Output

      quotaon: cannot find //aquota.group on /dev/vda1 [/] quotaon: cannot find //aquota.user on /dev/vda1 [/]

      This is an error you might see if you tried to turn on quotas (using quotaon) before running the initial quotacheck command. The quotacheck command creates the aquota or quota files needed to turn on the quota system. See Step 3 for more information.

      quota Output

      quota: Cannot open quotafile //aquota.user: Permission denied quota: Cannot open quotafile //aquota.user: Permission denied quota: Cannot open quotafile //quota.user: No such file or directory

      This is the error you’ll see if you run quota and your current user does not have permission to read the quota files for your filesystem. You (or your system administrator) will need to adjust the file permissions appropriately, or use sudo when running commands that require access to the quota file.

      To learn more about Linux permissions, including user and group ownership, please read An Introduction to Linux Permissions



      Source link