One place for hosting & domains

      Install Plex Media Server on Ubuntu 18.04 Using Salt Masterless


      Updated by Linode Contributed by Linode

      Plex is a media server that allows you to stream video and audio content that you own to many different types of devices. In this guide you will learn how to use a masterless Salt minion to set up a Plex server, attach and use a Block Storage Volume, and how to connect to your media server to stream content to your devices.

      Before You Begin

      1. Familiarize yourself with our Getting Started guide and complete the steps for setting your Linode’s hostname and timezone.

      2. Follow the steps in the How to Secure Your Server guide.

      3. Update your system:

        sudo apt-get update && sudo apt-get upgrade
        
      4. You will need to create a Block Storage Volume and attach it to your Linode. You will format and mount the drive as part of this guide. This volume will be used to store your media, so you should pick a size that’s appropriate for your media collection, though you can resize the volume later if you need more storage. For more on Block Storage, see our How to Use Block Storage guide.

      5. Plex requires an account to use their service. Visit the Plex website to sign up for an account if you do not already have one.

      Note

      The steps in this guide require root privileges. Be sure to run the steps below with the sudo prefix. For more information on privileges, see our Users and Groups guide.

      Prepare the Salt Minion

      1. On your Linode, create the /srv/salt and /srv/pillar directories. These are where the Salt state files and Pillar files will be housed.

        mkdir /srv/salt && mkdir /srv/pillar
        
      2. Install salt-minion via the Salt bootstrap script:

        curl -L https://bootstrap.saltstack.com -o bootstrap_salt.sh
        sudo sh bootstrap_salt.sh
        
      3. The Salt minion will use the official Plex Salt Formula, which is hosted on the SaltStack GitHub repository. In order to use a Salt formula hosted on an external repository, you will need GitPython installed. Install GitPython:

        sudo apt-get install python-git
        

      Modify the Salt Minion Configuration

      1. Because the Salt minion is running in masterless mode, you will need to modify the minion configuration file (/etc/salt/minion) to instruct Salt to look for state files locally. Open the minion configuration file in a text editor, uncomment the line #file_client: remote, and set it to local:

        /etc/salt/minion
        1
        2
        3
        4
        5
        6
        7
        8
        9
        
        ...
        
        # Set the file client. The client defaults to looking on the master server for
        # files, but can be directed to look at the local file directory setting
        # defined below by setting it to "local". Setting a local file_client runs the
        # minion in masterless mode.
        file_client: local
        
        ...
      2. There are some configuration values that do not normally exist in /etc/salt/minion which you will need to add in order to run your minion in masterless mode. Copy the following lines into the end of /etc/salt/minion:

        /etc/salt/minion
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        
        ...
        
        fileserver_backend:
          - roots
          - gitfs
        
        gitfs_remotes:
          - https://github.com/saltstack-formulas/plex-formula.git
        
        gitfs_provider: gitpython

        The fileserver_backend block instructs the Salt minion to look for Salt configuration files in two places. First, it tells Salt to look for Salt state files in our minion’s roots backend (/srv/salt). Secondly, it instructs Salt to use the Git Fileserver (gitfs) to look for Salt configuration files in any Git remote repositories that have been named in the gitfs_remotes section. The address for the Plex Salt formula’s Git repository is included in the gitfs_remotes section.

        Note

        It is best practice to create a fork of the Plex formula’s Git repository on GitHub and to add your fork’s Git repository address in the gitfs_remotes section. This will ensure that any further changes to the upstream Plex formula which might break your current configuration can be reviewed and handled accordingly, before applying them.

        Lastly, GitPython is specified as the gitfs_provider.

      Create the Salt State Tree

      1. Create a Salt state top file at /srv/salt/top.sls and copy in the following configuration. This file tells Salt to look for state files in the plex folder of the Plex formula’s Git repository, and for a state files named disk.sls and directory.sls, which you will create in the next steps.

        /srv/salt/top.sls
        1
        2
        3
        4
        5
        
        base:
          '*':
            - plex
            - disk
            - directory
      2. Create the disk.sls file in /srv/salt:

        /srv/salt/disk.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        
        disk.format:
          module.run:
            - device: /dev/disk/by-id/scsi-0Linode_Volume_{{ pillar['volume_name'] }}
            - fs_type: ext4
        
        /mnt/plex:
          mount.mounted:
            - device: /dev/disk/by-id/scsi-0Linode_Volume_{{ pillar['volume_name'] }}
            - fstype: ext4
            - mkmnt: True
            - persist: True

        This file instructs Salt to prepare your Block Storage Volume for use with Plex. It first formats your Block Storage Volume with the ext4 filesystem type by using the disk.format Salt module, which can be run in a state file using module.run. Then disk.sls instructs Salt to mount your volume at /mnt/plex, creating the mount target if it does not already exist with mkmnt, and persisting the mount to /etc/fstab so that the volume is always mounted at boot.

      3. Create the directory.sls file in /srv/salt:

        /srv/salt/directory.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        
        /mnt/plex/plex-media:
          file.directory:
            - require:
              - mount: /mnt/plex
            - user: username
            - group: plex
        
        /mnt/plex/plex-media/movies:
          file.directory:
            - require:
              - mount: /mnt/plex
            - user: username
            - group: plex
        
        /mnt/plex/plex-media/television:
          file.directory:
            - require:
              - mount: /mnt/plex
            - user: username
            - group: plex

        The directories that are created during this step are for organizational purposes, and will house your media. Make sure you replace username with the name of the limited user account you created when following the How to Secure Your Server guide. The location of the directories is the volume you mounted in the previous step. If you wish to add more directories, perhaps one for your music media, you can do so here, just be sure to include the - require block, as this prevents Salt from trying to create the directory before the Block Storage Volume has been mounted.

      4. Go to the Plex Media Server download page and note the most recent version of their Linux distribution. At the time of writing, the most recent version is 1.13.9.5456-ecd600442. Create the plex.sls Pillar file in /srv/pillar and change the Plex version number and the name of your Block Storage Volume as necessary:

        /srv/pillar/plex.sls
        1
        2
        3
        
        plex:
          version: 1.13.9.5456-ecd600442
        volume_name: plex
      5. Create the Salt Pillar top file in /srv/pillar. This file will instruct Salt to look for the plex.sls Pillar file you created in the previous step.

        /srv/pillar/top.sls
      6. Apply your Salt state locally using salt-call:

        salt-call --local state.apply
        

        You should see a list of the changes Salt applied to your system. You have successfully installed Plex using Salt.

      Set Up Plex

      Initial Configuration

      1. You’ll need to create an SSH tunnel to your Linode to connect to Plex’s web UI. On your local computer, run the following command, replacing <your_ip_address> with your Plex server’s IP address.:

        ssh username@<your_ip_address> -L 8888:localhost:32400
        
      2. In a browser, navigate to http://localhost:8888/web/.

      3. Sign in with your Plex username and password.

      4. Name your media server. This example uses the name linode-plex. Be sure to check the box that reads Allow me to access my media outside my home and then click Next.

        Name your media server

      Organize Your Media

      1. Click on the Add Library button:

        Click on Add Media

      2. Select Movies and click Next:

        Select Movies and click next

      3. Click Browse for Media Folder and select the appropriate folder at /mnt/plex/plex-media/movies. Then click Add:

        Select the appropriate folder

      4. Repeat the process to add your ‘Television’ folder.

      5. When you are done adding your libraries, click Add Library.

      6. To continue the configuration process, click Next.

      7. Click on Get Plex Apps to download the appropriate Plex client for your device. Then click Done.

        Download the appropriate client for your device

      8. In the future you can add more libraries by hovering over the menu and clicking the plus sign (+) next to LIBRARIES.

        Add more libraries

      DLNA is a protocol that incorporates Universal Plug and Play (or UPnP) standards for digital media sharing across devices. If you do not wish to make use of it, it’s recommended that you disable this feature, as it is openly connectable on port 1900. From the Plex web interface, click the wrench icon in the upper right corner, and navigate to the DLNA section under SETTINGS. Uncheck Enable the DLNA server, and click Save Changes:

      Disable DLNA

      Connect to Your Plex Server

      1. Visit the Plex Apps download page or the app store on your device to download Plex Media Player if you have not already done so.

      2. Open your Plex app. The example provided here will use the Plex Media Player for macOS.

      3. Sign in to Plex.

      4. On the left there’s a dropdown menu where you can select your server by the name you chose. Select your server.

        Connect to your Plex Server

      5. You are now able to stream your content with Plex.

        Plex's macOS App

      1. You can use SCP to transfer media to your server from your local computer. Replace your username and 123.456.7.8 with the IP address of your Linode.

        scp example_video.mp4 [email protected]:/mnt/plex/plex-media/movies
        
      2. Once you’ve transferred files to your Plex media server, you may need to scan for new files before they show up in your Library. Click on the ellipsis next to a Library and select Scan Library Files.

        Scan your Library for new files

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Automate Static Site Deployments with Salt, Git, and Webhooks


      Updated by Linode Contributed by Nathan Melehan

      Use promo code DOCS10 for $10 credit on a new account.

      This guide will walk through the deployment of a static site using SaltStack, which is a flexible configuration management system. The configuration files created for Salt will be version controlled using Git. Updates to your static site’s code will be automatically communicated to the production system using webhooks, an event notification system for the web.

      Setting up these mechanisms offers an array of benefits:

      • Using webhooks will keep your production website in sync with your development without any actions needed on your part.

      • Using Salt provides an extensible, reliable way to alter your production systems and minimize human error.

      • Version controlling your configuration management helps you track or revert the changes you’ve made to your systems and collaborate with others on your deployments.

      Development and Deployment Workflow

      The static site generator used in this guide is Hugo, a fast framework written in Go. Static site generators compile markdown or other content files into HTML files. This guide can easily be adapted to other frameworks.

      Two Git repositories will be created: one will track changes to the Hugo site, and the other will track Salt’s configuration files. Remote repositories will be created for both on GitHub.

      Two Linodes will be created: one will act as the Salt master, and the other as the Salt minion. This guide was tested under Debian 9, but the instructions may work with other distributions as well. The Salt minion will run the production webserver which serves the Hugo site, and the master will configure the minion’s software. The minion will also run a webhook server which will receive code update notifications from GitHub.

      It is possible to run Salt in a masterless mode, but using a Salt master will make it easier to expand on your deployment in the future.

      Note

      The workflow described in this guide is similar to how Linode’s own Guides & Tutorials website is developed and deployed.

      Before You Begin

      Set Up the Development Environment

      Development of your Hugo site and your Salt formula will take place on your personal computer. Some software will need to be installed on your computer first:

      1. Install Git using one of the methods in Linode’s guide. If you have a Mac, use the Homebrew method, as it will also be used to install Hugo.

      2. Install Hugo. The Hugo documentation has a full list of installation methods, and instructions for some popular platforms are as follows:

        • Debian/Ubuntu:

          sudo apt-get install hugo
          
        • Fedora, Red Hat and CentOS:

          sudo dnf install hugo
          
        • Mac, using Homebrew:

          brew install hugo
          
        • Windows, using Chocolatey

          choco install hugo -confirm
          

      Deploy the Linodes

      1. Follow the Getting Started guide and deploy two Linodes running Debian 9.

      2. In the settings tab of your Linodes’ dashboards, label one of the Linodes as salt-master and the other as salt-minion. This is not required, but it will help keep track of which Linode serves which purpose.

      3. Complete the Securing Your Server guide on each Linode to create a limited Linux user account with sudo privileges, harden SSH access, and remove unnecessary network services.

        Note

        This guide is written for a non-root user. Commands that require elevated privileges are prefixed with sudo. If you’re not familiar with the sudo command, visit our Users and Groups guide.

        All configuration files should be edited with elevated privileges. Remember to include sudo before running your text editor.

      4. Configure DNS for your site by adding a domain zone and setting up reverse DNS on your Salt minion’s IP address.

      Set Up the Salt Master and Salt Minion

      Before you can start setting up the Salt formulas for the minion, you first need to install the Salt software on the master and minion and set up communication between them.

      1. Log into the Salt master Linode via SSH and run the Salt installation bootstrap script:

        wget -O bootstrap-salt.sh https://bootstrap.saltstack.com
        sudo sh bootstrap-salt.sh -M -N
        

        Note

        The -M option tells the script to install the Salt master software, and the -N option tells the script to not install the minion software.

      2. Log into the Salt minion Linode via SSH and set the hostname. This guide uses hugo-webserver as the example hostname:

        sudo hostnamectl set-hostname hugo-webserver
        

        Note

        This step needs to be completed before installing Salt on the minion, as Salt will use your hostname to generate the minion’s Salt ID.

      3. Edit the minion’s /etc/hosts file and append a new line for your hostname after the localhost line; replace 192.0.2.3 with your minion’s public IP address:

        /etc/hosts
        1
        2
        3
        
        127.0.0.1       localhost
        192.0.2.3       hugo-webserver
        # [...]
      4. Run the bootstrap script on the minion:

        wget -O bootstrap-salt.sh https://bootstrap.saltstack.com
        sudo sh bootstrap-salt.sh
        
      5. Edit /etc/salt/minion on the Salt minion. Uncomment the line that begins with #master: and enter your Salt master’s IP after the colon (in place of 192.0.2.2):

        /etc/salt/minion
        1
        2
        3
        
        # [...]
        master: 192.0.2.2
        # [...]

        Note

        Linode does not charge for traffic within a datacenter across private IP addresses. If your Salt master and minion are in the same datacenter, and both have a private IP addresses, you can use your Salt master’s private IP address in this step to avoid incurring data traffic charges.

      6. Restart Salt on the minion:

        sudo systemctl restart salt-minion
        

      Salt Minion Authentication

      The minion should now be able to find the master, but it has not yet been authenticated to communicate with the master. Salt uses public-private keypairs to authenticate minions to masters.

      1. On the master, list fingerprints for all the master’s local keys, accepted minion keys, and unaccepted keys:

        sudo salt-key --finger-all
        

        The output should resemble:

          
        Local Keys:
        master.pem:  fe:1f:e8:3d:26:83:1c:...
        master.pub:  2b:93:72:b3:3a:ae:cb:...
        Unaccepted Keys:
        hugo-webserver:  29:d8:f3:ed:91:9b:51:...
        
        

        Note

        The example fingerprints in this section have been truncated for brevity.

      2. Copy the fingerprint for master.pub from the output of salt-key --finger-all. On your Salt minion, open /etc/salt/minion in a text editor. Uncomment the line that begins with #master_finger: and enter the value for your master.pub after the colon in single-quotes:

        /etc/salt/minion
        1
        2
        3
        
        # [...]
        master_finger: '0f:d6:5f:5e:f3:4f:d3:...'
        # [...]
      3. Restart Salt on the minion:

        sudo systemctl restart salt-minion
        
      4. View the minion’s local key fingerprint:

        sudo salt-call key.finger --local
        
          
        local:
            29:d8:f3:ed:91:9b:51:...
        
        

        Compare the output’s listed fingerprint to the fingerprints listed by the Salt master for any Unaccepted Keys. This is the output of salt-key --finger-all run on the master in the beginning of this section.

      5. After verifying, that the minion’s fingerprint is the same as the fingerprint detected by the Salt master, run the following command on the master to accept the minion’s key:

        sudo salt-key -a hugo-webserver
        
      6. From the master, verify that the minion is running:

        sudo salt-run manage.up
        

        You can also run a Salt test ping from the master to the minion:

        sudo salt 'hugo-webserver' test.ping
        
          
        hugo-webserver:
            True
        
        

      Initialize the Salt Minion’s Formula

      The Salt minion is ready to be configured by the master. These configurations will be written in a Salt formula which will be hosted on GitHub.

      1. On your computer, create a new directory to hold your minion’s formula and change to that directory:

        mkdir hugo-webserver-salt-formula
        cd hugo-webserver-salt-formula
        
      2. Inside the formula directory, create a new hugo directory to hold your webserver’s configuration:

        mkdir hugo
        
      3. Inside the hugo directory, create a new install.sls file:

        hugo-webserver-salt-formula/hugo/install.sls
        1
        2
        3
        
        nginx_pkg:
          pkg.installed:
            - name: nginx

        Note

        Salt configurations are declared in YAML– a markup language that incorporates whitespace/indentation in its syntax. Be sure to use the same indentation as the snippets presented in this guide.

        A .sls file is a SaLt State file. Salt states describe the state a minion should be in after the state is applied to it: e.g., all the software that should be installed, all the services that should be run, and so on.

        The above snippet says that a package with name nginx (i.e. the NGINX web server) should be installed via the distribution’s package manager. Salt knows how to negotiate software installation via the built-in package manager for various distributions. Salt also knows how to install software via NPM and other package managers.

        The string nginx_pkg is the ID for the state component, pkg is the name of the Salt module used, and pkg.installed is referred to as a function declaration. The component ID is arbitrary, so you can name it however you prefer.

        Note

        If you were to name the ID to be the same as the relevant installed package, then you do not need to specify the - name option, as it will be inferred from the ID. For example, this snippet also installs NGINX:

        hugo-webserver-salt-formula/hugo/install.sls

        The same name/ID convention is true for other Salt modules.

      4. Inside the hugo directory, create a new service.sls file:

        hugo-webserver-salt-formula/hugo/service.sls
        1
        2
        3
        4
        5
        6
        
        nginx_service:
          service.running:
            - name: nginx
            - enable: True
            - require:
              - pkg: nginx_pkg

        This state says that the nginx service should be immediately run and be enabled to run at boot. For a Debian 9 system, Salt will set the appropriate systemd configurations to enable the service. Salt also supports other init systems.

        The require lines specify that this state component should not be applied until after the nginx_pkg component has been applied.

        Note

        Unless specified by a require declaration, Salt makes no guarantees about the order that different components are applied. The order that components are listed in a state file does not necessarily correspond with the order that they are applied.

      5. Inside the hugo directory, create a new init.sls file with the following contents:

        hugo-webserver-salt-formula/hugo/init.sls
        1
        2
        3
        
        include:
          - hugo.install
          - hugo.service

        Using the include declaration in this way simply concatenates the install.sls and service.sls files into a single combined state file.

        Right now, these state files only install and enable NGINX. More functionality will be enabled later in this guide.

        The install and service states will not be applied to the minion on their own–instead, only the combined init state will ever be applied. In Salt, when a file named init.sls exists inside a directory, Salt will refer to that particular state by the name of the directory it belongs to (i.e. hugo in our example).

        Note

        The organization of the state files used here is not mandated by Salt. Salt does not place restrictions on how you organize your states. This specific structure is presented as an example of a best practice.

      Push the Salt Formula to GitHub

      1. Inside your hugo-webserver-salt-formula directory on your computer, initialize a new Git repository:

        cd ~/hugo-webserver-salt-formula
        git init
        
      2. Stage the files you just created:

        git add .
        
      3. Review the staged files:

        git status
        
          
        On branch master
        No commits yet
        Changes to be committed:
          (use "git rm --cached ..." to unstage)
        
          new file:   hugo/init.sls
          new file:   hugo/install.sls
          new file:   hugo/service.sls
        
        
      4. Commit the files:

        git commit -m "Initial commit"
        
      5. Log into the GitHub website in your browser and navigate to the Create a New Repository page.

      6. Create a new public repository with the name hugo-webserver-salt-formula:

        GitHub New Repository - Add New Salt Formula Repo

      7. Copy the HTTPS URL for your new repository:

        GitHub New Repository - New Salt Formula Repo

      8. In your local Salt formula repository, add the GitHub repository as the origin remote and push your new files to it. Replace github-username with your GitHub user:

        git remote add origin https://github.com/github-username/hugo-webserver-salt-formula.git
        git push -u origin master
        

        Note

        If you haven’t pushed anything else to your GitHub account from the command line before, you may be prompted to authenticate with GitHub. If you have two-factor authentication enabled for your account, you will need to create and use a personal access token.
      9. If you navigate back to your hugo-webserver-salt-formula repository on GitHub and refresh the page, you should now see your new files.

      Enable GitFS on the Salt Master

      Update your Salt master to serve the new formula from GitHub:

      1. Salt requires that you install a Python interface to Git to use GitFS. On the Salt master Linode:

        sudo apt-get install python-git
        
      2. Open /etc/salt/master in a text editor. Uncomment the fileserver_backend declaration and enter roots and gitfs in the declaration list:

        /etc/salt/master
        1
        2
        3
        
        fileserver_backend:
          - roots
          - gitfs

        roots refers to Salt files stored on the master’s filesystem. While the Hugo webserver Salt formula is stored on GitHub, the Salt Top file will be stored on the master. The Top file is how Salt maps states to the minions they will be applied to.

      3. In the same file, uncomment the gitfs_remotes declaration and enter your Salt formula’s repository URL:

        /etc/salt/master
        1
        2
        
        gitfs_remotes:
          - https://github.com/your_github_user/hugo-webserver-salt-formula.git
      4. Uncomment the gitfs_provider declaration and set its value to gitpython:

        /etc/salt/master
        1
        
        gitfs_provider: gitpython

      Apply the Formula’s State to the Minion

      1. In /etc/salt/master, uncomment the file_roots declaration and set the following values:

        /etc/salt/master
        1
        2
        3
        
        file_roots:
          base:
            - /srv/salt/

        file_roots specifies where state files are kept on the Master’s filesystem. This is referenced when - roots is declared in the fileserver_backend section. base refers to a Salt environment, which is a tree of state files that can be applied to minions. This guide will only use the base environment, but other environments could be created for development, QA, and so on.

      2. Restart Salt on the master to enable the changes in /etc/salt/master:

        sudo systemctl restart salt-master
        
      3. Create the /srv/salt directory on the Salt master:

        sudo mkdir /srv/salt
        
      4. Create a new top.sls file in /srv/salt:

        /srv/salt/top.sls
        1
        2
        3
        
        base:
          'hugo-webserver':
            - hugo

        This is Salt’s Top file, and the snippet declares that the hugo-webserver minion should receive the init.sls state from the hugo directory (from your GitHub-hosted Salt formula).

      5. Tell Salt to apply states from the Top file to the minion:

        sudo salt 'hugo-webserver' state.apply
        

        Salt as refers to this command as a highstate. Running a highstate can take a bit of time to complete, and the output of the command will describe what actions were taken on the minion. The output will also show if any actions failed.

        Note

        If you see an error similar to:

          
        No matching sls found for 'hugo' in env 'base'
        
        

        Try running this command to manually fetch the Salt formula from GitHub, then run the state.apply command again:

        sudo salt-run fileserver.update
        

        Salt’s GitFS fetches files from remotes periodically, and this period can be configured.

      6. If you visit your domain name in a web browser, you should now see NGINX’s default test page served by the Salt minion.

      Initialize the Hugo Site

      1. On your computer, create a new Hugo site. Make sure you are not running this command in your hugo-webserver-salt-formula directory:

        hugo new site example-hugo-site
        
      2. Navigate to the new Hugo site directory and initialize a Git repository:

        cd example-hugo-site
        git init
        
      3. Install a theme into the themes/ directory. This guide uses the Cactus theme:

        git submodule add https://github.com/digitalcraftsman/hugo-cactus-theme.git themes/hugo-cactus-theme
        
      4. The theme comes with some example content. Copy it into the root of your site so that it can be viewed:

        cp -r themes/hugo-cactus-theme/exampleSite/ .
        
      5. Edit the baseurl, themesDir, and name options in config.toml as follows; replace example.com with your own domain and Your Name with your own name:

        example-hugo-site/config.toml
        1
        2
        3
        4
        5
        6
        
        # [...]
        baseURL = "http://example.com"
        # [...]
        themesDir = "themes"
        # [...]
          name = "Your Name"
      6. Run the Hugo development server on your computer:

        hugo server
        

        The output from this command will end with a line like:

          
        Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
        
        
      7. If you view the URL from this output in a browser, you can see your new Hugo site:

        New Hugo Site - Development Server

      8. Enter CTRL-C in the terminal session on your computer to stop the Hugo development server. Open the .gitignore file and make sure public/ is listed. The default .gitignore from the Cactus theme should look like:

        example-hugo-site/config.toml

        The public directory is the result of Hugo compiling the Markdown content files into HTML. These files can be regenerated by anyone who downloads your site code, so they won’t be checked into version control.

      Push the Hugo Site to GitHub

      1. In the Hugo site directory, commit the new site files:

        git add .
        git commit -m "Initial commit"
        
      2. Create a new public repository on GitHub named example-hugo-site and copy the repository’s HTTPS URL.

      3. In the site directory, add the GitHub repository as the origin remote and push your new files to it; replace github-username with your GitHub user:

        git remote add origin https://github.com/github-username/example-hugo-site.git
        git push -u origin master
        

      Deploy the Hugo Site

      The Salt minion’s formula needs to be updated in order to serve the Hugo site. Specifically, the formula will need to have states which:

      • Install Git and clone the Hugo site repository from GitHub.

      • Install Hugo and build the HTML files from the markdown content.

      • Update the NGINX configuration to serve the built site.

      Some of the new state components will refer to data stored in Salt Pillar. Pillar is a Salt system that stores private data and other parameters that you don’t want to list in your formulas. The Pillar data will be kept as a file on the Salt master and not checked into version control.

      Note

      There are methods for securely checking this data into version control or using other backends to host the data, but those strategies are outside the scope of this guide.

      Pillar data is injected into state files with Salt’s Jinja templating feature. State files are first evaluated as Jinja templates and then as YAML afterwards.

      Install Git and Hugo

      In your local Salt formula’s repository, edit the install.sls file to append the git_pkg and hugo_pkg states:

      hugo-webserver-salt-formula/hugo/install.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      
      # [...]
      
      git_pkg:
        pkg.installed:
          - name: git
      
      hugo_pkg:
        pkg.installed:
          - name: hugo
          - sources:
            - hugo: https://github.com/gohugoio/hugo/releases/download/v{{ pillar['hugo_deployment_data']['hugo_version'] }}/hugo_{{ pillar['hugo_deployment_data']['hugo_version'] }}_Linux-64bit.deb

      The first state component installs Git, and the second component installs Hugo. The second component’s sources declaration specifies that the package should be downloaded from Hugo’s GitHub repository (instead of from the distribution package manager).

      The {{ }} syntax that appears in {{ pillar['hugo_deployment_data']['hugo_version'] }} is a Jinja substitution statement. pillar['hugo_deployment_data']['hugo_version'] returns the value of the hugo_version key from a dictionary named hugo_deployment_data in Pillar. Keeping the Hugo version in Pillar lets you update Hugo without needing to update your formulas.

      Clone the Hugo Site Git Repository

      Create a new config.sls file in your local Salt formula repository’s hugo directory:

      hugo-webserver-salt-formula/hugo/config.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      
      hugo_group:
        group.present:
          - name: {{ pillar['hugo_deployment_data']['group'] }}
      
      hugo_user:
        user.present:
          - name: {{ pillar['hugo_deployment_data']['user'] }}
          - gid: {{ pillar['hugo_deployment_data']['group'] }}
          - home: {{ pillar['hugo_deployment_data']['home_dir'] }}
          - createhome: True
          - require:
            - group: hugo_group
      
      hugo_site_repo:
        cmd.run:
          - name: git clone --recurse-submodules https://github.com/{{ pillar['hugo_deployment_data']['github_account'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}.git
          - cwd: {{ pillar['hugo_deployment_data']['home_dir'] }}
          - runas: {{ pillar['hugo_deployment_data']['user'] }}
          - creates: {{ pillar['hugo_deployment_data']['home_dir'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}
          - require:
            - pkg: git_pkg
            - user: hugo_user

      The final hugo_site_repo component in this snippet is responsible for cloning the example Hugo site repository from GitHub. This cloned repo is placed in the home directory of a system user that Salt creates in the preceding components. The clone command also recursively downloads the Cactus theme submodule.

      Note

      The - creates declaration tells Salt that running the cmd command module will result in the creation of the file that’s specified. If the state is applied again later, Salt will check if that file already exists. If it exists, Salt will not run the module again.

      The require declarations in each component ensure that:

      • The clone is not run until the system user and home directory have been created, and until the software package for Git has been installed.
      • The user is not created until the group it belongs to is created.

      Instead of hard-coding the parameters for the user, group, home directory, GitHub account, and repository name, these are retrieved from Pillar.

      Configure NGINX

      1. Append the following states to your config.sls:

        hugo-webserver-salt-formula/hugo/config.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        
        nginx_default:
          file.absent:
            - name: '/etc/nginx/sites-enabled/default'
            - require:
              - pkg: nginx_pkg
        
        nginx_config:
          file.managed:
            - name: /etc/nginx/sites-available/hugo_site
            - source: salt://hugo/files/hugo_site
            - user: root
            - group: root
            - mode: 0644
            - template: jinja
            - require:
              - pkg: nginx_pkg
        
        nginx_symlink:
          file.symlink:
            - name: /etc/nginx/sites-enabled/hugo_site
            - target: /etc/nginx/sites-available/hugo_site
            - user: root
            - group: root
            - require:
              - file: nginx_config
        
        nginx_document_root:
          file.directory:
            - name: {{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}
            - user: {{ pillar['hugo_deployment_data']['user'] }}
            - group: {{ pillar['hugo_deployment_data']['group'] }}
            - dir_mode: 0755
            - require:
              - user: hugo_user
        • The nginx_default component removes the symlink in sites-enabled for the default NGINX config, which disables that configuration.
        • nginx_config and nginx_symlink then create a new configuration file in sites-available and a symlink to it in sites-enabled.
        • The nginx_document_root component creates the directory that NGINX will serve your Hugo site files from (when filled in with Pillar data, this will directory will look like /var/www/example-hugo-site).
      2. The - source: salt://hugo/files/hugo_site declaration in nginx_config refers to an NGINX configuration file that doesn’t exist in your repository yet. Create the files/ directory:

        cd ~/hugo-webserver-salt-formula/hugo
        mkdir files
        
      3. Create the hugo_site file inside files/:

        hugo-webserver-salt-formula/hugo/files/hugo_site
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        
        server {
            listen 80;
            listen [::]:80;
            server_name {{ pillar['hugo_deployment_data']['domain_name'] }};
        
            root {{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }};
        
            index index.html index.htm index.nginx-debian.html;
        
            location / {
                try_files $uri $uri/ = /404.html;
            }
        }

        The nginx_config component that manages this file also listed the - template: jinja declaration, so the source file is interpreted as a Jinja template. The source file is able to substitute values from Pillar using the Jinja substitution syntax.

      4. Replace the content of your service.sls with this snippet:

        hugo-webserver-salt-formula/hugo/service.sls
        1
        2
        3
        4
        5
        6
        7
        8
        
        nginx_service:
          service.running:
            - name: nginx
            - enable: True
            - require:
              - file: nginx_symlink
            - watch:
              - file: nginx_config

        The nginx_service component now requires nginx_symlink instead of nginx_pkg. Without this change, the service may be enabled and run before the new NGINX configuration is set up. The - watch declaration also instructs NGINX to restart whenever a change to nginx_config is made.

      Build Hugo

      1. Append a build_script state to config.sls:

        hugo-webserver-salt-formula/hugo/config.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        
        build_script:
          file.managed:
            - name: {{ pillar['hugo_deployment_data']['home_dir'] }}/deploy.sh
            - source: salt://hugo/files/deploy.sh
            - user: {{ pillar['hugo_deployment_data']['user'] }}
            - group: {{ pillar['hugo_deployment_data']['group'] }}
            - mode: 0755
            - template: jinja
            - require:
              - user: hugo_user
          cmd.run:
            - name: ./deploy.sh
            - cwd: {{ pillar['hugo_deployment_data']['home_dir'] }}
            - runas: {{ pillar['hugo_deployment_data']['user'] }}
            - creates: {{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}/index.html
            - require:
              - file: build_script
              - cmd: hugo_site_repo
              - file: nginx_document_root

        This state uses more than one module. The first module will download the deploy.sh file from the salt master and place it on the minion. This script will be responsible for compiling your Hugo site files. The second module then calls that script. The first module is listed as a requirement of the second module, along with the Git clone command, and the creation of the document root folder.

        Note

        The - creates option in the second module ensures that Salt doesn’t rebuild Hugo if the state is re-applied to the minion.

      2. Create the deploy.sh script in files/:

        hugo-webserver-salt-formula/hugo/files/deploy.sh
        1
        2
        3
        4
        
        #!/bin/bash
        
        cd {{ pillar['hugo_deployment_data']['site_repo_name'] }}
        hugo --destination={{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}

        Hugo’s build function is called with NGINX’s document root as the destination for the built files.

      3. Update init.sls to include the new config.sls file:

        hugo-webserver-salt-formula/hugo/init.sls
        1
        2
        3
        4
        
        include:
          - hugo.install
          - hugo.config
          - hugo.service

      Push the Salt Formula Updates to GitHub

      Your state files should now have these contents: init.sls, install.sls, config.sls, service.sls.

      The files present in your Salt formula repository should be:

        
      hugo
      ├── config.sls
      ├── files
      │   ├── deploy.sh
      │   └── hugo_site
      ├── init.sls
      ├── install.sls
      └── service.sls
      
      
      1. Stage all the changes you made to your local Salt formula files in the previous steps and then commit the changes:

        cd ~/hugo-webserver-salt-formula
        git add .
        git commit -m "Deploy the Hugo site"
        
      2. Push the commit to your GitHub repository:

        git push origin master
        

      Create the Salt Pillar File

      1. Open /etc/salt/master on the Salt master in a text editor. Uncomment the pillar_roots section:

        /etc/salt/master
        1
        2
        3
        
        pillar_roots:
          base:
            - /srv/pillar

        pillar_roots performs an analogous function to file_roots: it specifies where Pillar data is stored on the master’s filesystem.

      2. Restart Salt on the master to enable the changes in /etc/salt/master:

        sudo systemctl restart salt-master
        
      3. Create the /srv/pillar directory on the Salt master:

        sudo mkdir /srv/pillar
        
      4. Create an example-hugo-site.sls file in /srv/pillar to contain the Pillar data for the minion. This file uses the same YAML syntax as other state files. Replace the values for github_account and domain_name with your GitHub account and your site’s domain name:

        /srv/pillar/example-hugo-site.sls
        1
        2
        3
        4
        5
        6
        7
        8
        9
        
        hugo_deployment_data:
          hugo_version: 0.49
          group: hugo
          user: hugo
          home_dir: /home/hugo
          github_account: your_github_user
          site_repo_name: example-hugo-site
          nginx_document_root: /var/www
          domain_name: yourdomain.com
      5. Create a top.sls file in /srv/pillar. Similar to the Top file in your state tree, the Pillar’s Top file maps Pillar data to minions:

        /srv/pillar/top.sls
        1
        2
        3
        
        base:
          'hugo-webserver':
            - example-hugo-site

      Apply State Updates to the Minion

      On the Salt master, apply the new states to all minions:

      sudo salt '*' state.apply
      

      Note

      In this guide there is only one minion, but Salt can use shell-style globbing and regular expressions to match against minion IDs when you have more than one. For example, this command would run a highstate on all minions whose IDs begin with hugo:

      sudo salt 'hugo*' state.apply
      

      If no changes are made, try manually fetching the Salt formula updates from GitHub and then run the state.apply command again:

      sudo salt-run fileserver.update
      

      When the operation finishes, your Hugo site should now be visible at your domain.

      Deploy Site Updates with Webhooks

      Your site is now deployed to production, but there is no automatic mechanism in place yet for updating the production server when you update your Hugo site’s content. To update the production server, your minion will need to:

      1. Pull the latest changes pushed to the master branch of your Hugo site repository on GitHub.

      2. Run the Hugo build process with the new content.

      The deploy.sh script can be altered to pull changes from GitHub. These script changes will be made in the Salt formula repository. Then, we’ll set up webhooks to notify the Salt minion that updates have been made to the Hugo site.

      Webhooks are HTTP POST requests specifically designed and sent by systems to communicate some kind of significant event. A webhook server listens for these requests and then takes some action when it receives one. For example, a GitHub repository can be configured to send webhook notifications whenever a push is made to the repository. This is the kind of notification we’ll configure, and the Salt minion will run a webhook server to receive them. Other event notifications can also be set up on GitHub.

      Set Up a Webhook Server on the Salt Minion

      1. In your local Salt formula repository, append a new webhook_pkg state to your install.sls that installs the webhook server package by adnanh:

        hugo-webserver-salt-formula/hugo/install.sls
        1
        2
        3
        
        webhook_pkg:
          pkg.installed:
            - name: webhook

        Note

        The webhook server written in Go by adnanh is a popular implementation of the concept, but it’s possible to write other HTTP servers that parse webhook payloads.

      2. Append two new components to your config.sls:

        hugo-webserver-salt-formula/hugo/config.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        
        webhook_systemd_unit:
          file.managed:
            - name: '/etc/systemd/system/webhook.service'
            - source: salt://hugo/files/webhook.service
            - user: root
            - group: root
            - mode: 0644
            - template: jinja
            - require:
              - pkg: webhook_pkg
          module.run:
            - name: service.systemctl_reload
            - onchanges:
              - file: webhook_systemd_unit
        
        webhook_config:
          file.managed:
            - name: '/etc/webhook.conf'
            - source: salt://hugo/files/webhook.conf
            - user: root
            - group: {{ pillar['hugo_deployment_data']['group'] }}
            - mode: 0640
            - template: jinja
            - require:
              - pkg: webhook_pkg
              - group: hugo_group

        The first state creates a systemd unit file for the webhook service. The second state creates a webhook configuration. The webhook server reads the configuration and generates a webhook URL from it.

      3. Create a webhook.service file in your repository’s files/ directory:

        hugo-webserver-salt-formula/hugo/files/webhook.service
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        
        [Unit]
        Description=Small server for creating HTTP endpoints (hooks)
        Documentation=https://github.com/adnanh/webhook/
        
        [Service]
        User={{ pillar['hugo_deployment_data']['user'] }}
        ExecStart=/usr/bin/webhook -nopanic -hooks /etc/webhook.conf
        
        [Install]
        WantedBy=multi-user.target
      4. Create a webhook.conf file in your repository’s files/ directory:

        hugo-webserver-salt-formula/hugo/files/webhook.conf
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        
        [
          {
            "id": "github_push",
            "execute-command": "{{ pillar['hugo_deployment_data']['home_dir'] }}/deploy.sh",
            "command-working-directory": "{{ pillar['hugo_deployment_data']['home_dir'] }}",
            "trigger-rule":
            {
              "and":
              [
                {
                  "match":
                  {
                    "type": "payload-hash-sha1",
                    "secret": "{{ pillar['hugo_deployment_data']['webhook_secret'] }}",
                    "parameter":
                    {
                      "source": "header",
                      "name": "X-Hub-Signature"
                    }
                  }
                },
                {
                  "match":
                  {
                    "type": "value",
                    "value": "refs/heads/master",
                    "parameter":
                    {
                      "source": "payload",
                      "name": "ref"
                    }
                  }
                }
              ]
            }
          }
        ]

        This configuration sets up a URL named http://example.com:9000/hooks/github_push, where the last component of the URL is derived from the value of the configuration’s id.

        Note

        The webhook server runs on port 9000 and places your webhooks inside a hooks/ directory by default.

        When a POST request is sent to the URL:

        • The webhook server checks if the header and payload data from the request satisfies the rules in the trigger-rule dictionary, which are:

          • That the SHA1 hash of the server’s webhook secret matches the secret in the request headers. This prevents people who don’t know your webhook secret from triggering the webhook’s action.
          • The ref parameter in the payload matches refs/heads/master. This ensures that only pushes to the master branch trigger the action.
        • If the rules are satisfied, then the command listed in execute-command is run, which is the deploy.sh script.

        Note

        Further documentation on the webhook configuration options can be reviewed on the project’s GitHub repository.
      5. Append a new webhook_service state to your service.sls that enables and starts the webhook server:

        hugo-webserver-salt-formula/hugo/service.sls
        1
        2
        3
        4
        5
        6
        7
        
        webhook_service:
          service.running:
            - name: webhook
            - enable: True
            - watch:
              - file: webhook_config
              - module: webhook_systemd_unit
      6. Update the deploy.sh script so that it pulls changes from master before building the site:

        hugo-webserver-salt-formula/hugo/files/deploy.sh
        1
        2
        3
        4
        5
        
        #!/bin/bash
        
        cd {{ pillar['hugo_deployment_data']['site_repo_name'] }}
        git pull origin master
        hugo --destination={{ pillar['hugo_deployment_data']['nginx_document_root'] }}//{{ pillar['hugo_deployment_data']['site_repo_name'] }}
      7. Your state files should now have these contents: init.sls (unchanged), install.sls, config.sls, service.sls. Save the changes made to your Salt files, then commit and push them to GitHub:

        cd ~/hugo-webserver-salt-formula
        git add .
        git commit -m "Webhook server states"
        git push origin master
        
      8. On the Salt master, add a webhook_secret to the example-hugo-site.sls Pillar. Your secret should be a complex, random alphanumeric string.

        /srv/pillar/example-hugo-site.sls
        1
        2
        3
        
        hugo_deployment_data:
          # [...]
          webhook_secret: your_webhook_secret
      9. From the Salt master, apply the formula updates to the minion:

        sudo salt-run fileserver.update
        sudo salt 'hugo-webserver' state.apply
        
      10. Your webhook server should now be running on the minion. If you run a curl against it, you should see:

        curl http://example.com:9000/hooks/github_push
        
          
        Hook rules were not satisfied.⏎
        
        

      Configure a Webhook on GitHub

      1. Visit your example Hugo site repository on GitHub and navigate to the Webhooks section of the Settings tab. Click on the Add webhook button:

        GitHub - Add Webhook Button

      2. Fill in the form:

        • Enter http://example.com:9000/hooks/github_push for the payload URL (substitute example.com for your own domain).

        • Select application/json for the content type.

        • Paste in the webhook secret that you previously added to Salt Pillar.

        The webhook is configured to notify on push events by default. Keep this option selected.

        GitHub - New Webhook Configuration

      3. Click the green Add webhook button to complete the setup.

      Update the Hugo Site

      1. In your local Hugo site repository, create a new post using Hugo’s archetypes feature:

        hugo new post/test-post.md
        
      2. This command creates a new partially filled in markdown document in content/post/. Open this file in your editor, remove the draft: true line from the frontmatter, and add some body text:

        example-hugo-site/content/post/test-post.md
        1
        2
        3
        4
        5
        6
        
        ---
        title: "Test Post"
        date: 2018-10-19T11:39:15-04:00
        ---
        
        Test post body text
      3. If you run hugo server in the repository directory, you can see the new post:

        Hugo Home Page - Test Post

      4. Commit and push the new post to GitHub:

        cd ~/example-hugo-site
        git add .
        git commit -m "Test post"
        git push origin master
        
      5. Visit your domain in your browser; your test post should automatically appear.

        Note

        If your post does not appear, review the Recent Deliveries section at the bottom of your webhook configuration page on GitHub:

        GitHub Webhook - Recent Deliveries

        If you click on a delivery, full information about the request headers and payload and the server response are shown, and these may provide some troubleshooting information. Editing the webhook.service file so that it starts the service in verbose mode may help.

      Next Steps

      The current Salt configuration can be used as a foundation for more complex deployments:

      • Host multiple Hugo sites by updating Pillar with further GitHub repositories.

      • Host different kinds of static sites by changing the Salt formula to support them.

      • Load balance your site by creating more minions and apply the same Pillar data and Salt states to them. Then, set up a NodeBalancer to direct traffic to the minions.

      • Set up a separate development branch and development server with Salt’s environments feature.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Join our Community

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      A Beginner's Guide to Salt


      Updated by Linode Written by Linode

      Use promo code DOCS10 for $10 credit on a new account.

      Salt (also referred to as SaltStack) is a Python-based configuration management and orchestration system. Salt uses a master/client model in which a dedicated Salt master server manages one or more Salt minion servers. Two of Salt’s primary jobs are:

      This guide will introduce the core concepts that Salt employs to fulfill these jobs.

      Masters and Minions

      The Salt master is a server that acts as a command-and-control center for its minions, and it is where Salt’s remote execution commands are run from. For example, this command reports the current disk usage for each of the minions that the master controls:

      salt '*' disk.usage
      

      Many other commands are available. This installs NGINX on the minion named webserver1:

      salt 'webserver1' pkg.install nginx
      

      Salt minions are your servers that actually run your applications and services. Each minion has an ID assigned to it (which can be automatically generated from the minion’s hostname), and the Salt master can refer to this ID to target commands to specific minions.

      Note

      When using Salt, you should configure and manage your minion servers from the master as much as possible, instead of logging into them directly via SSH or another protocol.

      To enable all of these functions, the Salt master server runs a daemon named salt-master, and the Salt minion servers run a daemon named salt-minion.

      Authentication

      Communication between the master and minions is performed over the ZeroMQ transport protocol, and all communication is encrypted with public/private keypairs. A keypair is generated by a minion when Salt is first installed on it, after which the minion will send its public key to the master. You will need to accept the minion’s key from the master; communication can then proceed between the two.

      Remote Execution

      Salt offers a very wide array of remote execution modules. An execution module is a collection of related functions that you can run on your minions from the master. For example:

      salt 'webserver1' npm.install gulp
      

      In this command npm is the module and install is the function. This command installs the Gulp Node.js package via the Node Package Manager (NPM). Other functions in the npm module handle uninstalling NPM packages, listing installed NPM packages, and related tasks.

      The execution modules that Salt makes available represent system administration tasks that you would otherwise perform in a shell, including but not limited to:

      Note

      cmd.run

      The cmd.run function is used to run arbitrary commands on your minions from the master:

      salt '*' cmd.run 'ls -l /etc'
      

      This would return the contents of /etc on each minion.

      Note

      Where possible, it’s better to use execution modules than to “shell out” with cmd.run.

      States, Formulas, and the Top File

      The previous section described how to use remote execution to perform specific actions on a minion. With remote execution, you could administer a minion by entering a series of such commands.

      Salt offers another way to configure a minion in which you declare the state that a minion should be in. This kind of configuration is called a Salt state, and the methodology is referred to generally as configuration management.

      The distinction between the two styles is subtle; to illustrate, here’s how installing NGINX is interpreted in each methodology:

      Salt states are defined in state files. Once you have recorded your states, you then apply them to a minion. Salt analyzes the state file and determines what it needs to do to make sure that the minion satisfies the state’s declarations.

      Note

      This sometimes results in the same command that would be run via remote execution, but sometimes it doesn’t. In the NGINX example, if Salt sees that NGINX was already installed previously, it won’t invoke the package manager again when the state is applied.

      Anatomy of a State

      Here’s an example state file which ensures that: rsync and curl are installed; NGINX is installed; and NGINX is run and enabled to run at boot:

      /srv/salt/webserver_setup.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      
      network_utilities:
        pkg.installed:
          - pkgs:
            - rsync
            - curl
      
      nginx_pkg:
        pkg.installed:
          - name: nginx
      
      nginx_service:
        service.running:
          - name: nginx
          - enable: True
          - require:
            - pkg: nginx_pkg

      State files end with the extension .sls (SaLt State). State files can have one or more state declarations, which are the top-level sections of the file (network_utilities, nginx_pkg, and nginx_service in the above example). State declarations IDs are arbitrary, so you can name them however you prefer.

      Note

      If you were to name the ID to be the same as the relevant installed package, then you do not need to specify the - name option, as it will be inferred from the ID. For example, this snippet also installs NGINX:

      The same name/ID inference convention is true for other Salt modules.

      State declarations contain state modules. State modules are distinct from execution modules but often perform similar jobs. For example, a pkg state module exists with functions analogous to the pkg execution module, as with the pkg.installed state function and the pkg.install execution function. As with execution modules, Salt provides a wide array of state modules for you to use.

      Note

      State declarations are not necessarily applied in the order they appear in a state file, but you can specify that a declaration depends on another one using the require option. This is the case in the above example; Salt will not attempt to run and enable NGINX until it is installed.

      State files are really just collections of dictionaries, lists, strings, and numbers that are then interpreted by Salt. By default, Salt uses the YAML syntax for representing states.

      State files are often kept on the Salt master’s filesystem, but they can also be stored in other fileserver locations, like a Git repository (in particular, GitHub).

      Applying a State to a Minion

      To apply a state to a minion, use the state.apply function from the master:

      salt `webserver1` state.apply webserver_setup
      

      This command applies the example webserver_setup.sls state to a minion named webserver1. When applying the state, the .sls suffix is not mentioned. All of the state declarations in the state file are applied.

      Salt Formulas

      Formulas are just collections of states that together configure an application or system component on a minion. Formulas are usually organized across several different .sls files. Splitting a formula’s states up across different files can make it easier to organize your work. State declarations can include and reference declarations across other files.

      Formulas that are sufficiently generic are often shared on GitHub to be used by others. The SaltStack organization maintains a collection of popular formulas. Salt’s documentation has a guide on using a formula hosted on GitHub.

      The definition of what constitutes a formula is somewhat loose, and the specific structure of a formula is not mandated by Salt.

      The Top File

      In addition to manually applying states to minions, Salt provides a way for you to automatically map which states should be applied to different minions. This map is called the top file.

      Here’s a simple top file:

      /srv/salt/top.sls
      1
      2
      3
      4
      5
      6
      
      base:
        '*':
          - universal_setup
      
        'webserver1':
          - webserver_setup

      base refers to the Salt environment. You can specify more than one environment corresponding to different phases of your work; for example: development, QA, production, etc. base is the default.

      Groups of minions are specified under the environment, and states are listed for each set of minions. The above example top file says that a universal_setup state should be applied to all minions ('*'), and the webserver_setup state should be applied to the webserver1 minion.

      If you run the state.apply function with no arguments, then Salt will inspect the top file and apply all states within it according to the mapping you’ve created:

      salt '*' state.apply
      

      Note

      This action is colloquially known as a highstate.

      Benefits of States and Configuration Management

      Defining your configurations in states eases system administration:

      • Setting up states minimizes human error, as you will not need to enter commands manually one-by-one.

      • Applying a state to minion multiple times generally does not result in any changes beyond the first application. Salt understands when a state has already been implemented on a minion and will not perform unnecessary actions.

      • If you update a state file and apply it to a minion, Salt will detect and only apply the changes, which makes updating your systems more efficient.

      • A state can be reused and applied to more than one minion, which will result in identical configurations across different servers.

      • State files can be entered into a version control system, which helps you track changes to your systems over time.

      Targeting Minions

      You can match against your minions’ IDs using shell style globbing. This works at either the command line or in the top file.

      These examples would apply the webserver_setup state to all minions whose ID begins with webserver (e.g. webserver1, webserver2, etc):

      Regular Expressions and lists can also be used to match against minion IDs.

      Grains

      Salt’s grains system provides access to information that is generated by and stored on a minion. Examples include a minion’s operating system, domain name, IP address, and so on. You can also specify custom grain data on a minion, as outlined in Salt’s documentation.

      You can use grain data to target minions from the command line. This command installs httpd on all minions running CentOS:

      salt -G 'os:CentOS' pkg.install httpd
      

      You can also use grains in a top file:

      1
      2
      3
      4
      
      base:
        'os:CentOS':
          - match: grain
          - centos_setup

      Grain information generally isn’t very dynamic, but it can change occasionally, and Salt will refresh its grain data when it does. To view your minions’ grain data:

      salt '*' grains.items
      

      Storing Data and Secrets in Pillar

      Salt’s pillar feature takes data defined on the Salt master and distributes it to minions. A primary use for pillar is to store secrets, such as account credentials. Pillar is also a useful place to store non-secret data that you wouldn’t want to record directly in your state files.

      Note

      Let’s say that you want to create system users on a minion and assign different shells to each of them. If you were to code this information into a state file, you would need a new declaration for each user. If you store the data in pillar instead, you can then just create one state declaration and inject the pillar data into it using Salt’s Jinja templating feature.

      Note

      Salt Pillar is sometimes confused with Salt Grains, as they both keep data that is used in states and remote execution. The data that grains maintains originates from the minions, while the data in pillar originates on the master (or another backend) and is delivered to the minions.

      Anatomy of Pillar Data

      Pillar data is kept in .sls files which are written in the same YAML syntax as states:

      /srv/pillar/user_info.sls
      1
      2
      3
      4
      5
      6
      7
      
      users:
        joe:
          shell: /bin/zsh
        amy:
          shell: /bin/bash
        sam
          shell: /bin/fish

      As with state files, a top file (separate from your states’ top file) maps pillar data to minions:

      /srv/pillar/top.sls
      1
      2
      3
      
      base:
        'webserver1':
          - user_info

      Jinja Templates

      To inject pillar data into your states, use Jinja’s template syntax. While Salt uses the YAML syntax for state and pillar files, the files are first interpreted as Jinja templates (by default).

      This example state file uses the pillar data from the previous section to create system users and set the shell for each:

      /srv/salt/user_setup.sls
      1
      2
      3
      4
      5
      
      {% for user_name, user_info in pillar['users'].iteritems() %}
      {{ user_name }}:
        user.present:
          - shell: {{ user_info['shell'] }}
      {% endfor %}

      Salt will compile the state file into something that looks like this before it is applied to the minion:

       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      
      joe:
        user.present:
          - shell: /bin/zsh
      
      amy:
        user.present:
          - shell: /bin/bash
      
      sam:
        user.present:
          - shell: /bin/fish

      You can also use Jinja to interact with grain data in your states. This example state will install Apache and adjust the name for the package according to the operating system:

      /srv/salt/webserver_setup.sls
      1
      2
      3
      4
      5
      6
      7
      
      install_apache:
        pkg.installed:
          {% if grains['os'] == 'CentOS' %}
          - name: httpd
          {% else %}
          - name: apache
          {% endif %}

      Note

      Beacons

      The beacon system is a way of monitoring a variety of system processes on Salt minions. There are a number of beacon modules available.

      Beacons can trigger reactors which can then help implement a change or troubleshoot an issue. For example, if a service’s response times out, the reactor system can restart the service.

      Getting Started with Salt

      Now that you’re familiar with some of Salt’s basic terminology and components, move on to our guide Getting Started with Salt – Basic Installation and Setup to set up a configuration to start running commands and provisioning minion servers.

      The SaltStack documentation also contains a page of best practices to be mindful of when working with Salt. You should review this page and implement those practices into your own workflow whenever possible.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Join our Community

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link