One place for hosting & domains

      Monitoring Salt Minions with Beacons


      Updated by Linode Written by Linode

      Use promo code DOCS10 for $10 credit on a new account.

      Every action performed by Salt, such as applying a highstate or restarting a minion, generates an event. Beacons emit events for non-salt processes, such as system state changes or file changes. This guide will use Salt beacons to notify the Salt master of changes to minions, and Salt reactors to react to those changes.

      Before You Begin

      If you don’t already have a Salt master and minion, follow the first steps in our Getting Started with Salt – Basic Installation and Setup guide.

      Note

      The steps in this guide require root privileges. Be sure to run the steps below as root or with the sudo prefix. For more information on privileges, see our Users and Groups guide.

      Example 1: Preventing Configuration Drift

      Configuration drift occurs when there are untracked changes to a system configuration file. Salt can help prevent configuration drift by ensuring that a file is immediately reverted to a safe state upon change. In order to do this, we first have to let Salt manage the file. This section will use an NGINX configuration file as an example, but you can choose any file.

      Manage Your File

      1. On your Salt master, create a directory for your managed files in /srv/salt/files:

        mkdir /srv/salt/files
        
      2. On your Salt master, place your nginx.conf, or whichever file you would like to manage, in the /srv/salt/files folder.

      3. On your Salt master, create a state file to manage the NGINX configuration file:

        /srv/salt/nginx_conf.sls
        1
        2
        3
        4
        5
        
        /etc/nginx/nginx.conf:
          file.managed:
            - source:
              - salt://files/nginx.conf
            - makedirs: True

        There are two file paths in this .sls file. The first file path is the path to your managed file on your minion. The second, under source and prefixed with salt://, points to the file path on your master. salt:// is a convenience file path that maps to /srv/salt.

      4. On your Salt master, create a top file if it does not already exist and add your nginx_conf.sls:

        /srv/salt/top.sls
        1
        2
        3
        
        base:
          '*':
            - nginx_conf
      5. Apply a highstate from your Salt master to run the nginx_conf.sls state on your minions.

        salt '*' state.apply
        

      Create a Beacon

      1. In order to be notified when a file changes, you will need the Python pyinotify package. Create a Salt state that will handle installing the pyinotify package on your minions:

        /srv/salt/packages.sls
        1
        2
        3
        4
        5
        6
        7
        8
        
        python-pip:
          pkg.installed
        
        pyinotify:
          pip.installed:
            - require:
              - pkg: python-pip
                

        Note

        The inotify beacon only works on OSes that have inotify kernel support. Currently this excludes FreeBSD, macOS, and Windows.

      2. On the Salt master create a minion.d directory to store the beacon configuration file:

        mkdir /srv/salt/files/minion.d
        
      3. Now create a beacon that will emit an event every time the nginx.conf file changes on your minion. Create the /etc/salt/minion.d/beacons.conf file and add the following lines:

        /etc/salt/minion.d/beacons.conf
        1
        2
        3
        4
        5
        6
        7
        
        beacons:
          inotify:
            - files:
                /etc/nginx/nginx.conf:
                  mask:
                    - modify
            - disable_during_state_run: True
      4. To apply this beacon to your minions, create a new file.managed Salt state:

        /srv/salt/beacons.sls
        1
        2
        3
        4
        5
        6
        
        /etc/salt/minion.d/beacons.conf:
          file.managed:
            - source:
              - salt://files/minion.d/beacons.conf
            - makedirs: True
            
      5. Add the new packages and beacons states to your Salt master’s top file:

        /srv/salt/top.sls
        1
        2
        3
        4
        5
        
        base:
          '*':
            - nginx_conf
            - packages
            - beacons
      6. Apply a highstate from your Salt master to implement these changes on your minions:

        salt '*' state.apply
        
      7. Open another shell to your Salt master and start the Salt event runner. You will use this to monitor for file change events from your beacon.

        salt-run state.event pretty=True
        
      8. On your Salt minion, make a change to your nginx.conf file, and then check out your Salt event runner shell. You should see an event like the following:

          
        salt/beacon/salt-minion/inotify//etc/nginx/nginx.conf	{
            "_stamp": "2018-10-10T13:53:47.163499",
            "change": "IN_MODIFY",
            "id": "salt-minion",
            "path": "/etc/nginx/nginx.conf"
        }
        
        

        Note that the first line is the name of the event, and it includes your Salt minion name and the path to your managed file. We will use this event name in the next section.

      9. To revert the nginx.conf file to it’s initial state, you can apply a highstate from your Salt master.

        salt '*' state.apply nginx_conf
        

        Open your managed file on your Salt minion and notice that the change has been reverted. We will automate this last step in the next section.

      Create a Reactor

      1. On your Salt master, create the /srv/reactor directory:

        mkdir /srv/reactor
        
      2. Then create a reactor state file in the /srv/reactor directory and include the following:

        /srv/reactor/nginx_conf_reactor.sls
        1
        2
        3
        4
        5
        
        /etc/nginx/nginx.conf:
          local.state.apply:
            - tgt: {{ data['id'] }}
            - arg:
              - nginx_conf

        The file path in the first line is simply the name of the reactor, and can be whatever you choose. The tgt, or target, is the Salt minion that will receive the highstate. In this case, the information passed to the reactor from the beacon event is used to programmatically choose the right Salt minion ID. This information is available as the data dictionary. The arg, or argument, is the name of the Salt state file that was created to manage the nginx.conf file.

      3. On your Salt master, create a reactor.conf file and include the new reactor state file:

        /etc/salt/master.d/reactor.conf
        1
        2
        3
        
        reactor:
          - 'salt/beacon/*/inotify//etc/nginx/nginx.conf':
            - /srv/reactor/nginx_conf_reactor.sls

        This reactor.conf file is essentially a list of event names matched to reactor state files. In this example we’ve used a glob (*) in the event name instead of specifying a specific minion ID, (which means that any change to a nginx.confon any minion will trigger the reactor), but you might find a specific minion ID better suits your needs.

      4. Restart the salt-master service to apply the reactor.conf file:

        systemctl restart salt-master
        
      5. On your Salt minion, make a change to the nginx.conf file. Then check out your event runner shell and you should see a number of events. Then, check your nginx.conf file. The changes you made should have automatically been reverted.

      Congratulations, you now know how to manage configuration drift with Salt. All future updates to nginx.conf should be made on the Salt master and applied using state.apply.

      Example 2: Monitoring Minion Memory Usage with Slack

      Salt comes with a number of system monitoring beacons. In this example we will monitor a minion’s memory usage and send a Slack notification when the memory usage has passed a certain threshold. For this section you will need to create a Slack bot, obtain an OAuth token, and configure the bot to be able to send Slack messages on your behalf.

      Configure Your Slack App

      1. Create a Slack app.

      2. From the Slack app settings page, navigate to OAuth & Permissions.

      3. Copy down the OAuth Access Token.

      4. Under Scopes, select Send Messages As < your app name >.

      Create a Beacon

      1. On your Salt master, open or create the /srv/salt/files/minion.d/beacons.conf file and add the following lines. If you already have a beacons.conf file from the previous example, leave out the beacons: line, but ensure that rest of the configuration is indented two spaces:

        /srv/salt/files/minion.d/beacons.conf
        1
        2
        3
        4
        5
        
        beacons:
          memusage:
            beacon.present:
              - percent: 15%
              - interval: 15

        In this example we’ve left the memory usage percentage low to ensure the beacon event will fire, and the event interval set to 15 seconds. In a production environment you should change these to more sane values.

      2. Apply a highstate from your Salt master to add the beacon to your minions:

        salt '*' state.apply
        
      3. If you haven’t already, open another shell into your Salt master and start the event runner:

        salt-run state.event pretty=True
        
      4. After a few seconds, assuming you’ve set the memory percentage low enough, you should see an event like the following:

          
        salt/beacon/salt-minion/memusage/	{
            "_stamp": "2018-10-10T15:48:53.165368",
            "id": "salt-minion",
            "memusage": 20.7
        }
        
        

        Note that the first line is the name of the event, and contains the minion name. We will use this event name in the next section.

      Create a Reactor

      1. On your Salt master, create the /srv/reactor directory if you have not already done so:

        mkdir /srv/reactor
        
      2. Then create a reactor state file and add the following lines, making sure to change the channel, api_key, and from_name keys to reflect your desired values. The api_key is the OAuth token you copied down in step 3 of the Configure Your Slack App section:

        /srv/reactor/memusage.sls
        1
        2
        3
        4
        5
        6
        7
        8
        
        Send memusage to Slack:
          local.slack.post_message:
            - tgt: {{ data['id'] }}
            - kwarg:
                channel: "#general"
                api_key: "xoxp-451607817121-453578458246..."
                message: "{{ data['id'] }} has hit a memory usage threshold: {{ data['memusage'] }}%."
                from_name: "Memusage Bot"

        We’re using the data dictionary provided to the reactor from the memusage event to populate the minion ID and the memory usage.

      3. Open or create the reactor.conf file. If you already have a reactor.conf file from the previous example, leave out the reactor: line, but ensure that rest of the configuration is indented two spaces:

        /etc/salt/master.d/reactor.conf
        1
        2
        3
        
        reactor:
          - 'salt/beacon/*/memusage/':
            - '/srv/reactor/memusage.sls'

        In this example we’ve used a glob (*) in the event name instead of specifying a specific minion ID, (which means that any memusage event will trigger the reactor), but you might find a specific minion ID better suits your needs.

      4. Restart salt-master to apply the reactor.conf:

        systemctl restart salt-master
        
      5. In your event-runner shell, after a few seconds, you should see an event like the following:

          
        salt/job/20181010161053393111/ret/salt-minion	{
            "_stamp": "2018-10-10T16:10:53.571956",
            "cmd": "_return",
            "fun": "slack.post_message",
            "fun_args": [
                {
                    "api_key": "xoxp-451607817121-453578458246-452348335312-2328ce145e5c0c724c3a8bc2afafee17",
                    "channel": "#general",
                    "from_name": "Memusage Bot",
                    "message": "salt-minion has hit a memory usage threshold: 20.7."
                }
            ],
            "id": "salt-minion",
            "jid": "20181010161053393111",
            "retcode": 0,
            "return": true,
            "success": true
        }
        
        
      6. Open Slack and you should see that your app has notified the room.

      Congratulations, you now know how to monitor your Salt minion’s memory usage with Slack integration. Salt can also monitor CPU load, disk usage, and a number of other things. Refer to the More Information section below for additional resources.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Join our Community

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Create a Salt Execution Module


      Updated by Linode Written by Linode

      Use promo code DOCS10 for $10 credit on a new account.

      A Salt execution module is a Python module that runs on a Salt minion. It perform tasks and returns data to the Salt master. In this tutorial you will create and install an execution module that will call the US National Weather Service API and return the current temperature at a specified weather station. This example could easily be adapted to access any API.

      Before You Begin

      If you haven’t already, set up a Salt master and at least one Salt minion. You can follow the first few steps of our Getting Started with Salt – Basic Installation and Setup guide.

      Note

      The steps in this guide require root privileges. Be sure to run the steps below with the sudo prefix. For more information on privileges, see our Users and Groups guide.

      Prepare Salt

      The files created in the following steps will be located in the /srv/salt directory. If you have changed Salt’s default file_roots configuration, use that directory location instead.

      1. Begin by creating the /srv/salt directory if it does not already exist. This is where you will place your top file and your Salt state file:

        mkdir /srv/salt
        
      2. Create a top file in /srv/salt which will be Salt’s point of entry for our Salt configuration:

        /srv/salt/top.sls
        1
        2
        3
        
        base:
          '*':
            - weather
      3. Create a state file named weather.sls and instruct Salt to make sure our minions have PIP installed, as well as the required Python library.

        /srv/salt/weather.sls
        1
        2
        3
        4
        5
        6
        7
        
        python-pip:
          pkg.installed
        
        requests:
          pip.installed:
            - require:
              - pkg: python-pip
      4. Apply these state changes:

        salt '*' state.apply
        
      5. Finally, create the /srv/salt/_modules directory which will contain our execution module:

        mkdir /srv/salt/_modules
        

      Create the Execution Module

      1. Create a file called weather.py in the /srv/salt/_modules directory, and add the following lines to set up Salt logging and import the requests module.

        /srv/salt/_modules/weather.py
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        
        import logging
        try:
            import requests
            HAS_REQUESTS = True
        except ImportError:
            HAS_REQUESTS = False
        
        log = logging.getLogger(__name__)
        
        . . .
      2. Add the __virtualname__ variable and the __virtual__ function.

        /srv/salt/_modules/weather.py
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        
        . . .
        
        __virtualname__ = 'weather'
        
        def __virtual__():
            '''
            Only load weather if requests is available
            '''
            if HAS_REQUESTS:
                return __virtualname__
            else:
                return False, 'The weather module cannot be loaded: requests package unavailable.'
        
        . . .

        The __virtual__ function either returns the module’s virtual name and loads the module, or returns False with an error string and the module is not loaded. The if HAS_REQUESTS conditional is tied to the try/except block created in the previous step through the use of the HAS_REQUESTS variable.

      3. Add the public get() function and the private _make_request() function:

        /srv/salt/_modules/weather.py
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        
        . . .
        
        def get(signs=None):
            '''
            Gets the Current Weather
        
            CLI Example::
        
                salt minion weather.get KPHL
        
            This module also accepts multiple values in a comma separated list::
        
                salt minion weather.get KPHL,KACY
            '''
            log.debug(signs)
            return_value = {}
            signs = signs.split(',')
            for sign in signs:
                return_value[sign] = _make_request(sign)
            return return_value
        
        def _make_request(sign):
            '''
            The function that makes the request for weather data from the National Weather Service.
            '''
            request = requests.get('https://api.weather.gov/stations/{}/observations/current'.format(sign))
            conditions = {
                "description:": request.json()["properties"]["textDescription"],
                "temperature": round(request.json()["properties"]["temperature"]["value"], 1)
            }
            return conditions

        There are two functions in this step. The get() function accepts one or more weather station call signs as a comma separated list. It calls _make_request() to make the HTTP request and returns a text description of the current weather and the temperature.

        It’s important to note that by adding an underscore to the beginning of the _make_request() function it becomes a private function, which means it is not directly accessible through the Salt command line or a state file.

        The complete file looks like this:

        /srv/salt/_modules/weather.py
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        38
        39
        40
        41
        42
        43
        44
        45
        46
        47
        48
        49
        50
        
        import logging
        try:
            import requests
            HAS_REQUESTS = True
        except ImportError:
            HAS_REQUESTS = False
        
        log = logging.getLogger(__name__)
        
        __virtual_name__ = 'weather'
        
        def __virtual__():
            '''
            Only load weather if requests is available
            '''
            if HAS_REQUESTS:
                return __virtual_name__
            else:
                return False, 'The weather module cannot be loaded: requests package unavailable.'
        
        
        def get(signs=None):
            '''
            Gets the Current Weather
        
            CLI Example::
        
                salt minion weather.get KPHL
        
            This module also accepts multiple values in a comma seperated list::
        
                salt minion weather.get KPHL,KACY
            '''
            log.debug(signs)
            return_value = {}
            signs = signs.split(',')
            for sign in signs:
                return_value[sign] = _make_request(sign)
            return return_value
        
        def _make_request(sign):
            '''
            The function that makes the request for weather data from the National Weather Service.
            '''
            request = requests.get('https://api.weather.gov/stations/{}/observations/current'.format(sign))
            conditions = {
                "description:": request.json()["properties"]["textDescription"],
                "temperature": round(request.json()["properties"]["temperature"]["value"], 1)
            }
            return conditions

      Run the Execution Module

      1. To run the execution module, you need to first sync it to your minions. To do this, you can call a highstate with state.apply, which will also try to apply the state changes you specified earlier in the weather.sls state file. Since the weather.sls state was already applied in the Preparing Salt section, use the saltutil.sync_modules function:

        salt '*' saltutil.sync_modules
        
      2. Run the execution module on your Salt master:

        salt '*' weather.get KPHL
        

        You should see an output like the following:

          
        salt-minion:
        ----------
        KPHL:
            ----------
            description::
                Cloudy
            temperature:
                17.2
        
        
      3. Alternatively, you can run the Salt execution module locally on your Salt minion by entering the following:

        salt-call weather.get KVAY,KACY
        

        You should get an output like the following:

          
        local:
            ----------
            KACY:
                ----------
                description::
                    Cloudy
                temperature:
                    18.9
            KVAY:
                ----------
                description::
                    Cloudy
                temperature:
                    16.7
        
        

      You have now successfully created and installed a Salt execution module.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Join our Community

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Introduction to Jinja Templates for Salt


      Updated by Linode Contributed by Linode

      Use promo code DOCS10 for $10 credit on a new account.

      Introduction to Templating Languages

      Jinja is a flexible templating language for Python that can be used to generate any text based format such as HTML, XML, and YAML. Templating languages like Jinja allow you to insert data into a structured format. You can also embed logic or control-flow statements into templates for greater reusability and modularity. Jinja’s template engine is responsible for processing the code within the templates and generating the output to the final text based document.

      Templating languages are well known within the context of creating web pages in a Model View Controller architecture. In this scenario the template engine processes source data, like the data found in a database, and a web template that includes a mixture of HTML and the templating language. These two pieces are then used to generate the final web page for users to consume. Templating languages, however, are not limited to web pages. Salt, a popular Python based configuration management software, supports Jinja to allow for abstraction and reuse within Salt state files and regular files.

      This guide will provide an overview of the Jinja templating language used primarily within Salt. If you are not yet familiar with Salt concepts, review the Beginner’s Guide to Salt before continuing. While you will not be creating Salt states of your own in this guide, it is also helpful to review the Getting Started with Salt – Basic Installation and Setup guide.

      Jinja Basics

      This section provides an introductory description of Jinja syntax and concepts along with examples of Jinja and Salt states. For an exhaustive dive into Jinja, consult the official Jinja Template Designer Documentation.

      Applications like Salt can define default behaviors for the Jinja templating engine. All examples in this guide use Salt’s default Jinja environment options. These settings can be changed in the Salt master configuration file:

      /etc/salt/master
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      
      # Default Jinja environment options for all templates except sls templates
      #jinja_env:
      #  block_start_string: '{%'
      #  block_end_string: '%}'
      #  variable_start_string: '{{'
      #  variable_end_string: '}}'
      #  comment_start_string: '{#'
      #  comment_end_string: '#}'
      #  line_statement_prefix:
      #  line_comment_prefix:
      #  trim_blocks: False
      #  lstrip_blocks: False
      #  newline_sequence: 'n'
      #  keep_trailing_newline: False
      
      # Jinja environment options for sls templates
      #jinja_sls_env:
      #  block_start_string: '{%'
      #  block_end_string: '%}'
      #  variable_start_string: '{{'
      #  variable_end_string: '}}'
      #  comment_start_string: '{#'
      #  comment_end_string: '#}'
      #  line_statement_prefix:
      #  line_comment_prefix:
      #  trim_blocks: False
      #  lstrip_blocks: False

      Note

      Before including Jinja in your Salt states, be sure to review the Salt and Jinja Best Practices section of this guide to ensure that you are creating maintainable and readable Salt states. More advanced Salt tools and concepts can be used to improve the modularity and reusability of some of the Jinja and Salt state examples used throughout this guide.

      Delimiters

      Templating language delimiters are used to denote the boundary between the templating language and another type of data format like HTML or YAML. Jinja uses the following delimiters:

      Delimiter SyntaxUsage
      {% ... %}Control structures
      {{ ... }}Evaluated expressions that will print to the template output
      {# ... #}Comments that will be ignored by the template engine
      # ... ##Line statements

      In this example Salt state file, you can differentiate the Jinja syntax from the YAML because of the {% ... %} delimiters surrounding the if/else conditionals:

      /srv/salt/webserver/init.sls
      1
      2
      3
      4
      5
      6
      7
      
      {% if grains['group'] == 'admin' %}
          America/Denver:
              timezone.system:
      {% else %}
          Europe/Minsk:
              timezone.system:
      {% endif %}

      See the control structures section for more information on conditionals.

      Template Variables

      Template variables are available via a template’s context dictionary. A template’s context dictionary is created automatically during the different stages of a template’s evaluation. These variables can be accessed using dot notation:

      {{ foo.bar }}
      

      Or they can be accessed by subscript syntax:

      {{ foo['bar'] }}
      

      Salt provides several context variables that are available by default to any Salt state file or file template:

      • Salt: The salt variable provides a powerful set of Salt library functions.

        {{ salt['pw_user.list_groups']('jdoe') }}
        

        You can run salt '*' sys.doc from the Salt master to view a list of all available functions.

      • Opts: The opts variable is a dictionary that provides access to the content of a Salt minion’s configuration file:

        {{ opts['log_file'] }}
        

        The location for a minion’s configuration file is /etc/salt/minion.

      • Pillar: The pillar variable is a dictionary used to access Salt’s pillar data:

        {{ pillar['my_key'] }}
        

        Although you can access pillar keys and values directly, it is recommended that you use Salt’s pillar.get variable library function, because it allows you to define a default value. This is useful when a value does not exist in the pillar:

        {{ salt['pillar.get']('my_key', 'default_value') }}
        
      • Grains: The grains variable is a dictionary and provides access to minions’ grains data:

        {{ grains['shell'] }}
        

        You can also use Salt’s grains.get variable library function to access grain data:

        {{ salt['grains.get']('shell') }}
        
      • Saltenv: You can define multiple salt environments for minions in a Salt master’s top file, such as base, prod, dev and test. The saltenv variable provides a way to access the current Salt environment within a Salt state file. This variable is only available within Salt state files.

        {{ saltenv }}
        
      • SLS: With the sls variable you can obtain the reference value for the current state file (e.g. apache, webserver, etc). This is the same value used in a top file to map minions to state files or via the include option in state files:

        {{ sls }}
        
      • Slspath: This variable provides the path to the current state file:

        {{ slspath }}
        

      Variable Assignments

      You can assign a value to a variable by using the set tag along with the following delimiter and syntax:

      {% set var_name = myvalue %}
      

      Follow Python naming conventions when creating variable names. If the variable is assigned at the top level of a template, the assignment is exported and available to be imported by other templates.

      Any value generated by a Salt template variable library function can be assigned to a new variable.

      {% set username = salt['user.info']('username') %}
      

      Filters

      Filters can be applied to any template variable via a | character. Filters are chainable and accept optional arguments within parentheses. When chaining filters, the output of one filter becomes the input of the following filter.

      {{ '/etc/salt/' | list_files | join('n') }}
      

      These chained filters will return a recursive list of all the files in the /etc/salt/ directory. Each list item will be joined with a new line.

        
        /etc/salt/master
        /etc/salt/proxy
        /etc/salt/minion
        /etc/salt/pillar/top.sls
        /etc/salt/pillar/device1.sls
        
      

      For a complete list of all built in Jinja filters, refer to the Jinja Template Design documentation. Salt’s official documentation includes a list of custom Jinja filters.

      Macros

      Macros are small, reusable templates that help you to minimize repetition when creating states. Define macros within Jinja templates to represent frequently used constructs and then reuse the macros in state files.

      /srv/salt/mysql/db_macro.sls
      1
      2
      3
      4
      5
      6
      7
      8
      
      {% macro mysql_privs(user, grant=select, database, host=localhost) %}
      {{ user }}_exampledb:
         mysql_grants.present:
          - grant: {{ grant }}
          - database: {{ database }}
          - user: {{user}}
          - host: {{ host }}
      {% endmacro %}
      db_privs.sls
      1
      2
      3
      
      {% import "/srv/salt/mysql/db_macro.sls" as db -%}
      
      db.mysql_privs('jane','exampledb.*','select,insert,update')

      The mysql_privs() macro is defined in the db_macro.sls file. The template is then imported to the db variable in the db_privs.sls state file and is used to create a MySQL grants state for a specific user.

      Refer to the Imports and Includes section for more information on importing templates and variables.

      Imports and Includes

      Imports

      Importing in Jinja is similar to importing in Python. You can import an entire template, a specific state, or a macro defined within a file.

      {% import '/srv/salt/users.sls' as users %}
      

      This example will import the state file users.sls into the variable users. All states and macros defined within the template will be available using dot notation.

      You can also import a specific state or macro from a file.

      {% from '/srv/salt/user.sls' import mysql_privs as grants %}
      

      This import targets the macro mysql_privs defined within the user.sls state file and is made available to the current template with the grants variable.

      Includes

      The {% include %} tag renders the output of another template into the position where the include tag is declared. When using the {% include %} tag the context of the included template is passed to the invoking template.

      /srv/salt/webserver/webserver_users.sls
      1
      2
      3
      4
      
      include:
        - groups
      
      {% include 'users.sls' %}

      Note

      Import Context Behavior

      By default, an import will not include the context of the imported template, because imports are cached. This can be overridden by adding with context to your import statements.

      {% from '/srv/salt/user.sls' import mysql_privs with context %}
      

      Similarly, if you would like to remove the context from an {% include %}, add without context:

      {% include 'users.sls' without context %}
      

      Whitespace Control

      Jinja provides several mechanisms for whitespace control of its rendered output. By default, Jinja strips single trailing new lines and leaves anything else unchanged, e.g. tabs, spaces, and multiple new lines. You can customize how Salt’s Jinja template engine handles whitespace in the Salt master configuration file. Some of the available environment options for whitespace control are:

      • trim_blocks: When set to True, the first newline after a template tag is removed automatically. This is set to False by default in Salt.
      • lstrip_blocks: When set to True, Jinja strips tabs and spaces from the beginning of a line to the start of a block. If other characters are present before the start of the block, nothing will be stripped. This is set to False by default in Salt.
      • keep_trailing_newline: When set to True, Jinja will keep single trailing newlines. This is set to False by default in Salt.

      To avoid running into YAML syntax errors, ensure that you take Jinja’s whitespace rendering behavior into consideration when inserting templating markup into Salt states. Remember, Jinja must produce valid YAML. When using control structures or macros, it may be necessary to strip whitespace from the template block to appropriately render valid YAML.

      To preserve the whitespace of contents within template blocks, you can set both the trim_blocks and lstrip_block options to True in the master configuration file. You can also manually enable and disable the white space environment options within each template block. A - character will set the behavior of trim_blocks and lstrip_blocks to False and a + character will set these options to True for the block:

      For example, to strip the whitespace after the beginning of the control structure include a - character before the closing %}:

      {% for item in [1,2,3,4,5] -%}
          {{ item }}
      {% endfor %}
      

      This will output the numbers 12345 without any leading whitespace. Without the - character, the output would preserve the spacing defined within the block.

      Control Structures

      Jinja provides control structures common to many programming languages such as loops, conditionals, macros, and blocks. The use of control structures within Salt states allow for fine-grained control of state execution flow.

      For Loops

      For loops allow you to iterate through a list of items and execute the same code or configuration for each item in the list. Loops provide a way to reduce repetition within Salt states.

      /srv/salt/users.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      
      {% set groups = ['sudo','wheel', 'admins'] %}
      include:
        - groups
      
      jane:
        user.present:
          - fullname: Jane Doe
          - shell: /bin/zsh
          - createhome: True
          - home: /home/jane
          - uid: 4001
          - groups:
          {%- for group in groups %}
            - {{ group }}
          {%- endfor -%}

      The previous for loop will assign the user jane to all the groups in the groups list set at the top of the users.sls file.

      Conditionals

      A conditional expression evaluates to either True or False and controls the flow of a program based on the result of the evaluated boolean expression. Jinja’s conditional expressions are prefixed with if/elif/else and placed within the {% ... %} delimiter.

      /srv/salt/users.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      
      {% set users = ['anna','juan','genaro','mirza'] %}
      {% set admin_users = ['genaro','mirza'] %}
      {% set admin_groups = ['sudo','wheel', 'admins'] %}
      {% set org_groups = ['games', 'webserver'] %}
      
      
      include:
        - groups
      
      {% for user in users %}
      {{ user }}:
        user.present:
          - shell: /bin/zsh
          - createhome: True
          - home: /home/{{ user }}
          - groups:
      {% if user in admin_users %}
          {%- for admin_group in admin_groups %}
            - {{ admin_group }}
          {%- endfor -%}
      {% else %}
          {%- for org_group in org_groups %}
            - {{ org_group }}
          {% endfor %}
      {%- endif -%}
      {% endfor %}

      In this example the presence of a user within the admin_users list determines which groups are set for that user in the state. Refer to the Salt Best Practices section for more information on using conditionals and control flow statements within state files.

      Template Inheritance

      With template inheritance you can define a base template that can be reused by child templates. The child template can override blocks designated by the base template.

      Use the {% block block_name %} tag with a block name to define an area of a base template that can be overridden.

      /srv/salt/users.jinja
      1
      2
      3
      4
      5
      6
      7
      8
      9
      
      {% block user %}jane{% endblock %}:
        user.present:
          - fullname: {% block fullname %}{% endblock %}
          - shell: /bin/zsh
          - createhome: True
          - home: /home/{% block home_dir %}
          - uid: 4000
          - groups:
            - sudo

      This example creates a base user state template. Any value containing a {% block %} tag can be overridden by a child template with its own value.

      To use a base template within a child template, use the {% extends "base.sls"%} tag with the location of the base template file.

      /srv/salt/webserver_users.sls
      1
      2
      3
      4
      
      {% extends "/srv/salt/users.jinja" %}
      
      {% block fullname %}{{ salt['pillar.get']('jane:fullname', '') }}{% endblock %}
      {% block home_dir %}{{ salt['pillar.get']('jane:home_dir', 'jane') }}{% endblock %}

      The webserver_users.sls state file extends the users.jinja template and defines values for the fullname and home_dir blocks. The values are generated using the salt context variable and pillar data. The rest of the state will be rendered as the parent user.jinja template has defined it.

      Salt and Jinja Best Practices

      If Jinja is overused, its power and versatility can create unmaintainable Salt state files that are difficult to read. Here are some best practices to ensure that you are using Jinja effectively:

      • Limit how much Jinja you use within state files. It is best to separate the data from the state that will use the data. This allows you to update your data without having to alter your states.
      • Do not overuse conditionals and looping within state files. Overuse will make it difficult to read, understand and maintain your states.
      • Use dictionaries of variables and directly serialize them into YAML, instead of trying to create valid YAML within a template. You can include your logic within the dictionary and retrieve the necessary variable within your states.

        The {% load_yaml %} tag will deserialize strings and variables passed to it.

         {% load_yaml as example_yaml %}
             user: jane
             firstname: Jane
             lastname: Doe
         {% endload %}
        
         {{ example_yaml.user }}:
            user.present:
              - fullname: {{ example_yaml.firstname }} {{ example_yaml.lastname }}
              - shell: /bin/zsh
              - createhome: True
              - home: /home/{{ example_yaml.user }}
              - uid: 4001
              - groups:
                - games
        

        Use {% import_yaml %} to import external files of data and make the data available as a Jinja variable.

         {% import_yaml "users.yml" as users %}
        
      • Use Salt Pillars to store general or sensitive data as variables. Access these variables inside state files and template files.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Join our Community

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link