One place for hosting & domains

      Ubuntu

      How to Set Up Squid Proxy for Private Connections on Ubuntu 20.04


      Introduction

      Proxy servers are a type of server application that functions as a gateway between an end user and an internet resource. Through a proxy server, an end user is able to control and monitor their web traffic for a wide variety of purposes, including privacy, security, and caching. For example, you can use a proxy server to make web requests from a different IP address than your own. You can also use a proxy server to research how the web is served differently from one jurisdiction to the next, or avoid some methods of surveillance or web traffic throttling.

      Squid is a stable, popular, open-source HTTP proxy. In this tutorial, you will be installing and configuring Squid to provide an HTTP proxy on a Ubuntu 20.04 server.

      Prerequisites

      To complete this guide, you will need:

      You will use the domain name your_domain in this tutorial, but you should substitute this with your own domain name, or IP address.

      Step 1 — Installing Squid Proxy

      Squid has many use cases beyond routing an individual user’s outbound traffic. In the context of large-scale server deployments, it can be used as a distributed caching mechanism, a load balancer, or another component of a routing stack. However, some methods of horizontally scaling server traffic that would typically have involved a proxy server have been surpassed in popularity by containerization frameworks such as Kubernetes, which distribute more components of an application. At the same time, using proxy servers to redirect web requests as an individual user has become increasingly popular for protecting your privacy. This is helpful to keep in mind when working with open-source proxy servers which may appear to have many dozens of features in a lower-priority maintenance mode. The use cases for a proxy have changed over time, but the fundamental technology has not.

      Begin by running the following commands as a non-root user to update your package listings and install Squid Proxy:

      • sudo apt update
      • sudo apt install squid

      Squid will automatically set up a background service and start after being installed. You can check that the service is running properly:

      • systemctl status squid.service

      Output

      ● squid.service - Squid Web Proxy Server Loaded: loaded (/lib/systemd/system/squid.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-12-15 21:45:15 UTC; 2min 11s ago

      By default, Squid does not allow any clients to connect to it from outside of this server. In order to enable that, you’ll need to make some changes to its configuration file, which is stored in /etc/squid/squid.conf. Open it in nano or your favorite text editor:

      • sudo nano /etc/squid/squid.conf

      Be advised that Squid’s default configuration file is very, very long, and contains a massive number of options that have been temporarily disabled by putting a # at the start of the line they’re on, also called being commented out. You will most likely want to search through the file to find the lines you want to edit. In nano, this is done by pressing Ctrl+W, entering your search term, pressing Enter, and then repeatedly pressing Alt+W to find the next instance of that term if needed.

      Begin by navigating to the line containing the phrase http_access deny all. You should see a block of text explaining Squid’s default access rules:

      /etc/squid/squid.conf

      . . . 
      #
      # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
      #
      include /etc/squid/conf.d/*
      # Example rule allowing access from your local networks.
      # Adapt localnet in the ACL section to list your (internal) IP networks
      # from where browsing should be allowed
      #http_access allow localnet
      http_access allow localhost
      
      # And finally deny all other access to this proxy
      http_access deny all
      . . . 
      

      From this, you can see the current behavior – localhost is allowed; other connections are not. Note that these rules are parsed sequentially, so it’s a good idea to keep the deny all rule at the bottom of this configuration block. You could change that rule to allow all, enabling anyone to connect to your proxy server, but you probably don’t want to do that. Instead, you can add a line above http_access allow localhost that includes your own IP address, like so:

      /etc/squid/squid.conf

      #
      # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
      #
      include /etc/squid/conf.d/*
      # Example rule allowing access from your local networks.
      acl localnet src your_ip_address
      # Adapt localnet in the ACL section to list your (internal) IP networks
      # from where browsing should be allowed
      #http_access allow localnet
      http_access allow localhost
      
      • acl means an Access Control List, a common term for permissions policies
      • localnet in this case is the name of your ACL.
      • src is where the request would originate from under this ACL, i.e., your IP address.

      If you don’t know your local IP address, it’s quickest to go to a site like What’s my IP which can tell you where you accessed it from. After making that change, save and close the file. If you are using nano, press Ctrl+X, and then when prompted, Y and then Enter.

      At this point, you could restart Squid and connect to it, but there’s more you can do in order to secure it first.

      Step 2 — Securing Squid

      Most proxies, and most client-side apps that connect to proxies (e.g., web browsers) support multiple methods of authentication. These can include shared keys, or separate authentication servers, but most commonly entail regular username-password pairs. Squid allows you to create username-password pairs using built-in Linux functionality, as an additional or an alternative step to restricting access to your proxy by IP address. To do that, you’ll create a file called /etc/squid/passwords and point Squid’s configuration to it.

      First, you’ll need to install some utilities from the Apache project in order to have access to a password generator that Squid likes.

      • sudo apt install apache2-utils

      This package provides the htpasswd command, which you can use in order to generate a password for a new Squid user. Squid’s usernames won’t overlap with system usernames in any way, so you can use the same name you’ve logged in with if you want. You’ll be prompted to add a password as well:

      • sudo htpasswd -c /etc/squid/passwords your_squid_username

      This will store your username along with a hash of your new password in /etc/squid/passwords, which will be used as an authentication source by Squid. You can cat the file afterward to see what that looks like:

      • sudo cat /etc/squid/passwords

      Output

      sammy:$apr1$Dgl.Mtnd$vdqLYjBGdtoWA47w4q1Td.

      After verifying that your username and password have been stored, you can update Squid’s configuration to use your new /etc/squid/passwords file. Using nano or your favorite text editor, reopen the Squid configuration file and add the following highlighted lines:

      • sudo nano /etc/squid/squid.conf

      /etc/squid/squid.conf

      …
      #
      # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
      #
      include /etc/squid/conf.d/*
      auth_param basic program /usr/lib/squid3/basic_ncsa_auth /etc/squid/passwords
      auth_param basic realm proxy
      acl authenticated proxy_auth REQUIRED
      # Example rule allowing access from your local networks.
      acl localnet src your_ip_address
      # Adapt localnet in the ACL section to list your (internal) IP networks
      # from where browsing should be allowed
      #http_access allow localnet
      http_access allow localhost
      http_access allow authenticated
      # And finally deny all other access to this proxy
      http_access deny all
      …
      

      These additional directives tell Squid to check in your new passwords file for password hashes that can be parsed using the basic_ncsa_auth mechanism, and to require authentication for access to your proxy. You can review Squid’s documentation for more information on this or other authentication methods. After that, you can finally restart Squid with your configuration changes. This might take a moment to complete.

      • sudo systemctl restart squid.service

      And don’t forget to open port 3128 in your firewall if you’re using ufw:

      In the next step, you’ll connect to your proxy at last.

      Step 3 — Connecting through Squid

      In order to demonstrate your Squid server, you’ll use a command line program called curl, which is popular for making different types of web requests. In general, if you want to verify whether a given connection should be working in a browser under ideal circumstances, you should always test first with curl. You’ll be using curl on your local machine in order to do this – it’s installed by default on all modern Windows, Mac, and Linux environments, so you can open any local shell to run this command:

      • curl -v -x http://your_squid_username:your_squid_password@your_server_ip:3128 http://www.google.com/

      The -x argument passes a proxy server to curl, and in this case you’re using the http:// protocol this time, specifying your username and password to this server, and then connecting to a known-working website like google.com. If the command was successful, you should see the following output:

      Output

      * Trying 138.197.103.77... * TCP_NODELAY set * Connected to 138.197.103.77 (138.197.103.77) port 3128 (#0) * Proxy auth using Basic with user 'sammy' > GET http://www.google.com/ HTTP/1.1

      It is also possible to access https:// websites with your Squid proxy without making any further configuration changes. These make use of a separate proxy directive called CONNECT in order to preserve SSL between the client and the server:

      • curl -v -x http://your_squid_username:your_squid_password@your_server_ip:3128 https://www.google.com/

      Output

      * Trying 138.197.103.77... * TCP_NODELAY set * Connected to 138.197.103.77 (138.197.103.77) port 3128 (#0) * allocate connect buffer! * Establish HTTP proxy tunnel to www.google.com:443 * Proxy auth using Basic with user 'sammy' > CONNECT www.google.com:443 HTTP/1.1 > Host: www.google.com:443 > Proxy-Authorization: Basic c2FtbXk6c2FtbXk= > User-Agent: curl/7.55.1 > Proxy-Connection: Keep-Alive > < HTTP/1.1 200 Connection established < * Proxy replied OK to CONNECT request * CONNECT phase completed!

      The credentials that you used for curl should now work anywhere else you might want to use your new proxy server.

      Conclusion

      In this tutorial, you learned to deploy a popular, open-source API endpoint for proxying traffic with little to no overhead. Many applications have built-in proxy support (often at the OS level) going back decades, making this proxy stack highly reusable.

      Next, you may want to learn how to deploy Dante, a SOCKS proxy which can run alongside Squid for proxying different types of web traffic.

      Because one of the most common use cases for proxy servers is proxying traffic to and from different global regions, you may want to review how to use Ansible to automate server deployments next, in case you find yourself wanting to duplicate this configuration in other data centers.



      Source link

      How to Set Up Dante Proxy for Private Connections on Ubuntu 20.04


      Introduction

      Proxy servers are a type of server application that functions as a gateway between an end user and an internet resource. Through a proxy server, an end user is able to control and monitor their web traffic for a wide variety of purposes, including privacy, security, and caching. For example, you can use a proxy server to make web requests from a different IP address than your own. You can also use a proxy server to research how the web is served differently from one jurisdiction to the next, or avoid some methods of surveillance or web traffic throttling.

      Dante is a stable, popular, open-source SOCKS proxy. In this tutorial, you will be installing and configuring Dante to provide a SOCKS proxy on a Ubuntu 20.04 server.

      Prerequisites

      To complete this guide, you will need:

      You will use the domain name your_domain in this tutorial, but you should substitute this with your own domain name, or IP address.

      Step 1 — Installing Dante

      Dante is an open-source SOCKS proxy server. SOCKS is a less widely used protocol, but it is more efficient for some peer-to-peer applications, and is preferred over HTTP for some kinds of traffic. Begin by running the following commands as a non-root user to update your package listings and install Dante:

      • sudo apt update
      • sudo apt install dante-server

      Dante will also automatically set up a background service and start after being installed. However, it is designed to gracefully quit with an error message the first time it runs, because it ships with all of its features disabled. You can verify this by using the systemctl command:

      • systemctl status danted.service

      Output

      ● danted.service - SOCKS (v4 and v5) proxy daemon (danted) Loaded: loaded (/lib/systemd/system/danted.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Wed 2021-12-15 21:48:22 UTC; 1min 45s ago Docs: man:danted(8) man:danted.conf(5) Main PID: 14496 (code=exited, status=1/FAILURE) Dec 15 21:48:21 proxies systemd[1]: Starting SOCKS (v4 and v5) proxy daemon (danted)... Dec 15 21:48:22 proxies systemd[1]: Started SOCKS (v4 and v5) proxy daemon (danted). Dec 15 21:48:22 proxies danted[14496]: Dec 15 21:48:22 (1639604902.102601) danted[14496]: warning: checkconfig(): no socks authentication methods enabled. This means all socks requests will be blocked after negotiation. Perhaps this is not intended?

      To successfully start Dante’s services, you’ll need to enable them in the config file.

      Dante’s config file is provided, by default, in /etc/danted.conf. If you open this file using nano or your favorite text editor, you will see a long list of configuration options, all of them disabled. You could try to navigate through this file and enable some options line-by-line, but in practice it will be more efficient and more readable to delete this file and replace it from scratch. Don’t worry about doing this. You can always review Dante’s default configuration by navigating to its online manual, and you could even redownload the package manually from Ubuntu’s package listing to reobtain the stock configuration file if you ever wanted. In the meantime, go ahead and delete it:

      Now you can replace it with something more concise. Opening a file with a text editor will automatically create the file if it doesn’t exist, so by using nano or your favorite text editor, you should now get an empty configuration file:

      • sudo nano /etc/danted.conf

      Add the following contents:

      /etc/danted.conf

      logoutput: syslog
      user.privileged: root
      user.unprivileged: nobody
      
      # The listening network interface or address.
      internal: 0.0.0.0 port=1080
      
      # The proxying network interface or address.
      external: eth0
      
      # socks-rules determine what is proxied through the external interface.
      socksmethod: username
      
      # client-rules determine who can connect to the internal interface.
      clientmethod: none
      
      client pass {
          from: 0.0.0.0/0 to: 0.0.0.0/0
      }
      
      socks pass {
          from: 0.0.0.0/0 to: 0.0.0.0/0
      }
      

      You now have a usable SOCKS server configuration, running on port 1080, which is a common convention for SOCKS. You can also break down the rest of this configuration file line-by-line:

      • logoutput refers to how Dante will log connections, in this case using regular system logging
      • user.privileged allows dante to have root permissions for checking permissions
      • user.unprivileged does not grant the server any permissions for running as an unprivileged user, as this is unnecessary when not granting more granular permissions
      • internal connection details specify the port that the service is running on and which IP addresses can connect
      • external connection details specify the network interface used for outbound connections, eth0 by default on most servers

      The rest of the configuration details deal with authentication methods, which are discussed in the next section. Don’t forget to open port 1080 in your firewall if you’re using ufw:

      At this point, you could restart Dante and connect to it, but you would have a SOCKS server that’s open to the entire world, which you probably don’t want, so you’ll learn how to secure it first.

      Step 2 — Securing Dante

      If you followed this tutorial so far, Dante will be making use of regular Linux user accounts for authentication. This is helpful, but the password used for that connection will be sent over plain text, so it’s important to create a dedicated SOCKS user that won’t have any other login privileges. To do that, you’ll use useradd with flags that won’t assign a login shell to the user, then set a password:

      • sudo useradd -r -s /bin/false your_dante_user
      • sudo passwd your_dante_user

      You’ll also want to avoid logging into this account over an unsecured wireless connection or sharing the server too widely. Otherwise, malicious actors can and will make repeated efforts to log in.

      Dante supports other authentication methods, but many clients (i.e., applications) that will connect to SOCKS proxies only support basic username and password authentication, so you may want to leave that part as-is. What you can do as an alternative is to restrict access to only specific IP addresses. This isn’t the most sophisticated option, but given the combination of technologies in use here, it’s a sensible one. You may have already learned how to restrict access to specific IP addresses with ufw from our prerequisite tutorials , but you can also do it within Dante directly. Edit your /etc/danted.conf:

      • sudo nano /etc/danted.conf

      /etc/danted.conf

      …
      client pass {
          from: your_ip_address/0 to: 0.0.0.0/0
      }
      

      In order to support multiple IP addresses, you can use CIDR notation, or just add another client pass {} configuration block:

      /etc/danted.conf

      client pass {
          from: your_ip_address/0 to: 0.0.0.0/0
      }
      
      client pass {
          from: another_ip_address/0 to: 0.0.0.0/0
      }
      

      After that, you can finally restart Dante with your configuration changes.

      • sudo systemctl restart danted.service

      This time, when you check the service status, you should see it running without any errors:

      • systemctl status danted.service

      Output

      ● danted.service - SOCKS (v4 and v5) proxy daemon (danted) Loaded: loaded (/lib/systemd/system/danted.service; enabled; vendor preset: enable> Active: active (running) since Thu 2021-12-16 18:06:26 UTC; 24h ago

      In the next step, you’ll connect to your proxy at last.

      Step 3 — Connecting through Dante

      In order to demonstrate your Dante server, you’ll use a command line program called curl, which is popular for making different types of web requests. In general, if you want to verify whether a given connection should be working in a browser under ideal circumstances, you should always test first with curl. You’ll be using curl on your local machine in order to do this – it’s installed by default on all modern Windows, Mac, and Linux environments, so you can open any local shell to run this command:

      • curl -v -x socks5://your_dante_user:your_dante_password@your_server_ip:1080 http://www.google.com/

      Output

      * Trying 138.197.103.77... * TCP_NODELAY set * SOCKS5 communication to www.google.com:80 * SOCKS5 connect to IPv4 142.250.189.228 (locally resolved) * SOCKS5 request granted. * Connected to 138.197.103.77 (138.197.103.77) port 1080 (#0) > GET / HTTP/1.1 …

      The credentials that you used for curl should now work anywhere else you might want to use your new proxy server.

      Conclusion

      In this tutorial, you learned to deploy a popular, open-source API endpoint for proxying traffic with little to no overhead. Many applications have built-in proxy support (often at the OS level) going back decades, making this proxy stack highly reusable.

      Next, you may want to learn how to deploy Squid, an HTTP proxy which can run alongside Dante for proxying different types of web traffic.

      Because one of the most common use cases for proxy servers is proxying traffic to and from different global regions, you may want to review how to use Ansible to automate server deployments next, in case you find yourself wanting to duplicate this configuration in other data centers.



      Source link

      How To Set Up Continuous Integration Pipelines in Jenkins on Ubuntu 20.04


      Introduction

      Jenkins is an open source automation server intended to automate repetitive technical tasks involved in the continuous integration and delivery of software. With a robust ecosystem of plugins and broad support, Jenkins can handle a diverse set of workloads to build, test, and deploy applications.

      In previous guides, we installed Jenkins on an Ubuntu 20.04 server and configured Jenkins with SSL using an Nginx reverse proxy. In this guide, we will demonstrate how to set up Jenkins to automatically test an application when changes are pushed to a repository.

      For this tutorial, we will be integrating Jenkins with GitHub so that Jenkins is notified when new code is pushed to the repository. When Jenkins is notified, it will checkout the code and then test it within Docker containers to isolate the test environment from the Jenkins host machine. We will be using an example Node.js application to show how to define the CI/CD process for a project.

      Prerequisites

      To follow along with this guide, you will need an Ubuntu 20.04 server with at least 1G of RAM configured with a secure Jenkins installation. To properly secure the web interface, you will need to assign a domain name to the Jenkins server. Follow these guides to learn how to set up Jenkins in the expected format:

      To best control our testing environment, we will run our application’s tests within Docker containers. After Jenkins is up and running, install Docker on the server by following steps one and two of this guide:

      When you have completed the above guides, you can continue on with this article.

      Add the Jenkins User to the Docker Group

      After following the prerequisites, both Jenkins and Docker are installed on your server. However, by default, the Linux user responsible for running the Jenkins process cannot access Docker.

      To fix this, we need to add the jenkins user to the docker group using the usermod command:

      • sudo usermod -aG docker jenkins

      You can list the members of the docker group to confirm that the jenkins user has been added successfully:

      Output

      docker:x:999:sammy,jenkins

      In order for the Jenkins to use its new membership, you need to restart the process:

      • sudo systemctl restart jenkins

      If you installed Jenkins with the default plugins, you may need to check to ensure that the docker and docker-pipeline plugins are also enabled. To do so, click Manage Jenkins from the sidebar, and then Manage Plugins from the next menu. Click on the Available tab of the plugin menu to search for new plugins, and type docker into the search bar. If both Docker Pipeline and Docker plugin are returned as options, and they are unselected, select both, and when prompted, allow Jenkins to restart with the new plugins enabled.

      Jenkins Docker plugins

      This should take approximately a minute and the page will refresh afterward.

      Create a Personal Access Token in GitHub

      In order for Jenkins to watch your GitHub projects, you will need to create a Personal Access Token in our GitHub account.

      Begin by visiting GitHub and signing into your account if you haven’t already done so. Afterwards, click on your user icon in the upper-right hand corner and select Settings from the drop down menu:

      GitHub settings item

      On the page that follows, locate the Developer settings section of the left-hand menu and click Personal access tokens:

      GitHub personal access tokens link

      Click on Generate new token button on the next page:

      GitHub generate new token button

      You will be taken to a page where you can define the scope for your new token.

      In the Token description box, add a description that will allow you to recognize it later:

      GitHub token description

      In the Select scopes section, check the repo:status, repo:public_repo and admin:org_hook boxes. These will allow Jenkins to update commit statuses and to create webhooks for the project. If you are using a private repository, you will need to select the general repo permission instead of the repo subitems:

      GitHub token scope

      When you are finished, click Generate token at the bottom.

      You will be redirected back to the Personal access tokens index page and your new token will displayed:

      GitHub view new token

      Copy the token now so that we can reference it later. As the message indicates, there is no way to retrieve the token once you leave this page.

      Note: As mentioned in the screenshot above, for security reasons, there is no way to redisplay the token once you leave this page. If you lose your token, delete the current token from your GitHub account and then create a new one.

      Now that you have a personal access token for your GitHub account, we can configure Jenkins to watch your project’s repository.

      Add the GitHub Personal Access Token to Jenkins

      Now that we have a token, we need to add it to our Jenkins server so it can automatically set up webhooks. Log into your Jenkins web interface using the administrative account you configured during installation.

      Click on your username in the top-right corner to access your user settings, and from there, click Credentials in the left-hand menu. :

      Jenkins credentials item

      On the next page, click the arrow next to (global) within the Jenkins scope. In the box that appears, click Add credentials:

      Jenkins add credentials button

      You will be taken to a form to add new credentials.

      Under the Kind drop down menu, select Secret text. In the Secret field, paste your GitHub personal access token. Fill out the Description field so that you will be able to identify this entry at a later date. You can leave the Scope as Global and the ID field blank:

      Jenkins credentials form

      Click the OK button when you are finished.

      You will now be able to reference these credentials from other parts of Jenkins to aid in configuration.

      Set Up Jenkins Access to GitHub

      Back in the main Jenkins dashboard, click Manage Jenkins in the left hand menu:

      Jenkins sidebar

      In the list of links on the following page, click Configure System:

      Jenkins configure system link

      Scroll through the options on the next page until you find the GitHub section. Click the Add GitHub Server button and then select GitHub Server:

      Jenkins add GitHub server

      The section will expand to prompt for some additional information. In the Credentials drop down menu, select your GitHub personal access token that you added in the last section:

      Jenkins select GitHub credentials

      Click the Test connection button. Jenkins will make a test API call to your account and verify connectivity:

      Jenkins test GitHub credentials

      When you are finished, click the Save button to implement your changes.

      Set Up the Demonstration Application in your GitHub Account

      To demonstrate how to use Jenkins to test an application, we will be using a “hello world” program created with Hapi.js. Because we are setting up Jenkins to react to pushes to the repository, you need to have your own copy of the demonstration code.

      Visit the project repository and click the Fork button in the upper-right corner to make a copy of the repository in your account:

      Fork example project

      A copy of the repository will be added to your account.

      The repository contains a package.json file that defines the runtime and development dependencies, as well as how to run the included test suite. The dependencies can be installed by running npm install and the tests can be run using npm test.

      We’ve added a Jenkinsfile to the repo as well. Jenkins reads this file to determine the actions to run against the repository to build, test, or deploy. It is written using the declarative version of the Jenkins Pipeline DSL.

      The Jenkinsfile included in the hello-hapi repository looks like this:

      Jenkinsfile

      #!/usr/bin/env groovy
      
      pipeline {
      
          agent {
              docker {
                  image 'node'
                  args '-u root'
              }
          }
      
          stages {
              stage('Build') {
                  steps {
                      echo 'Building...'
                      sh 'npm install'
                  }
              }
              stage('Test') {
                  steps {
                      echo 'Testing...'
                      sh 'npm test'
                  }
              }
          }
      }
      

      The pipeline contains the entire definition that Jenkins will evaluate. Inside, we have an agent section that specifies where the actions in the pipeline will execute. To isolate our environments from the host system, we will be testing in Docker containers, specified by the docker agent.

      Since Hapi.js is a framework for Node.js, we will be using the node Docker image as our base. We specify the root user within the container so that the user can simultaneously write to both the attached volume containing the checked out code, and to the volume the script writes its output to.

      Next, the file defines two stages, i.e., logical divisions of work. We’ve named the first one “Build” and the second “Test”. The build step prints a diagnostic message and then runs npm install to obtain the required dependencies. The test step prints another message and then run the tests as defined in the package.json file.

      Now that you have a repository with a valid Jenkinsfile, we can set up Jenkins to watch this repository and run the file when changes are introduced.

      Create a New Pipeline in Jenkins

      Next, we can set up Jenkins to use the GitHub personal access token to watch our repository.

      Back in the main Jenkins dashboard, click New Item in the left hand menu:

      Jenkins side menu

      Enter a name for your new pipeline in the Enter an item name field. Afterwards, select Pipeline as the item type:

      Jenkins pipeline type

      Click the OK button at the bottom to move on.

      On the next screen, check the GitHub project box. In the Project url field that appears, enter your project’s GitHub repository URL.

      Note: Make sure to point to your fork of the Hello Hapi application so that Jenkins will have permission to configure webhooks.

      Jenkins add GitHub project

      Next, in the Build Triggers section, check the GitHub hook trigger for GITScm polling box:

      Jenkins GitHub hook box

      In the Pipeline section, we need to tell Jenkins to run the pipeline defined in the Jenkinsfile in our repository. Change the Definition type to Pipeline script from SCM.

      In the new section that appears, choose Git in the SCM menu. In the Repository URL field that appears, enter the URL to your fork of the repository again:

      Note: Again, make sure to point to your fork of the Hello Hapi application.

      Jenkins GitHub add pipeline repository

      Note: Our example references a Jenkinsfile available within a public repository. If your project is not publicly accessible, you will need to use the add credentials button to add additional access to the repository. You can add a personal access token as we did with the hooks configuration earlier.

      When you are finished, click the Save button at the bottom of the page.

      Performing an Initial Build and Configuring the Webhooks

      Jenkins does not automatically configure webhooks when you define the pipeline for the repository in the interface. In order to trigger Jenkins to set up the appropriate hooks, we need to perform a manual build the first time.

      In your pipeline’s main page, click Build Now in the left hand menu:

      Jenkins build pipeline now

      A new build will be scheduled. In the Build History box in the lower left corner, a new build should appear in a moment. Additionally, a Stage View will begin to be drawn in the main area of the interface. This will track the progress of your testing run as the different stages are completed:

      Jenkins build progress

      In the Build History box, click on the number associated with the build to go to the build detail page. From here, you can click the Console Output button in the left hand menu to see details of the steps that were run:

      Jenkins console output

      Click the Back to Project item in the left hand menu when you are finished in order to return to the main pipeline view.

      Now that we’ve built the project once, we can have Jenkins create the webhooks for our project. Click Configure in the left hand menu of the pipeline:

      Jenkins configure item

      No changes are necessary on this screen, just click the Save button at the bottom. Now that the Jenkins has information about the project from the initial build process, it will register a webhook with our GitHub project when you save the page.

      You can verify this by going to your GitHub repository and clicking the Settings button. On the next page, click Webhooks from the side menu. You should see your Jenkins server webhook in the main interface:

      Jenkins view webhooks

      If for any reason Jenkins failed to register the hook (for example, due to upstream API changes or outages between Jenkins and Github), you can quickly add one yourself by clicking Add webhook and ensuring that the Payload URL is set to https://my-jenkins-server:8080/github-webhook and the Content type is set to application/json, then clicking Add webhook again at the bottom of the prompt.

      Github add webhook

      Now, when you push new changes to your repository, Jenkins will be notified. It will then pull the new code and retest it using the same procedure.

      To approximate this, in our repository page on GitHub, you can click the Create new file button to the left of the green Clone or download button:

      Jenkins create new file button

      On the next page, choose a filename and some dummy contents:

      Jenkins new file contents

      Click the Commit new file button at the bottom when you are finished.

      If you return to your Jenkins interface, you will see a new build automatically started:

      Jenkins new build started

      You can kick off additional builds by making commits to a local copy of the repository and pushing it back up to GitHub.

      Conclusion

      In this guide, we configured Jenkins to watch a GitHub project and automatically test any new changes that are committed. Jenkins pulls code from the repository and then runs the build and testing procedures from within isolated Docker containers. The resulting code can be deployed or stored by adding additional instructions to the same Jenkinsfile.

      To learn more about GitHub Actions, refer to GitHub’s documentation.



      Source link