One place for hosting & domains

      LKE

      How to Deploy TOBS on LKE


      Updated
      , by Rajakavitha Kodhandapani

      Traducciones al Español

      Estamos traduciendo nuestros guías y tutoriales al Español. Es
      posible que usted esté viendo una traducción generada
      automáticamente. Estamos trabajando con traductores profesionales
      para verificar las traducciones de nuestro sitio web. Este proyecto
      es un trabajo en curso.

      Create a Linode account
      to try this guide with a $100 credit.

      This credit will be applied to any valid services used during your first 60 days.

      In this guide, deploy
      TOBS to your Linode Kubernetes Engine (LKE) cluster using
      Helm. And use kubectl port-forward for local access to your monitoring interfaces.

      The Prometheus Operator Monitoring Stack

      TOBS, short for The Observability Stack, is a pre-packaged distribution of monitoring tools and dashboard interfaces which can be installed on any existing Kubernetes cluster. It includes many of the most popular open-source observability tools such as Prometheus, Grafana, Promlens, TimescaleDB, and others. Together, these provide a maintainable solution to analyze the traffic on the server and identify any potential problems with a deployment. You can use Helm charts to configure and update TOBS deployments.

      TOBS includes the following components:

      • OpenTelemetry collector is deployed to collect traces.
      • Alertmanager, is deployed alongside Prometheus, forms the alerting layer of the stack, and handles alerts generated by Prometheus.
      • Grafana is a data visualization and analytics tool that allows you to build dashboards and graphs for your metrics data.
      • PromLens helps users build PromQL queries with ease. PromLens is a PromQL query builder that helps you build, understand, and fix your queries much more effectively.
      • TimescaleDB is for long-term storage of metric data. Long-term storage provides the ability to perform post-hoc analysis on metric data over long periods of time. Such data analysis can be used for capacity planning, identifying slow-moving regressions, trend analysis, auditing, and more. For information about connecting to the database from the cluster, see
        TimescaleDB Documentation
      • Promscale provides the translation layer between Prometheus and the database. It allows the Prometheus server to store and retrieve metrics from TimescaleDB, and allows users to use PromQL on Promscale and Prometheus.
      • Prometheus is an open-source systems monitoring and altering stack. It has become the de-facto standard in metric monitoring and is the basis of standards such as OpenMetrics. It allows you to monitor and understand how your infrastructure and applications are performing. Service discovery allows Prometheus to automagically discover components within your Kubernetes cluster that are already emitting metrics.
      • kube-state-metrics exports the metrics related to Kubernetes resources such as the status and count of Kubernetes resources, with visibility of the desired resources and the current resources, as well as the trends in your cluster.
      • Node-Exporter is deployed to export node related metrics such as CPU, memory usage, and others from the Kubernetes cluster.

      Before You Begin

      Note

      This guide was written using
      Kubernetes version 1.23.

      1. Deploy an LKE Cluster. This guide was written using an example node pool with three
        2 GB Linodes. Depending on the workloads you plan to deploy on your cluster, you may consider using Linodes with more available resources.

      2. Install
        Helm 3 to your local environment.

      3. Install
        kubectl to your local environment and
        connect to your cluster.

      4. Create the monitoring namespace on your LKE cluster:

        kubectl create namespace monitoring

      5. Add the stable Helm charts repository to your Helm repos:

        helm repo add stable https://charts.helm.sh/stable

      6. Update your Helm repositories:

        helm repo update

      TOBS Minimal Deployment

      In this section, learn to deploy TOBS for individual/local access with kubectl
      Port-Forward.

      Deploy The Observability Stack

      1. Install a certificate manager for your LKE cluster:

         kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.0/cert-manager.yaml

      2. Using Helm, deploy the TOBS release labeled lke-monitor in the monitoring namespace on your LKE cluster:

        helm repo add timescale https://charts.timescale.com/
helm repo update
helm install --wait lke-monitor timescale/tobs --namespace monitoring

      3. Verify that the Prometheus Operator has been deployed to your LKE cluster and its components are running and ready by checking the pods in the monitoring namespace:

        kubectl -n monitoring get pods

        You should see a similar output to the following:

        NAME                                                        READY   STATUS      RESTARTS      AGE
alertmanager-tobs-kube-prometheus-alertmanager-0            2/2     Running     0             2m13s
lke-monitor-connection-secret-j4sdh                         0/1     Completed   0             2m35s
lke-monitor-grafana-54d979dcf5-tkkgj                        3/3     Running     2 (65s ago)   2m32s
lke-monitor-grafana-db-swm8g                                0/1     Completed   3             2m35s
lke-monitor-kube-state-metrics-6bc5c44b9-g8r5g              1/1     Running     0             2m27s
lke-monitor-prometheus-node-exporter-b4vvg                  1/1     Running     0             2m33s
lke-monitor-prometheus-node-exporter-bbcnd                  1/1     Running     0             2m34s
lke-monitor-prometheus-node-exporter-frrfp                  1/1     Running     0             2m26s
lke-monitor-promlens-569cfbd586-bkhrr                       1/1     Running     0             2m34s
lke-monitor-promscale-86d574986c-9wj2z                      1/1     Running     4 (64s ago)   2m27s
lke-monitor-timescaledb-0                                   1/1     Running     0             2m30s
opentelemetry-operator-controller-manager-8cf5c85c8-krdj5   2/2     Running     0             2m27s
prometheus-tobs-kube-prometheus-prometheus-0                2/2     Running     0             2m13s
tobs-kube-prometheus-operator-5b4f674986-55r4k              1/1     Running     0             2m34s

      Access Monitoring Interfaces with Port-Forward

      1. List the services running in the monitoring namespace and review their respective ports:

        kubectl -n monitoring get svc

        You should see an output similar to the following:

        NAME                                                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                           AGE
alertmanager-operated                                       ClusterIP   None             <none>        9093/TCP,9094/TCP,9094/UDP   3m41s
lke-monitor                                                 ClusterIP   10.128.40.142    <none>        5432/TCP                     4m3s
lke-monitor-config                                          ClusterIP   None             <none>        8008/TCP                     4m3s
lke-monitor-grafana                                         ClusterIP   10.128.102.243   <none>        80/TCP                       4m3s
lke-monitor-kube-state-metrics                              ClusterIP   10.128.208.39    <none>        8080/TCP                     4m3s
lke-monitor-prometheus-node-exporter                        ClusterIP   10.128.170.88    <none>        9100/TCP                     4m3s
lke-monitor-promlens                                        ClusterIP   10.128.45.92     <none>        80/TCP                       4m3s
lke-monitor-promscale-connector                             ClusterIP   10.128.198.88    <none>        9201/TCP,9202/TCP            4m3s
lke-monitor-replica                                         ClusterIP   10.128.137.189   <none>        5432/TCP                     4m3s
opentelemetry-operator-controller-manager-metrics-service   ClusterIP   10.128.45.42     <none>        8443/TCP                     4m3s
opentelemetry-operator-webhook-service                      ClusterIP   10.128.12.89     <none>        443/TCP                      4m3s
prometheus-operated                                         ClusterIP   None             <none>        9090/TCP                     3m41s
tobs-kube-prometheus-alertmanager                           ClusterIP   10.128.33.44     <none>        9093/TCP                     4m3s
tobs-kube-prometheus-operator                               ClusterIP   10.128.175.39    <none>        443/TCP                      4m3s
tobs-kube-prometheus-prometheus                             ClusterIP   10.128.106.173   <none>        9090/TCP                     4m3s

        From the above output, the resource services you will access have the corresponding ports:

        ResourceService NamePort
        Prometheustobs-kube-prometheus-prometheus9090
        Alertmanagertobs-kube-prometheus-alertmanager9093
        Grafanalke-monitor-grafana80

      2. Use kubectl
        port-forward to open a connection to a service, then access the service’s interface by entering the corresponding address in your web browser:

        Note

        Press control+C on your keyboard to terminate a port-forward process after entering any of the following commands.

        • To provide access to the Prometheus interface at the address 127.0.0.1:9090 in your web browser, enter:

          kubectl -n monitoring \
port-forward \
svc/tobs-kube-prometheus-prometheus \
9090

        • To provide access to the Alertmanager interface at the address 127.0.0.1:9093 in your web browser, enter:

          kubectl -n monitoring \
port-forward \
svc/tobs-kube-prometheus-alertmanager  \
9093

        • To provide access to the Grafana interface at the address 127.0.0.1:8081 in your web browser, enter:

          kubectl -n monitoring \
port-forward \
svc/lke-monitor-grafana  \
8081:80

          When accessing the Grafana interface, log in as admin. You can get the password using:

          kubectl get secret --namespace monitoring lke-monitor-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

          The Grafana dashboards are accessible at Dashboards > Manage from the left navigation bar.

      TOBS eliminates the need to maintain configuration details for each of the applications, while providing standardized monitoring for the applications running on your cluster.

      More Information

      You may wish to consult the following resources for additional information
      on this topic. While these are provided in the hope that they will be
      useful, please note that we cannot vouch for the accuracy or timeliness of
      externally hosted materials.

      This page was originally published on

      Join the conversation.
      Read other comments or post your own below. Comments must be respectful,
      constructive, and relevant to the topic of the guide. Do not post external
      links or advertisements. Before posting, consider if your comment would be
      better addressed by contacting our
      Support team or asking on
      our
      Community Site.



      Source link

      Getting Started with Load Balancing on a Linode Kubernetes Engine (LKE) Cluster


      Updated by Linode Contributed by Linode

      The Linode Kubernetes Engine (LKE) is Linode’s managed Kubernetes service. When you deploy an LKE cluster, you receive a Kubernetes Master which runs your cluster’s control plane components, at no additional cost. The control plane includes Linode’s Cloud Controller Manager (CCM), which provides a way for your cluster to access additional Linode services. Linode’s CCM provides access to Linode’s load balancing service, Linode NodeBalancers.

      NodeBalancers provide your Kubernetes cluster with a reliable way of exposing resources to the public internet. The LKE control plane handles the creation and deletion of the NodeBalancer, and correctly identifies the resources, and their networking, that the NodeBalancer will route traffic to. Whenever a Kubernetes Service of the LoadBalancer type is created, your Kubernetes cluster will create a Linode NodeBalancer service with the help of the Linode CCM.

      Note

      Adding external Linode NodeBalancers to your LKE cluster will incur additional costs. See Linode’s Pricing page for details.

      Note

      All existing LKE clusters receive CCM updates automatically every two weeks when a new LKE release is deployed. See the LKE Changelog for information on the latest LKE release.

      Note

      The Linode Terraform K8s module also deploys a Kubernetes cluster with the Linode CCM installed by default. Any Kubernetes cluster with a Linode CCM installation can make use of Linode NodeBalancers in the ways described in this guide.

      In this Guide

      This guide will show you:

      Before You Begin

      This guide assumes you have a working Kubernetes cluster that was deployed using the Linode Kubernetes Engine (LKE). You can deploy a Kubernetes cluster using LKE in the following ways:

      Adding Linode NodeBalancers to your Kubernetes Cluster

      To add an external load balancer to your Kubernetes cluster you can add the example lines to a new configuration file, or more commonly, to a Service file. When the configuration is applied to your cluster, Linode NodeBalancers will be created, and added to your Kubernetes cluster. Your cluster will be accessible via a public IP address and the NodeBalancers will route external traffic to a Service running on healthy nodes in your cluster.

      Note

      Billing for Linode NodeBalancers begin as soon as the example configuration is successfully applied to your Kubernetes cluster.

      1
2
3
4
5
6
7

      spec:
  type: LoadBalancer
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
      • The spec.type of LoadBalancer is responsible for telling Kubernetes to create a Linode NodeBalancer.
      • The remaining lines provide port definitions for your Service’s Pods and maps an incoming port to a container’s targetPort.

      Viewing Linode NodeBalancer Details

      To view details about running NodeBalancers on your cluster:

      1. Get the services running on your cluster:

        kubectl get services

        You will see a similar output:

          
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
kubernetes      ClusterIP      10.128.0.1      none           443/TCP        3h5m
example-service LoadBalancer   10.128.171.88   45.79.246.55   80:30028/TCP   36m
        • Viewing the entry for the example-service, you can find your NodeBalancer’s public IP under the EXTERNAL-IP column.
        • The PORT(S) column displays the example-service incoming port and NodePort.

      2. View details about the example-service to retrieve information about the deployed NodeBalancers:

        kubectl describe service example-service

          
Name:                     nginx-service
Namespace:                default
Labels:                   app=nginx
Annotations:              service.beta.kubernetes.io/linode-loadbalancer-throttle: 4
Selector:                 app=nginx
Type:                     LoadBalancer
IP:                       10.128.171.88
LoadBalancer Ingress:     192.0.2.0
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  30028/TCP
Endpoints:                10.2.1.2:80,10.2.1.3:80,10.2.2.2:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:

      Configuring your Linode NodeBalancers with Annotations

      The Linode CCM accepts annotations that configure the behavior and settings of your cluster’s underlying NodeBalancers.

      • The table below provides a list of all available annotation suffixes.
      • Each annotation must be prefixed with service.beta.kubernetes.io/linode-loadbalancer-. For example, the complete value for the throttle annotation is service.beta.kubernetes.io/linode-loadbalancer-throttle.
      • Annotation values such as http are case-sensitive.

      Annotations Reference

      Annotation (suffix)ValuesDefault ValueDescription
      throttle• integer
      • 0–20
      • 0 disables the throttle      		20The client connection throttle limits the number of new connections-per-second from the same client IP.
      default-protocol• string
      •tcp, http, https      		tcpSpecifies the protocol for the NodeBalancer to use.
      port-*A JSON object of port configurations
      For example:
      { "tls-secret-name": "prod-app-tls", "protocol": "https"})      		None• Specifies a NodeBalancer port to configure, i.e. port-443.

      • Ports 1-65534 are available for balancing.

      • The available port configurations are:

      "tls-secret-name" use this key to provide a Kubernetes secret name when setting up TLS termination for a service to be accessed over HTTPS. The secret type should be kubernetes.io/tls.

      "protocol" specifies the protocol to use for this port, i.e. tcp, http, https. The default protocol is tcp, unless you provided a different configuration for the default-protocol annotation.

      check-type• string
      • none, connection, http, http_body      		None• The type of health check to perform on Nodes to ensure that they are serving requests. The behavior for each check is the following:

      none no check is performed

      connection checks for a valid TCP handshake

      http checks for a 2xx or 3xx response code

      http_body checks for a specific string within the response body of the healthcheck URL. Use the check-body annotation to provide the string to use for the check.

      check-pathstringNoneThe URL path that the NodeBalancer will use to check on the health of the back-end Nodes.
      check-bodystringNoneThe string that must be present in the response body of the URL path used for health checks. You must have a check-type annotation configured for a http_body check.
      check-intervalintegerNoneThe duration, in seconds, between health checks.
      check-timeout• integer
      • value between 1–30      		NoneDuration, in seconds, to wait for a health check to succeed before it is considered a failure.
      check-attempts• integer
      • value between 1–30      		NoneNumber of health checks to perform before removing a back-end Node from service.
      check-passivebooleanfalseWhen true, 5xx status codes will cause the health check to fail.
      preservebooleanfalseWhen true, deleting a LoadBalancer service does not delete the underlying NodeBalancer

      Note

      Configuring Linode NodeBalancers for TLS Encryption

      This section describes how to set up TLS termination on your Linode NodeBalancers so a Kubernetes Service can be accessed over HTTPS.

      Generating a TLS type Secret

      Kubernetes allows you to store sensitive information in a Secret object for use within your cluster. This is useful for storing things like passwords and API tokens. In this section, you will create a Kubernetes secret to store Transport Layer Security (TLS) certificates and keys that you will then use to configure TLS termination on your Linode NodeBalancers.

      In the context of the Linode CCM, Secrets are useful for storing Transport Layer Security (TLS) certificates and keys. The linode-loadbalancer-tls annotation requires TLS certificates and keys to be stored as Kubernetes Secrets with the type tls. Follow the steps in this section to create a Kubernetes TLS Secret.

      Note

      1. Generate a TLS key and certificate using a TLS toolkit like OpenSSL. Be sure to change the CN and O values to those of your own website domain.

        openssl req -newkey rsa:4096 
    -x509 
    -sha256 
    -days 3650 
    -nodes 
    -out tls.crt 
    -keyout tls.key 
    -subj "/CN=mywebsite.com/O=mywebsite.com"

      2. Create the secret using the create secret tls command. Ensure you substitute $SECRET_NAME for the name you’d like to give to your secret. This will be how you reference the secret in your Service manifest.

        kubectl create secret tls $SECRET_NAME --cert tls.crt --key tls.key

      3. You can check to make sure your Secret has been successfully stored by using describe:

        kubectl describe secret $SECRET_NAME

        You should see output like the following:

          
kubectl describe secret docteamdemosite
Name:         my-secret
Namespace:    default
Labels:       
Annotations:  

Type:  kubernetes.io/tls

Data
====
tls.crt:  1164 bytes
tls.key:  1704 bytes

        If your key is not formatted correctly you’ll receive an error stating that there is no PEM formatted data within the key file.

      Configuring TLS within a Service

      In order to use https you’ll need to instruct the Service to use the correct port using the required annotations. You can add the following code snippet to a Service file to enable TLS termination on your NodeBalancers:

      example-serivce.yaml
      1
2
3
4
5
6

      ...
metadata:
  annotations:
    service.beta.kubernetes.io/linode-loadbalancer-default-protocol: http
    service.beta.kubernetes.io/linode-loadbalancer-port-443: '{ "tls-secret-name": "example-secret", "protocol": "https" }'
...

      • The service.beta.kubernetes.io/linode-loadbalancer-default-protocol annotation configures the NodeBalancer’s default protocol.

      • service.beta.kubernetes.io/linode-loadbalancer-port-443 specifies port 443 as the port to be configured. The value of this annotation is a JSON object designating the TLS secret name to use (example-secret) and the protocol to use for the port being configured (https).

      If you have multiple Secrets and ports for different environments (testing, staging, etc.), you can define more than one secret and port pair:

      example-serivce.yaml
      1
2
3
4
5
6
7

      ...
metadata:
  annotations:
    service.beta.kubernetes.io/linode-loadbalancer-default-protocol: http
    service.beta.kubernetes.io/linode-loadbalancer-port-443: '{ "tls-secret-name": "example-secret", "protocol": "https" }'
    service.beta.kubernetes.io/linode-loadbalancer-port-8443: '{ "tls-secret-name": "example-secret-staging", "protocol": "https" }'
...

      Configuring Session Affinity for Cluster Pods

      kube-proxy will always attempt to proxy traffic to a random backend Pod. To direct traffic to the same Pod, you can use the sessionAffinity mechanism. When set to clientIP, sessionAffinity will ensure that all traffic from the same IP will be directed to the same Pod. You can add the example lines to a Service configuration file to

      1
2
3
4
5
6
7
8

      spec:
  type: LoadBalancer
  selector:
    app: example-app
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 100

      Removing Linode NodeBalancers from your Kubernetes Cluster

      To delete a NodeBalancer and the Service that it represents, you can use the Service manifest file you used to create the NodeBalancer. Simply use the delete command and supply your file name with the f flag:

      kubectl delete -f example-service.yaml

      Similarly, you can delete the Service by name:

      kubectl delete service example-service

      After deleting your service, its corresponding NodeBalancer will be removed from your Linode account.

      Note

      If your Service file used the preserve annotation, the underlying NodeBalancer will not be removed from your Linode account. See the annotations reference for details.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Forgot Your Password ?