One place for hosting & domains

      Redis

      How To Implement PHP Rate Limiting with Redis on Ubuntu 20.04


      The author selected the Apache Software Foundation to receive a donation as part of the Write for DOnations program.

      Introduction

      Redis (Remote Dictionary Server ) is an in-memory open source software. It is a data-structure store that uses a server’s RAM, which is several times faster than even the fastest Solid State Drive (SSD). This makes Redis highly responsive, and therefore, suitable for rate limiting.

      Rate limiting is a technology that puts a cap on the number of times a user can request a resource from a server. Many services implement rate limiting to prevent abuse to a service when a user may try to put too much load on a server.

      For instance, when you’re implementing a public API (Application Programming Interface) for your web application with PHP, you need some form of rate limiting. The reason is that when you release an API to the public, you’d want to put a control on the number of times an application user can repeat an action in a specific timeframe. Without any control, users may bring your system to a complete halt.

      Rejecting users’ requests that exceed a certain limit allows your application to run smoothly. If you have a lot of customers, rate limiting enforces a fair-usage policy that allows each customer to have high-speed access to your application. Rate limiting is also good for reducing bandwidth costs and minimizing congestion on your server.

      It might be practical to code a rate-limiting module by logging user activities in a database like MySQL. However, the end product may not be scalable when many users access the system since the data must be fetched from disk and compared against the set limit. This is not only slow, but relational database management systems are not designed for this purpose.

      Since Redis works as an in-memory database, it is a qualified candidate for creating a rate limiter, and it has been proven reliable for this purpose.

      In this tutorial, you’ll implement a PHP script for rate limiting with Redis on an Ubuntu 20.04 server.

      Prerequisites

      Before you begin, you’ll need the following:

      Step 1 — Installing the Redis Library for PHP

      First, you’ll begin by updating your Ubuntu server package repository index. Then, install the php-redis extension. This is a library that allows you to implement Redis in your PHP code. To do this, run the following commands:

      • sudo apt update
      • sudo apt install -y php-redis

      Next, restart the Apache server to load the php-redis library:

      • sudo systemctl restart apache2

      Once you’ve updated your software information index and installed the Redis library for PHP, you’ll now create a PHP resource that caps users’ access based on their IP address.

      Step 2 — Building a PHP Web Resource for Rate Limiting

      In this step, you’ll create a test.php file in the root directory (/var/www/html/) of your web server. This file will be accessible to the public and users can type its address in a web browser to run it. However, for the basis of this guide, you’ll later test access to the resource using the curl command.

      The sample resource file allows users to access it three times in a timeframe of 10 seconds. Users trying to exceed the limit will get an error informing them that they have been rate limited.

      The core functionality of this file relies heavily on the Redis server. When a user requests the resource for the first time, the PHP code in the file will create a key on the Redis server based on the user’s IP address.

      When the user visits the resource again, the PHP code will try to match the user’s IP address with the keys stored in the Redis server and increment the value by one if the key exists. The PHP code will keep checking if the incremented value hits the maximum limit set.

      The Redis key, which is based on the user’s IP address, will expire after 10 seconds; after this time period, logging the user’s visits to the web resource will begin again.

      To begin, open the /var/www/html/test.php file:

      • sudo nano /var/www/html/test.php

      Next, enter the following information to initialize the Redis class. Remember to enter the appropriate value for REDIS_PASSWORD:

      /var/www/html/test.php

      <?php
      
      $redis = new Redis();
      $redis->connect('127.0.0.1', 6379);
      $redis->auth('REDIS_PASSWORD');
      

      $redis->auth implements plain text authentication to the Redis server. This is OK while you’re working locally (via localhost), but if you’re using a remote Redis server, consider using SSL authentication.

      Next, in the same file, initialize the following variables:

      /var/www/html/test.php

      . . .
      $max_calls_limit  = 3;
      $time_period      = 10;
      $total_user_calls = 0;
      

      You’ve defined:

      • $max_calls_limit: is the maximum number of calls a user can access the resource.
      • $time_period: defines the timeframe in seconds within which a user is allowed to access the resource per the $max_calls_limit.
      • $total_user_calls: initializes a variable that retrieves the number of times a user has requested access to the resource in the given timeframe.

      Next, add the following code to retrieve the IP address of the user requesting the web resource:

      /var/www/html/test.php

      . . .
      if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
          $user_ip_address = $_SERVER['HTTP_CLIENT_IP'];
      } elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
          $user_ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
      } else {
          $user_ip_address = $_SERVER['REMOTE_ADDR'];
      }
      

      While this code uses the users’ IP address for demonstration purposes, if you’ve got a protected resource on the server that requires authentication, you might log users’ activities using their usernames or access tokens.

      In such a scenario, every user authenticated into your system will have a unique identifier (for example, a customer ID, developer ID, vendor ID, or even a user ID). (If you configure this, remember to use these identifiers in place of the $user_ip_address.)

      For this guide, the user IP address is sufficient for proving the concept. So, once you’ve retrieved the user’s IP address in the previous code snippet, add the next code block to your file:

      /var/www/html/test.php

      . . .
      if (!$redis->exists($user_ip_address)) {
          $redis->set($user_ip_address, 1);
          $redis->expire($user_ip_address, $time_period);
          $total_user_calls = 1;
      } else {
          $redis->INCR($user_ip_address);
          $total_user_calls = $redis->get($user_ip_address);
          if ($total_user_calls > $max_calls_limit) {
              echo "User " . $user_ip_address . " limit exceeded.";
              exit();
          }
      }
      
      echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds";
      

      In this code, you use an if...else statement to check if there is a key defined with the IP address on the Redis server. If the key doesn’t exist, if (!$redis->exists($user_ip_address)) {...}, you set it and define its value to 1 using the code $redis->set($user_ip_address, 1);.

      The $redis->expire($user_ip_address, $time_period); sets the key to expire within the time period—in this case, 10 seconds.

      If the user’s IP address does not exist as a Redis key, you set the variable $total_user_calls to 1.

      In the ...else {...}... statement block, you use the $redis->INCR($user_ip_address); command to increment the value of the Redis key set for each IP address key by 1. This only happens when the key is already set in the Redis server and counts as a repeat request.

      The statement $total_user_calls = $redis->get($user_ip_address); retrieves the total requests the user makes by checking their IP address-based key on the Redis server.

      Toward the end of the file, you use the ...if ($total_user_calls > $max_calls_limit) {... }.. statement to check if the limit is exceeded; if so, you alert the user with echo "User " . $user_ip_address . " limit exceeded.";. Finally, you’re informing the user about the visits they make in the time period using the echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds"; statement.

      After adding all the code, your /var/www/html/test.php file will be as follows:

      /var/www/html/test.php

      <?php
      $redis = new Redis();
      $redis->connect('127.0.0.1', 6379);
      $redis->auth('REDIS_PASSWORD');
      
      $max_calls_limit  = 3;
      $time_period      = 10;
      $total_user_calls = 0;
      
      if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
          $user_ip_address = $_SERVER['HTTP_CLIENT_IP'];
      } elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
          $user_ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
      } else {
          $user_ip_address = $_SERVER['REMOTE_ADDR'];
      }
      
      if (!$redis->exists($user_ip_address)) {
          $redis->set($user_ip_address, 1);
          $redis->expire($user_ip_address, $time_period);
          $total_user_calls = 1;
      } else {
          $redis->INCR($user_ip_address);
          $total_user_calls = $redis->get($user_ip_address);
          if ($total_user_calls > $max_calls_limit) {
              echo "User " . $user_ip_address . " limit exceeded.";
              exit();
          }
      }
      
      echo "Welcome " . $user_ip_address . " total calls made " . $total_user_calls . " in " . $time_period . " seconds";
      

      When you’ve finished editing the /var/www/html/test.php file, save and close it.

      You’ve now coded the logic needed to rate limit users on the test.php web resource. In the next step, you’ll test your script.

      Step 3 — Testing Redis Rate Limiting

      In this step, you’ll use the curl command to request the web resource that you’ve coded in Step 2. To fully check the script, you’ll request the resource five times in a single command. It is possible to do this by including a placeholder URL parameter at the end of the test.php file. Here, you use the value ?[1-5] at the end of your request to execute the curl commands five times.

      Run the following command:

      • curl -H "Accept: text/plain" -H "Content-Type: text/plain" -X GET http://localhost/test.php?[1-5]

      After running the code, you will receive output similar to the following:

      Output

      [1/5]: http://localhost/test.php?1 --> <stdout> --_curl_--http://localhost/test.php?1 Welcome 127.0.0.1 total calls made 1 in 10 seconds [2/5]: http://localhost/test.php?2 --> <stdout> --_curl_--http://localhost/test.php?2 Welcome 127.0.0.1 total calls made 2 in 10 seconds [3/5]: http://localhost/test.php?3 --> <stdout> --_curl_--http://localhost/test.php?3 Welcome 127.0.0.1 total calls made 3 in 10 seconds [4/5]: http://localhost/test.php?4 --> <stdout> --_curl_--http://localhost/test.php?4 User 127.0.0.1 limit exceeded. [5/5]: http://localhost/test.php?5 --> <stdout> --_curl_--http://localhost/test.php?5 User 127.0.0.1 limit exceeded.

      As you’ll note, the first three requests ran without a problem. However, your script has rate limited the fourth and fifth requests. This confirms that the Redis server is rate limiting users’ requests.

      In this guide, you’ve set low values for the two variables following:

      /var/www/html/test.php

      ...
      $max_calls_limit  = 3;
      $time_period      = 10;
      ...
      

      When designing your application in a production environment, you could consider higher values depending on how often you expect users to hit your application.

      It is best practice to check real-time stats before setting these values. For instance, if your server logs show that an average user hits your application 1,000 times every 60 seconds, you may use those values as a benchmark for throttling users.

      To put things in a better perspective, here are some real-world examples of rate-limiting implementations (as of 2021):

      Conclusion

      This tutorial implemented a PHP script for rate limiting with Redis on an Ubuntu 20.04 server to prevent your web application from inadvertent or malicious overuse. You could extend the code to further suit your needs depending on your use case.

      You might want to secure your Apache server for production use; follow the How To Secure Apache with Let’s Encrypt on Ubuntu 20.04 tutorial.

      You might also consider reading how Redis works as a database cache. Try out our How To Set Up Redis as a Cache for MySQL with PHP on Ubuntu 20.04 tutorial.

      You can find further resources on our PHP and Redis topic pages.



      Source link

      What is Redis?


      Redis is an open-source key-value data store known for its flexibility, performance, and wide language support. An in-memory database, Redis is also known for its speed and has found wide use as a cache and message broker, as well as a database.

      A NoSQL database, Redis doesn’t use structured query language, otherwise known as SQL. Redis instead comes with its own set of commands for managing and accessing data.

      To learn more about Redis, please visit:

      A complete list of our educational resources on Redis can be found on our Redis tag page.



      Source link

      How To Set Up Redis as a Cache for MySQL with PHP on Ubuntu 20.04


      The author selected Girls Who Code to receive a donation as part of the Write for DOnations program.

      Introduction

      Redis (Remote Dictionary Server) is a fast open-source, in-memory database that you can use as a key-value store for a highly scalable and performance-oriented system. Some of Redis’ use cases include: caching, high-speed transactions, real-time analytics, live notifications, machine learning, searching, and queue/job processing. Since Redis is an in-memory key-value store, its performance makes it suitable for caching data in your application.

      Caching is storing data temporarily in a high-speed storage layer (for example, in a computer RAM) to serve data faster when clients make the same future requests. This enhances the re-use of previously computed data instead of fetching it each time from the disk.

      When you’re working with PHP and MySQL, using Redis as a cache improves your application performance because Redis stores data in RAM, which is several times faster than a hard disk (HDD) or a solid-state drive (SSD). Caching also reduces database costs—that is, the number of round trips made to the back-end database—and avoids overloading the backend.

      Caching data is an integral design feature when you’re designing web applications with higher reads than writes. Such applications include blogs, online stores, and social media sites.

      In this tutorial, you’ll use Redis to cache MySQL data with PHP on Ubuntu 20.04.

      Prerequisites

      To complete this tutorial, you’ll need the following:

      Step 1 — Installing the Redis Library for PHP

      To begin you’ll install the php-redis extension, which will allow you to use PHP to communicate with Redis. Run the following commands to update your server and install the extension:

      • sudo apt update
      • sudo apt install php-redis

      Confirm the installation and restart the Apache web server to load the extension:

      • sudo systemctl restart apache2

      Now that you have installed your dependencies, you’ll set up your database.

      Step 2 — Setting Up a Test Database, Table, and Sample Data

      In this step, you’ll create a MySQL database to store data permanently to disk. You’ll also create some tables and a user account with full privileges to the database.

      First, log in to your MySQL server as a root user:

      Enter the root password of your MySQL server that you set up in the LAMP prerequisite. Then, press ENTER to continue.

      Next, create a test_store database with the following command:

      • CREATE database test_store;

      Make sure the action is successful by confirming the output:

      Output

      Query OK, 1 row affected (0.00 sec)

      Next, create a user for your database. We’ll call this user test_user in this tutorial. Replace PASSWORD with a strong password as well:

      • CREATE USER 'test_user'@'localhost' IDENTIFIED WITH mysql_native_password BY 'PASSWORD';

      Then grant test_user full privileges to the test_store database with:

      • GRANT ALL PRIVILEGES ON test_store.* TO 'test_user'@'localhost';

      Finally run the following command to reload the grant tables in MySQL:

      Ensure you get the following output after each successful command:

      Output

      Query OK, 0 rows affected (0.01 sec)

      End the MySQL root session:

      You’ll receive the word Bye and the system will take you back to the server’s command line interface.

      Log back in to the MySQL server with the credentials for the test_user that you just created:

      Enter the password for the test_user to proceed. Then, switch to the test_store database when you’re in the mysql> prompt:

      Ensure you receive the following output:

      Output

      Database Changed.

      Next, you’ll create a products table with three columns. You’ll use the product_id column to uniquely identify each product. To avoid assigning the IDs manually, you’ll use the AUTO_INCREMENT keyword. Then, you’ll use the BIGINT data type for the product_id column to support a large data set. The BIGINT data type can hold a minimum value of -2^63 and a maximum value of 2^63 - 1.

      The product_name field will hold the actual names of your items. In this case, a VARCHAR data type with a length of 50 characters will be enough. The last column in the products table is the price—you’ll use the DOUBLE data type to accommodate prices with decimals (for example, 16.33).

      To create the products table, run the following command:

      • CREATE table products
      • (
      • product_id BIGINT PRIMARY KEY AUTO_INCREMENT,
      • product_name VARCHAR(50),
      • price DOUBLE
      • ) Engine = InnoDB;

      You will receive the following output:

      Output

      Query OK, 0 rows affected (0.01 sec)

      Now you’ll populate the products table with some records for testing purposes.

      You don’t need to enter data to the product_id column manually since the AUTO_INCREMENT column will complete this. Run the following commands one by one:

      • INSERT INTO products(product_name, price) VALUES ('Virtual Private Servers', '5.00');
      • INSERT INTO products(product_name, price) VALUES ('Managed Databases', '15.00');
      • INSERT INTO products(product_name, price) VALUES ('Block Storage', '10.00');
      • INSERT INTO products(product_name, price) VALUES ('Managed Kubernetes', '60.00');
      • INSERT INTO products(product_name, price) VALUES ('Load Balancer', '10.00');

      After running each command, ensure you get this output:

      Output

      Query OK, 1 row affected (0.00 sec)

      Verify the data using the SELECT command:

      You will receive output similar to the following:

      Output

      +------------+-------------------------+-------+ | product_id | product_name | price | +------------+-------------------------+-------+ | 1 | Virtual Private Servers | 5 | | 2 | Managed Databases | 15 | | 3 | Block Storage | 10 | | 4 | Managed Kubernetes | 60 | | 5 | Load Balancer | 10 | +------------+-------------------------+-------+ 5 rows in set (0.00 sec)

      End the MySQL session for the test_user:

      Once you’ve set up the test_store database, products table, and test_user, you’ll code a PHP script to retrieve data from the MySQL database and cache it to Redis.

      Step 3 — Designing a PHP Script for Fetching and Caching MySQL Data

      In this step, you’ll create a PHP script for retrieving the sample data that you’ve created in the previous step.

      When you run the script for the first time, it will read the data from MySQL (that is, from disk) and then cache it to Redis. As a result subsequent reads of the products’ data will be from Redis (that is, from system RAM). System memory is multiple times faster than even the fastest solid-state drive, thus data will be retrieved faster from the Redis cache than reading from the system disk.

      Note: While you might not get any performance boost, since you are retrieving just a few records from the MySQL database, several benchmarks prove that retrieving cached data from Redis is several times faster than reading it from MySQL when dealing with several hundred thousand records.

      Create a products.php file in the root directory of your website:

      • sudo nano /var/www/html/products.php

      To start, enter the following information to connect and create an instance of Redis and store it as an object in a $redis variable.

      The address 127.0.0.1 connects to the localhost. You may change this value if you’re running Redis from a remote server. Remember to replace REDIS_PASSWORD with the specific password for Redis set in the /etc/redis/redis.conf configuration file.

      Also, enter the appropriate port number. By default, Redis runs on port 6379:

      /var/www/html/products.php

      <?php
      
      $redis = new Redis();
      $redis->connect('127.0.0.1', 6379);
      $redis->auth('REDIS_PASSWORD');
      

      Note: In this guide, the $redis->auth('REDIS_PASSWORD') command sends your password to Redis in plain text. In a production environment, you may consider securing end-to-end communication between Redis and the client server running PHP code with a more powerful access control layer, such as TLS (Transport Layer Security). Also, when configuring your Redis password in the /etc/redis/redis.conf file, make sure you set a long and strong value to prevent brute-force attacks.

      The next step is initializing a PHP variable you’ll use as a key in Redis.

      As mentioned earlier in this guide, Redis acts as a key-value database and therefore you must have a unique key for the data that you intend to store and retrieve from it.

      So, define a PRODUCTS key by adding the following information to the /var/www/html/products.php file. You are free to use any name in place of PRODUCTS key.

      Your PHP script will use this key to cache information to Redis once data gets retrieved from the MySQL database:

      /var/www/html/products.php

      ...
      $key = 'PRODUCTS';
      

      Next, include a conditional PHP if...else statement to check if the PRODUCTS key exists in Redis:

      /var/www/html/products.php

      ...
      if (!$redis->get($key)) {
          $source="MySQL Server";
          $database_name="test_store";
          $database_user="test_user";
          $database_password = 'PASSWORD';
          $mysql_host="localhost";
      
          $pdo = new PDO('mysql:host=" . $mysql_host . "; dbname=" . $database_name, $database_user, $database_password);
          $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
      
          $sql  = "SELECT * FROM products";
          $stmt = $pdo->prepare($sql);
          $stmt->execute();
      
          while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
             $products[] = $row;
          }
      
          $redis->set($key, serialize($products));
          $redis->expire($key, 10);
      
      } else {
           $source = "Redis Server';
           $products = unserialize($redis->get($key));
      
      }
      
      echo $source . ': <br>';
      print_r($products);
      

      If the key doesn’t exist in Redis, the script connects to the database that you created earlier, queries the products table, and stores the data in Redis using the $redis->set($key, serialize($products)) command.

      The $redis->expire($key, 10); command sets the expiration to 10 seconds. You may tweak this value depending on your cache policy.

      The $source variable helps you to identify the source of the data once it is echoed as an array at the end of the script using the echo $source and print_r($products) commands.

      Once you’ve put everything together, your /var/www/html/products.php file will be as follows:

      /var/www/html/products.php

      <?php
      
      $redis = new Redis();
      $redis->connect('127.0.0.1', 6379);
      $redis->auth('REDIS_PASSWORD');
      
      $key = 'PRODUCTS';
      
      if (!$redis->get($key)) {
          $source="MySQL Server";
          $database_name="test_store";
          $database_user="test_user";
          $database_password = 'PASSWORD';
          $mysql_host="localhost";
      
          $pdo = new PDO('mysql:host=" . $mysql_host . "; dbname=" . $database_name, $database_user, $database_password);
          $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
      
          $sql  = "SELECT * FROM products";
          $stmt = $pdo->prepare($sql);
          $stmt->execute();
      
          while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
             $products[] = $row;
          }
      
          $redis->set($key, serialize($products));
          $redis->expire($key, 10);
      
      } else {
           $source = "Redis Server';
           $products = unserialize($redis->get($key));
      
      }
      
      echo $source . ': <br>';
      print_r($products);
      
      

      Save and close the file.

      You’ve now set up a PHP script that will connect to MySQL and cache data to Redis. You’ll test your script in the next step.

      Step 4 — Testing the PHP Script

      To test if Redis is caching data from the MySQL database, you’ll enter the path of the PHP script in a browser window. Remember to replace your_server_IP with the public IP address of your server, like so: http://your_server_IP/products.php.

      When you run the script for the first time, you will receive the following output that displays data from the MySQL database because, at this point, the PHP script has not yet cached any data in Redis:

      MySQL Server
      Array ( [0] => Array ( [product_id] => 1 [product_name] => Virtual Private Servers [price] => 5 ) [1] => Array ( [product_id] => 2 [product_name] => Managed Databases [price] => 15 ) [2] => Array ( [product_id] => 3 [product_name] => Block Storage [price] => 10 ) [3] => Array ( [product_id] => 4 [product_name] => Managed Kubernetes [price] => 60 ) [4] => Array ( [product_id] => 5 [product_name] => Load Balancer [price] => 10 ) )
      

      Once you run the script again, you’ll get an output confirming that it’s reading data from Redis, which is acting as a cache for MySQL.

      Redis Server
      Array ( [0] => Array ( [product_id] => 1 [product_name] => Virtual Private Servers [price] => 5 ) [1] => Array ( [product_id] => 2 [product_name] => Managed Databases [price] => 15 ) [2] => Array ( [product_id] => 3 [product_name] => Block Storage [price] => 10 ) [3] => Array ( [product_id] => 4 [product_name] => Managed Kubernetes [price] => 60 ) [4] => Array ( [product_id] => 5 [product_name] => Load Balancer [price] => 10 ) )
      

      Remember that the key will expire after 10 seconds and data will again be retrieved from MySQL.

      Conclusion

      In this guide, you’ve used Redis to cache MySQL data with PHP on Ubuntu 20.04. You may use the coding in this guide to set up a caching mechanism for your MySQL data, which is especially useful for high-traffic web applications.

      You can check out our Redis topic page for more educational resources. Or, learn more about coding in PHP with further tutorials and content on the PHP topic page.



      Source link