One place for hosting & domains


      How To Generate Pages from Markdown in Gatsby

      The author selected the Internet Archive to receive a donation as part of the Write for DOnations program.


      One of the key features of the popular static site generator Gatsby is its flexibility in consuming content sources. Gatsby can generate static pages from almost any data source, such as a Content Management System (CMS), API, database, or even a local file system.

      Markdown files are a popular file-based source to use with Gatsby. Markdown is a markup language that is lightweight and designed for readability, which makes it ideal for documentation projects, blog posts, or any text-heavy websites.

      In this tutorial, you will create a Gatsby-powered static site that builds itself from local Markdown source files, using the gatsby-source-filesystem plugin to collect the files and the gatsby-transformer-remark plugin to convert them into HTML.


      Before starting, here are a few things you will need:

      • A local installation of Node.js for running Gatsby and building your site. The installation procedure varies by operating system, but DigitalOcean has guides for Ubuntu 20.04 and macOS, and you can always find the latest release on the official Node.js download page.
      • Some familiarity with JavaScript, for working in Gatsby. The JavaScript language is an expansive topic, but a good starting spot is our How to Code in JavaScript series.
      • Some familiarity with React and JSX, as well as HTML elements, if you want to customize the user interface (UI) of your posts beyond what is covered in this tutorial.
      • A new Gatsby project named markdown-tutorial, scaffolded from gatsby-starter-default. To build a new Gatsby project from scratch, you can refer to Step 1 of the How To Set Up Your First Gatsby Website tutorial.

      This tutorial was tested on Node.js v14.16.1, npm v6.14.12, Gatsby v3.10.2, gatsby-source-filesystem v3.10.0, and gatsby-transformer-remark v4.7.0.

      Step 1 — Standardizing Your Markdown Content

      Given Markdown’s flexibility as a markup language, documents written in Markdown can take on various forms and levels of complexity. Gatsby does not require you to format all your Markdown files in the same way, but to make your site and URLs easier to organize, you will first standardize how you write and store your Markdown files within this project.

      For this tutorial, you will add blog posts to your site, and each post will have a title, customized URL, publication date, and post body. You will start with a single post, Learning About Gatsby, where you can share your experience of learning about Gatsby with the world.

      In preparation for storing this and future posts, create a new directory under src, named blog. If you prefer to do this from the command line, you can do so with mkdir from the root directory of the project:

      Next, create an empty Markdown file within that new directory. For the purposes of this tutorial, filenames do not control the published URL or SEO, so the naming conventions here are just to keep things organized. Give your new post a filename of

      You can either do this in your IDE, or use the terminal with touch:

      • touch src/blog/

      Finally, fill in the actual content of the post. Open the file in your editor of choice and add the following:


      title: "Learning about Gatsby"
      slug: "/blog/learning-about-gatsby"
      date: "2021-08-01"
      ## What I'm Working On
      Right now, I'm working through a [DigitalOcean tutorial]( on using [Gatsby]( with Markdown.

      The top section enclosed by --- is called frontmatter and consists of key-value pairs. It is written in YAML, which is a configuration language outside of Markdown. Property keys go on the left, and your values go on the right. In your frontmatter block, you have defined values for your post’s title, the publication date, and the slug. The slug is the customized portion of the post’s URL that comes after your domain name (also known as the path).

      Any text below the frontmatter will become the post body. Within that area, you have a heading of What I'm Working On, with two hash symbols (##) at the beginning of the line. In Markdown, this indicates a level-2 heading, which Gatsby will convert into <h2></h2> in HTML. Within the text below that heading, you have some links that are written using Markdown syntax: [link_text](link_destination).

      Save your changes to and then close the file.

      You have just created your first post using Markdown and saved it within your project’s source code. You have also standardized the format, including specific property values in the frontmatter of the blog posts. This keeps your source code organized and will be important for later steps. For now, the next step is to install and configure the plugins that are required for Gatsby to process this new Markdown post.

      Step 2 — Installing and Configuring the Required Plugins

      With your Markdown file in place, it is time to tell Gatsby where to find them and process them. In this step, you will install the plugins that are required to achieve this and update the Gatsby configuration to load them.

      Two plugins are required to process Markdown in Gatsby: gatsby-transformer-remark and gatsby-source-filesystem. gatsby-transformer-remark will parse your Markdown and convert your frontmatter into fields that Gatsby can query, and gatsby-source-filesystem will allow you to bring data from your local filesystem into your application.

      Install both at the same time by running the following command in your Gatsby project directory:

      • npm install gatsby-source-filesystem gatsby-transformer-remark

      After you install both plugins, open up the main Gatsby configuration file that lives in the root of your project folder: gatsby-config.js. By editing this file, you are telling Gatsby how to use the newly installed plugins to read your Markdown files and begin processing them. Add the following code to your config file:


      module.exports = {
        plugins: [
            resolve: `gatsby-source-filesystem`,
            options: {
              name: `images`,
              path: `${__dirname}/src/images`,
            resolve: `gatsby-source-filesystem`,
            options: {
              name: `blog`,
              path: `${__dirname}/src/blog`,

      The first block loads the gatsby-source-filesystem plugin and passes it an options object telling it to scan the src/blog directory for files and use the blog name as a label for the collection. The final line loads the gatsby-transformer-remark plugin with all default options, since you don’t need to customize them for this tutorial.

      You have now configured Gatsby to load the two plugins that are necessary for scanning in and parsing your Markdown files. In the next step, you will create a page template file for Gatsby to combine with your Markdown content and render as web pages.

      Step 3 — Creating a Page Template File

      Gatsby will now scan your Markdown files and process them with gatsby-transformer-remark, but it still needs instructions on how to display them in the browser. This step covers how to give it those instructions by adding a page template file (also known as a page template component).

      First, create an empty file in src/pages with a filename of {MarkdownRemark.frontmatter__slug}.js. The filename needs to exactly match this, because it uses a Gatsby API called the File System Route API, in which filenames dictate the routes (URLs) created on your site.

      Note: The File System Route API is a newer feature in Gatsby, and you might still see other Markdown page tutorials that involve the createPages API in gatsby-node.js instead. Although it is not deprecated, that API is no longer necessary for the use case covered by this tutorial. However, if you have a project that involves creating arbitrary pages that don’t mirror the file system or exceeds the capabilities of the File System Route API in some other way, you might still need to use the createPages approach.

      To create your component, add the following code to the file:


      import { graphql } from "gatsby";
      import * as React from "react";
      import Layout from "../components/layout";
      import Seo from "../components/seo";
      export default function BlogPostTemplate({ data: { markdownRemark } }) {
        const { frontmatter, html } = markdownRemark;
        return (
            <Seo title={frontmatter.title} />
            <div className="post-body" dangerouslySetInnerHTML={{ __html: html }} />
      export const pageQuery = graphql`
        query ($id: String!) {
          markdownRemark(id: { eq: $id }) {
            frontmatter {
              date(formatString: "MMMM DD, YYYY")

      This code has two main sections. The one at the end of the file is pageQuery, which uses the Gatsby GraphQL tag to evaluate the GraphQL query that follows it. The results of that query are passed to the BlogPostTemplate function and give you access to properties of the post fetched over GraphQL.

      The BlogPostTemplate function is a React component that returns JSX. Within the BlogPostTemplate you display the values of each post’s standardized frontmatter fields, the title and the date, in header elements. The actual body of the post you have placed into a <div> with a class of post-body, using React’s dangerouslySetInnerHTML property to directly echo out the HTML into the rendered result.

      Finally, using export default before declaring the BlogPostTemplate function is necessary in your code, because Gatsby will expect the default export of each page template file to be the React component responsible for producing the final rendered page.

      Now that you have added the template code to the file, save the changes and close it.

      In completing this step, you added a brand new page template file to your Gatsby project. With a filename that uses the File System Route API, it dynamically creates routes and pages based on GraphQL results and your Markdown source files. This was the final step to get Gatsby to generate pages from Markdown. In the next step, you will see these pages in action, as well as add new ones.

      With all the code now in place to turn your Markdown blog posts into web pages, you can now preview your work and add more Markdown content to your site.

      To preview your new blog posts and all the work you have done so far, run this command in your project directory:

      Once Gatsby is ready, it will prompt you to open your project in your web browser, which you can do by navigating to localhost:8000. However, this will first only show you the homepage of your Gatsby application instead of the blog post. To see your new Markdown blog post, navigate to localhost:8000/blog/learning-about-gatsby/. You will find your Markdown file rendered as HTML:

      Blog post titled

      From now on, you can continue to add new blog posts by creating Markdown files in the src/blog directory. Each time you create a new post, remember to follow the conventions you set up for your frontmatter. This means making sure to start the slug value with /blog/ followed by the custom path you want, as well as providing the post with a title and date.

      To test this out, copy your Markdown file to be the base of a new post named

      • cp src/blog/ src/blog/

      Next, open the new file and make the following highlighted changes to the content:


      title: "Continuing to Learn"
      slug: "/blog/continuing-learning"
      date: "2021-08-01" 
      ## Update
      I'm continuing to learn Gatsby to build some fast static sites!

      In this file, you kept the formatting the same but changed the title and slug of the frontmatter and the content of the blog post. Save the file then exit from it.

      Once your server has rebuilt your Gatsby site, navigate to http://localhost:8000/blog/continuing-learning in your browser. You will find the new post rendered at this URL:

      Blog post titled

      Note: If you wanted to add Markdown pages outside of blog, you could do so by modifying the slug to whatever path you would like. If you do this, make sure to use folders to keep your files organized.

      You now have new pages generated from Markdown in Gatsby and have previewed the results.


      By following the steps in this tutorial, you added Markdown content to your Gatsby project, configured Gatsby to find the files and process them, and created template code to render each post as a new page. Although some of these steps could be omitted by using a Markdown-specific Gatsby Starter template, going through the process manually allows you to customize your Markdown-powered Gatsby site exactly how you want it.

      Although you can link to any new post or page using the Gatsby Link component, if your site has a large amount of Markdown files that are rapidly changing you might want to explore adding a dynamic listing as a next step, so visitors to your site can quickly find all your most recent posts and navigate to each one. To do so, you could use a Gatsby GraphQL tag to query the Markdown posts you want to list, and then iterate over them and display them as links. You can read more about this in this Adding a List of Markdown Blog Posts tutorial.

      If you would like to read more on Gatsby, check out the rest of the How To Create Static Web Sites with Gatsby.js series.

      Source link

      How To Use Python-Markdown to Convert Markdown Text to HTML

      The author selected the COVID-19 Relief Fund to receive a donation as part of the Write for DOnations program.


      Markdown is a markup language commonly used to simplify the process of writing content in an easy-to-read text format, which a software tool or programming library can convert into HTML to display in a browser or another writing program. Because it uses plain-text syntax, Markdown is compatible with any text editor and can convert headings, lists, links, and other components. Bloggers, tutorial authors, and documentation writers use Markdown widely and websites, such as Github, StackOverflow, and The Python Package Index (PyPI), support it.

      You can learn how to use Markdown from the Markdown syntax standard. Alternatively, you can also try a different Markdown implementation in a web editor, like the DigitalOcean Markdown Preview or the StackEdit editor.

      Python-Markdown is a Python library that allows you to convert Markdown text to HTML in various ways. You can extend its functionality using its different extensions that provide additional features. Note however, that the Python-Markdown has a few minor differences with the standard Markdown syntax.

      In this tutorial, you will install the Python-Markdown library, use it to convert Markdown strings to HTML, convert Markdown files to HTML files, and use the Python-Markdown command line interface to convert Markdown to HTML.


      Before you start following this guide, you will need:

      Step 1 — Installing Python-Markdown

      In this step, you will install Python-Markdown and explore one of its functions to convert Markdown strings to HTML in the Python REPL.

      If you haven’t already activated your programming environment, make sure you’re in your project directory (pymark) and use the following command to activate the environment:

      Once you have activated your programming environment, your prompt will now have an env prefix like the following:

      Now you’ll install Python packages and isolate your project code away from the main Python system installation.

      Use pip to install the Python-Markdown library (markdown) by running the following command:

      Once the installation finishes successfully, you can experiment with it in the Python REPL, which you can open by typing the following command:

      You will notice a new prompt with the prefix >>>; you can use this to type in Python code and receive immediate output.

      First you will import the markdown package and use it to convert a piece of Markdown text from Markdown syntax to HTML:

      • import markdown
      • markdown.markdown('#Hi')

      In this code, you import the markdown package you installed earlier. You use the markdown.markdown() function to convert the Markdown text #Hi (with # representing an H1-level header) to its HTML equivalent. If you type the code into the Python REPL, you will receive the following output:



      The HTML output is the equivalent of the #Hi Markdown text.

      You can use triple single quotes (''') to type multi-line Markdown text into the Python REPL like so:

      • import markdown
      • output = markdown.markdown('''
      • # Step 1
      • ## Step 2
      • * item 1
      • * item 2
      • Visit [the tutorials page]( for more tutorials!
      • ''')
      • print(output)

      In this example, you pass an H1 header, an H2 header, two list items, and a paragraph containing a link. You then save the output in a variable called output and print it with the print() Python function.

      You will receive the following output:


      <h1>Step 1</h1> <h2>Step 2</h2> <ul> <li>item 1</li> <li>item 2</li> </ul> <p>Visit <a href="">the tutorials page</a> for more tutorials!</p>

      You’ll notice that the output results in the HTML version of the provided Markdown text.

      Now that you’ve used the markdown package to convert Markdown text to HTML, you will make a small program to read and convert Markdown files to HTML files.

      Step 2 — Creating a Program to Convert Markdown Files to HTML

      In this step, you will create a Python program that reads a Markdown file, converts its contents to HTML using the markdown.markdown() function, and saves the HTML code in a new file.

      First, open a new file called to hold the Markdown text:

      Type the following Markdown text into it:


      # Things to bring
      * Food.
      * Water.
      * Knife.
      * Plates.

      In this file you have an H1 header and four list items.

      Once you’re done, save and close the file.

      Next, open a new file called to hold the code for converting the Markdown file to an HTML file:

      Type the following Python code into it:


      import markdown
      with open('', 'r') as f:
          text =
          html = markdown.markdown(text)
      with open('Picnic.html', 'w') as f:

      Here, you first import the markdown package. You use the open() function to open the file; passing the value 'r' to the mode parameter to signify that Python should open it for reading.

      You save the file object in a variable called f, which you can use to reference the file. Then you read the file and save its contents inside the text variable. After, you convert the text using markdown.markdown(), saving the result in a variable called html.

      With the same pattern, you open a new file called Picnic.html in writing mode ('w')—note that this file does not yet exist—and write the contents of the html variable to the file. This creates and saves the new file on your system. Using the with statement when opening a file guarantees that Python will close it once processing has finished.

      Save and close the file.

      Run the program:

      This creates a new file called Picnic.html in your project directory with the following contents:


      <h1>Things to bring</h1>

      Now that you know how to open and convert Markdown files using the markdown.markdown() function, you can generate Markdown text in Python and convert Markdown files without the need to read them first.

      Step 3 — Generating Markdown from Data and Converting it to HTML

      In this step, you will create a program that generates Markdown text from a Python dictionary, saves it to a Markdown file, and converts the Markdown text to an HTML file using the markdown.markdownFromFile() function.

      Your program will generate a Markdown file called with a list of countries and their top three largest cities. After, the program will convert the generated Markdown text into HTML, then it will save the HTML in a file called cities.html.

      First open a new Python file called

      Then add the following Python code:


      import markdown
      country_cities = {'Japan': ['Tokyo', 'Osaka', 'Nagoya'],
                        'France': ['Paris', 'Marseille', 'Lyon'],
                        'Germany': ['Berlin', 'Hamburg', 'Munich'],

      In this code you first import the Python-Markdown library with import markdown. Then you define a country_cities dictionary containing a few countries as the keys and a list of the largest three cities for each country as the value. This dictionary is an example data structure; you can replace it with fetched data from a web API, a database, or any other data source.

      Next add the following code after your dictionary:


      . . .
      with open('', 'bw+') as f:
          for country, cities in country_cities.items():
              f.write('# {}n'.format(country).encode('utf-8'))
              for city in cities:
                  f.write('* {}n'.format(city).encode('utf-8'))

          markdown.markdownFromFile(input=f, output="cities.html")

      After constructing the dictionary that holds the data, you use the with open(...) as ... syntax to open a file called, which doesn’t exist yet. You open it in binary mode ('b') for writing and reading ('w+'). You use binary mode, because if you pass a string to markdown.markdownFromFile(), it will be interpreted as a path to a readable file on the file system (that is, '/home/'). Also binary mode allows you to avoid issues related to converting characters to a platform-specific representation; this guarantees that the Python program will behave the same way on any platform.

      You then go through the dictionary’s items extracting each key that contains the country’s name and saving it in the country variable. Alongside this, you extract the value that represents the list of the country’s largest cities and save it in the cities variable.

      Inside the first loop, you write the country’s name to the new file in a # Markdown header (the <h1> HTML tag). n is a special character for inserting a new line. You use .encode() because you have opened the file in binary mode. The second for loop iterates through each city and writes its name to the Markdown file as a * list item (the <li> HTML tag).

      After the first loop finishes, you have moved to the end of the file, which means markdown.markdownFromFile() won’t be able to read its contents; therefore, you use to go back to the top of the file. Before passing the f object to markdown.markdownFromFile() as input, to convert it to HTML and save it to a new file called cities.html.

      Once you’re done, save and close the file.

      Run the program:

      This command will generate two files:

      • A Markdown file with the following contents:


      # Japan
      * Tokyo
      * Osaka
      * Nagoya
      # France
      * Paris
      * Marseille
      * Lyon
      # Germany
      * Berlin
      * Hamburg
      * Munich
      • cities.html: An HTML file that contains the result of converting the contents of



      You can also use the function markdown.markdownFromFile() to convert an existing Markdown file. For example, you can convert the file to a file called Picnic-out.html using the following code:

      import markdown
      markdown.markdownFromFile(input="", output="Picnic-out.html")

      You can use the markdown.markdownFromFile() function to directly convert a file, if the file does not need any modification. If you do need to modify the Markdown file, you can read it, then convert it using the method demonstrated in Step 2.

      You’ve converted Markdown text to HTML in Python code, but Python-Markdown also provides a helpful command line interface (CLI) to quickly convert Markdown files to HTML—you’ll review this tool in the next step.

      Step 4 — Using Python-Markdown’s Command Line Interface

      In this step you will use Python-Markdown’s CLI to convert a Markdown file to HTML and print the output, or save it to an HTML file.

      You can run the Python-Markdown command line script using the -m flag supported by Python, which runs a library module as a script. For example, to convert a Markdown file, you can pass it to the markdown command as follows, replacing with the name of the file you want to convert:

      • python -m markdown

      Executing this command will print the HTML code for the Markdown text that’s present in the file.

      For example, to convert the file, run the following command:

      • python -m markdown

      This will print the following output:


      <h1>Things to bring</h1> <ul> <li>Food.</li> <li>Water.</li> <li>Knife.</li> <li>Plates.</li> </ul>

      To save the output to a file called output.html, use the following command:

      • python -m markdown -f output.html

      With this, you’ve now used the markdown command line interface to convert a Markdown file to HTML.


      In this tutorial, you have used Python to convert Markdown text to HTML. You can now write your own Python programs that take advantage of the Markdown syntax in different contexts, such as web applications using a web framework like Flask or Django. For more on how to use Markdown, check out the Markdown website. For more information on using Markdown with Python, check out the Python-Markdown documentation.

      Here are a few extensions officially supported by Python-Markdown:

      • Extra: An extension that adds extra features to the standard Markdown syntax, such as defining abbreviations, adding attributes to various HTML elements, footnotes, tables, and other features.
      • CodeHilite: An extension that adds syntax highlighting to code blocks.
      • Table of Contents: An extension that generates a table of contents from a Markdown document and adds it into the resulting HTML document.

      Source link