Learning Assemble, Step 10 - Pages from Data

Assemble is typically configured to build output pages from Handlebars templates or markdown files, where one source file maps to one output page. However, it is possible to dynamically generate pages from data rather than from individual source files. This technique is demonstrated in the assemble-blog-theme template. I found it to be quite mind-blowing at first, it challenged my understanding of Assemble and how I thought it should work. Experimenting with the technique helps build a better understanding of Assemble, and we'll try it here.

• Understanding how Assemble pages may be dynamically generated from data
• Implementing basic dynamic page generation

Analyzing assemble-blog-theme

Let's take a look at a few critical segments of assemble-blog-theme to get some pointers about creating pages from dynamic data. The important sections appear to be the page data and the Gruntfile configuration that associates the data to the pages model.

Data

The blog pages are all defined in data/pages.json:

{
  "content": "./content",
  "thumbnail": "<img class="thumbnail" data-src="holder.js/100%x200/auto">",
  "layouts": {
    "blog": "blog.hbs"
  },
  "posts": [
    {
      "data": {
        "title": "My First Post",
        "date": "2014-01-01",
        "layout": "<%= pages.layouts.blog %>",
        "categories": [],
        "tags": []
      },
      "content": "<%= pages.thumbnail %>{{lorem ipsum.paragraph}}",
      "filename": "2014-01-01.html"
    },
    {
      "data": {
        "title": "My Second Post",
        "date": "2014-01-02",
        "layout": "<%= pages.layouts.blog %>",
        "categories": [],
        "tags": []
      },
      "content": "<%= pages.thumbnail %>{{lorem ipsum.paragraph}}",
      "filename": "2014-01-02.html"
    },
    {
      "data": {
        "title": "My Third Post",
        "date": "2014-01-02",
        "layout": "<%= pages.layouts.blog %>",
        "categories": [],
        "tags": []
      },
      "content": "<%= pages.thumbnail %>{{lorem ipsum.paragraph}}",
      "filename": "2014-01-03.html"
    }
  ]
}

The file defines the posts array of three pages, plus some additional shared attributes. Each page has some familiar parts:

• data - We have been defining data as YAML Front-Matter, and only using data for context in page iteration loops, but here it is defined as literal.
• content - The body content of each page, including Handlebars and Underscore-style expressions.
• filename - The name of the output file that is generated.

Gruntfile

There are a few critical portions of the Gruntfile that put the page data to work. First, the pages.json file is loaded into a variable:

  // Project configuration.
  grunt.initConfig({

    // Project metadata
    pkg   : grunt.file.readJSON('package.json'),
    vendor: grunt.file.readJSON('.bowerrc').directory,
    site  : grunt.file.readYAML('_config.yml'),
    pages : grunt.file.readJSON('data/pages.json'),

In the Assemble task, a separate target is configured to output blog pages from the posts data.

      // Generate posts from "./data/pages.json"
      blog: {
        options: {
          pages: '<%= pages.posts %>'
        },
        files: {
          '<%= site.dest %>/': ['templates/index.hbs']
        }
      },

On line 78, the pages object is assigned the posts array from the JSON file. It seems like black magic to me. It appears to utilize the "Custom Options" technique documented in Assemble Options Overview, and does so to override the normal page creation scheme.

On line 81, there is a file dependency defined. In fact, the index page gets processed by Assemble when this task runs. I believe this to be a necessary side effect, the task requires at least one source file to run, but index.hbs is otherwise processed under the site task (not show in my Gruntfile snippet).

Adding Dynamic Pages to Our Project

Now we will apply the lessons from assemble-blog-theme to dynamically create some pages for our own static site. Suppose we wish to generate documentation for an API, where there would eventually be a large number of API pages, too many to manage by hand.

Basic Configuration

We know we need to do at least two things: create some page data, and configure an Assemble target that uses the data. I created some page data in source/data/api.json:

{
    "pages": [
        {
            "content": "Docs for API method: authenticate",
            "filename": "authenticate.html"
        }
    ]
}

Just one page, no frills, as basic as I could keep it. Now we'll modify the Gruntfile to put this to work:

        api : grunt.file.readJSON('source/data/api.json'),
        assemble: {
            hello: {
                options: {
                    data: 'source/data/*.{json,yml}',
                    layoutdir: 'source/templates/layouts',
                    layout: 'default.hbs',
                    partials: 'source/templates/partials/**/*.hbs',
                    collections: [
                        { name: 'navTags', inflection: 'navTag' }
                    ],
                    helpers: ['source/helpers/**/*.js'],
                    plugins: ['assemble-middleware-sitemap'],
                    sitemap: {
                        homepage: "http://awesome-site.bogus",
                        relativedest: true,
                        exclude: ['diagnostics'],
                        changefreq: 'monthly'
                    }
                },
                files: [
                    { expand: true, cwd: 'source/templates/pages', src: '**/*.{hbs,md}', dest: 'output/' }
                ]
            },
            api: {
                options: {
                    flatten: true,
                    pages: '<%= api.pages %>'
                }
                ,
                files: {
                  'output/api/': ['source/templates/pages/index.hbs']
                }
            }
        },

I got this to work, but there are a few ugly parts:

1. I had to change the source/destination file specification to the {dest} : {source} format. I had first copied the specification from our hello target and modified it, but this resulted in all out the api output files having the same file name. Using this format also now requires the flatten: true option.
2. The generated authenticate.html page has the correct content, but no layout, since we were too cheap to provide one. The formatting could be greatly improved.
3. The output of the api task has two files, but our index.html is useless as part of our API output.

Running "assemble:api" (assemble) task
Assembling output/api/index.html OK
Assembling output/api/authenticate.html OK
>> 2 pages assembled.

Adding a Layout

The assemble-blog-theme pages have a layout, and so should we. Let's create a layout for our API pages. Our new layout, source/templates/layouts/api-layout.hbs, is very simple:

<!DOCTYPE html>
<html>
    <head>
        <title>{{title}} | {{site.title}}</title>
    </head>
    <body>
        <h1>API - {{title}}</h1>
        {{> body }}
    </body>
</html>

You might notice this layout uses the title data attribute, so we'll expand out page data model to include title:

{
    "pages": [
        {
            "data": {
                "title": "authenticate"
            },
            "content": "Docs for API method: authenticate",
            "filename": "authenticate.html"
        }
    ]
}

And then we'll update Gruntfile to use this with the apiAssemble target as the default layout. This is a bit different than the assemble-blog-theme approach, where the layout was specified for each page based on a single variable in the data file. I think specifying a default layout will work well for most dynamic generation scenarios, and it's always possible to specify an override on a page-by-page basis.

            api: {
                options: {
                    flatten: true,
                    layout: 'source/templates/layouts/api-layout.hbs',
                    pages: '<%= api.pages %>'
                }
                ,
                files: {
                  'output/api/': ['source/templates/pages/index.hbs']
                }
            }

Now when Assemble generates our API pages, they have some proper HTML formatting, albeit extremely minimal.

Adding an API Index Page

We need to have at least one actual source file to make Grunt go through the motions and create our dynamic pages. One solution is to just pick a page, any page, and have it write nonsense output, which we simply ignore. A second realistic option is to include some kind of index page. Let's try doing that now, calling our API index source/templates/pages/api-index.hbs:

---
title: "Index"
---
<ul>
    {{#each pages}}
        {{#unless this.isCurrentPage}}
            <li>
                <a href="{{relativeLink}}">{{data.title}}</a>
            </li>
        {{/unless}}
    {{/each}}
</ul>

This dead-simple index lists the API pages, except itself. This limits the list to dynamic pages. Now we'll update the Gruntfile to use this page rather than re-using the index.hbs page:

        assemble: {
            hello: {
                options: {
                    data: 'source/data/*.{json,yml}',
                    layoutdir: 'source/templates/layouts',
                    layout: 'default.hbs',
                    partials: 'source/templates/partials/**/*.hbs',
                    collections: [
                        { name: 'navTags', inflection: 'navTag' }
                    ],
                    helpers: ['source/helpers/**/*.js'],
                    plugins: ['assemble-middleware-sitemap'],
                    sitemap: {
                        homepage: "http://awesome-site.bogus",
                        relativedest: true,
                        exclude: ['diagnostics'],
                        changefreq: 'monthly'
                    }
                },
                files: [
                    { expand: true, cwd: 'source/templates/pages', src: ['**/*.{hbs,md}', '!api-index.hbs'], dest: 'output/' }
                ]
            },
            api: {
                options: {
                    flatten: true,
                    layout: 'source/templates/layouts/api-layout.hbs',
                    pages: '<%= api.pages %>'
                }
                ,
                files: {
                  'output/api/': ['source/templates/pages/api-index.hbs']
                }
            }
        },

I included the larger Gruntfile snippet to show two changes. First, I updated the api target to use the api-index.hbs page on line 35. There was a side effect however, the api-index.hbs page was picked up in our hello target as well. Since that is not what we intended, I added an exclusion for the file on line 24. Another solution would be to put the api templates in a different folder, but I was too lazy.

For the code, please see the learning-assemble repository on GitHub. The branch step10-pages-from-data contains the completed code from this post.