Credit: This article is based off of the templating library mote. I was inspired by the simplicity of the library and it makes a great study piece for those who haven’t looked into the internals of templating engines before.

Preface: What is templating?

Template engines are tools that generate text (strings) from templates and help separate presentation from application logic.

Unless you’ve been stuck on some legacy software codebase (or haven’t been developing software with a user interface) you’ve probably used one or more template engines already.

But how do they actually work? How do you build one? Quick inspection of a few major template libraries show that they can be several hundred (erb) if not thousands of lines (erubis) of code. Even the aptly named slim isn’t so slim.

So you may think templating is a hard problem, but I want to break the problem down step by step and show you that you can build your own template engine in just a few lines of code.

Ok let’s dive in…

Defining the features

For this article the template engine will only have two rules:

  1. Lines that start with % are evaluated as ruby code.
  2. Interpolate ruby within any line between the {{ ... }} symbols. We can use this for things like {{article.title}}

That’s it? Just two rules? That’s right — keep in mind that the first rule gives us access to all of ruby. This means your common templating features (loops, calling higher order functions, embedding partials) are all available. They even come with a bonus: you don’t need to learn a new templating language or DSL since you already know ruby.

You can call another template like:

% render("path/to/template.template", {})

And you can make comments:

% # this is a comment

And execute blocks:

% 3.times do |i|
  {{i}}
% end

Given the features above here is an example template:

<html>
<body>
% if access == 0
   <div> no access :( </div>
% else
   <ul>
  % data.each do |i|
      <li>{{i}}</li>
  % end
  </ul>
% end
% # comments are just normal ruby comments
</body>
</html>

I’ll call this index.template for now.

Now we just need to figure out how to write a method to parse this template and give the correct output string. To figure out how it should work let’s think about our first intermediary step: how to just render the html output in pure ruby.

A render function that acts like index.template

In a world where templating engines don’t exist, you could implement the same logic that we hope to achieve with index.template in pure ruby as follows:

def render_index(access, data)
  output = "" #a new string to hold our output in
  output << "<html>"
  output << "<body>"

  if access == 0
     output << "<div> no access :( </div>"
  else
     output << "<ul>"
     data.each do |i|
        output << "<li>#{ i }</li>"
     end
     output << "</ul>"
 end

  output << "</body>"
  output << "</html>"

  return output
end

You can paste this into IRB and get these results:

>> render_index(0,["foo", "bar"])
=> "<html><body><div> no access :( </div></body></html>"
>> render_index(1,["foo", "bar"])
=> "<html><body><ul><li>foo</li><li>bar</li></ul></body></html>"

If your application is very limited in scope then maybe you don’t need a template engine at all and you are done! You can just hand-write your render_index(), render_header(), render_footer() methods as you need. PHP itself IS a template engine and shows why people in PHP-land do this frequently.

But the purpose of looking at render_index() is to see that if we have some way to translate index.template into render_index(), and do it generally for any template, then we’ve got our template engine. But we don’t want to actually write methods like render_index(), render_header(), render_footer() anywhere, nor do we want code-generators that do that. What we want is to dynamically generate a method that acts like render_index() whenever we need it but not have to write the actual code for render_index().

Lets look at how to do that with another intermediary step:

def define_render_index()

  func = "" # new empty string to store the string we're using to build our function in
  func << "def render_index(access, data) \n"
  func << "output = \"\" \n"
  func << "output << \"<html>\" \n"
  func << "output << \"<body>\" \n"
  func << "if access == 0\n"
  func << "  output << \"<div> no access :( </div>\" \n"
  func << "else\n"
  func << "   output << \"<ul>\" \n"
  func << "      data.each do |i|\n"
  func << "        output << \"<li> \#{ i } </li>\" \n"
  func << "      end \n"
  func << "   output << \"</ul>\" \n"
  func << "end\n"
  func << "  output << \"</body>\" \n"
  func << "  output << \"</html>\" \n"
  func << "  return output \n"
  func << "end\n"

  eval(func)
end

You can paste this into IRB and call:

>> define_render_index()
=> nil
>> render_index(1, ["foo", "bar"])
=> "<html><body><ul><li>foo</li><li>bar</li></ul></body></html>"

So now we’ve got more of a complete picture: A series of strings can be created that are a line-by-line representation of the original template, yet modified so they can be evaluated in ruby. This method, when called, will exhibit behavior as expected from the template.

The process of doing this line-by-line translation will be the root of our parse function and we’ll look at that now.

The Parse function

1. Using a Proc

Unlike define_render_index() we we don’t start our func string with a named function - instead we’ll use a Proc, then store it in a variable and .call it as needed.

2. Setting variables for the Proc

define_render_index() also hard-coded it’s variables: access & data. But we’ll need to pass the names of those variables to the parse function so it can build the proper Proc definition string. In this case we’re literally passing the variable names as a string to the parse method like

parse(template, "access, data")

3. Line by line translation of the template into the function string

Looking at define_render_index() above shows us all the rules we need to apply to a template line-by-line in order to create a new ruby method to eval. Here they are:

  • In all cases double quotes must be escaped, the line’s contents are surroundedby double quotes and linebreaks “\n” are added to the end of each line.

  • If the first line of the character (not including whitespace) is %, remove the %

before:
  % if data.empty?
after:
  "if data.empty?\n"
  • Any other line is prepended with "output <<"
before:
  <html>
after:
  "output << \"<html>\" \n"

4. {{ ... }} will be transformed to #{ … }

We’ll do this with a regex

before:
  <li></li>
after:
  "output << \"<li> \#{ i } </li>\" \n"

To execute the above rules we will split the original template file into array with .split("\n") so that each element in the array is a string that was a line of the template. The resulting array is then looped through to build a string func that we will eval.

Putting this all together gives us a parse function:

def parse(template,vars = "")
  lines = File.read(template).split("\n")

  func = "Proc.new do |#{vars}| \n output = \"\" \n "

  lines.each do |line|
      if line =~ /^\s*(%)(.*?)$/
         func << " #{line.gsub(/^\s*%(.*?)$/, '\1') } \n"
      else
         func << " output << \" #{line.gsub(/\{\{([^\r\n\{]*)\}\}/, '#{\1}') }\" \n "
      end
  end

  func << " output; end \n "

  eval(func)
end

You can try this in IRB yourself:

>> index = parse("index.template", "access, data")
=> nil
>> index.call(1,["Foo"])
=> " <html>\n <body>\n<ul>\n<li>foo</li> \n</ul>\n </body>\n </html>\n"

And that’s about it! With just a few lines of code you get a good deal of power in a template engine. Without going for lots of extra features we also get a big reduction in complexity and an increase in explicitness of the templating language. Reductions in complexity make applications easier to reason about, less error prone, and faster to develop features for.

Does it scale?

Not my code! But, mote, the library this article was inspired by sure does. It comes with some helpers and caching and we use it in production in all sorts of large web applications with success. Not to mention mote is very fast.

I also want to stress an important point about simplicity - while mote is extremely small, it is a focused and complete solution for the problem it was designed to solve.

I hope this was informative for those who never looked under the hood of a template engine or considered how to build one. If you have comments or feedback, please let me know