Custom Parser Tags and Blocks

If none of the empty, identifier, or expression tags or blocks fit, we can take it one step further and actually re-configure the parser behind Fluid to detect something specific – some unique tag structure or syntax that we invent.

Fluid is built on a parser library called Parlot. Fluid and Parlot were both developed by the same person. Parlot is behind-the-scenes in Fluid, and we won’t normally interact with it…but we can.

Parlot is a “parser combinator.” This is a software architecture/theory that consists of small functions which detect patterns, then are combined to form more complex patterns. Parlot comes with several pre-built parsers for common text constructs like strings of characters, whitespace, numbers, etc.

If you want to learn about parser combinations, here are some options:

An article called You could have invented Parser Combinators
A video called How Parser Combinators Work
The Parlot doc
The source code for the Fluid parser itself, in which are all the built-in parsers

Let’s build an example –

Say we want our template developers to easily embed SQL query results as HTML tables into their documents. We want to give them a special sql tag that takes three things:

A connection string
A quoted SQL statement
An optional CSS class for the resulting HTML table

It might look like this:

Liquid

{% sql my_db "SELECT * FROM table" theme-formal %}

That tag should parse into some construct, then execute during template rendering.

When writing our parser, we only have to worry about the part after {% sql . We need to write a parser that will detect my_db "SELECT * FROM table" theme-formal, and turn it into a data construct. (We don’t have to worry about the closing tag either – the Fluid parser will add that.)

First, let’s take our three elements, and use the Parlot library to specify parsers for all of them. Remember that parser combinators are all about taking small patterns and “growing” them into larger patterns, so we start the smallest pieces and then combine them.

Parlot has several pre-built parsers in the Terms static class.

var connectionString = Terms.NonWhiteSpace();
var sql = Terms.String();
var optionalCssClass = ZeroOrOne(Terms.NonWhiteSpace());

What we’re saying here is –

The connection string is anything that’s not whitespace
The SQL string is a quoted string (honestly, Terms.String() might be better called Terms.QuotedString())
The CSS class is again anything not whitespace, but it’s optional

These are three individual parsers. The last one is actually a parser (ZeroOrOne) with another parser (NonWhiteSpace) “inside” of it.

The key with parser combinator library is that we combine or “roll up” small, focused parsers to form more complicated constructs. So lets combine these three things into a single parser.

var sqlTagParser =
  connectionString
  .And(sql)
  .And(optionalCssClass);

We now have a single parser which contains three other parsers (one of which contains yet another parser). For this parser to activate, it needs to find some (1) non-whitespace, and (2) a quoted string, and maybe (3) another stretch of non-whitespace.

(Note: the Terms set of parsers assumes and handles the whitespace between tokens. There’s another set called Literals that doesn’t assume there will be whitespace between tokens.)

Now we need to get something out of this – when this parser activates, we need to turn it into some typed construct, so let’s make a simple SqlTag struct to hold the data.

internal struct SqlTag
{
  public string ConnectionString { get; set; }
  public string Sql { get; set; }
  public string CssClass { get; set; }
}

At the end of our parser combination (or parser “tree”), we can call a method called Then and use a tuple of the “parser hits” to populate the object. Since we were looking for three things, our tuple will have three items, the last of which might be null.

var sqlTagParser =
  connectionString
  .And(sql)
  .And(optionalCssClass)
  .Then<SqlTag>(v =>
  {
    return new SqlTag()
    {
      ConnectionString = v.Item1.ToString(),
      Sql = v.Item2.ToString(),
      CssClass = v.Item3.ToString() ?? "default"
    };
});

To be clear: everything above will execute during the parse. This is why we don’t execute the SQL there, because then the SQL would execute during the parse, and if we cached the template, it wouldn’t execute again. We would just have the same set of results from when the template was parsed.

Finally, we need to register our new tag with the Fluid parser. We need to specify:

The name of the tag (this is the {% sql part; remember the Fluid parser will the %} part internally)
The parser combination/tree we created
A method to execute for this tag when the template renders

To do this, we instantiate our parser, then call RegisterParserTag:

parser.RegisterParserTag("sql", sqlTagParser, static async (sqlTag, w, e, c) =>
{
  // We now have the populated SqlTag object in this method...
  w.Write("Execute the SQL and write out an HTML table...")
  return Completion.Normal;
});

That method will execute during every render. This is the “real-time” part of our sql tag – each time we render the template, our method runs, and (presumably) queries our database at that moment.

To re-cap, we:

Told the parser the name of the tag
Gave it a parser specific to everything after the tag name
Gave it a method to run when it finds a matching tag

Creating a block is exactly the same process, with two small differences:

Call RegisterTagBlock
The render method takes in an additional input parameter which is a List<Statement> representing what was in the block (see the chapter on blocks for examples of how to handle that). So, the signature for that method now looks like this: (sqlTag, i, w, e, c)

If we want to just generally take in named arguments to a tag or block, there is a parser for a list of arguments inside FluidParser.

Unfortunately, it’s protected, so we can’t access it directly. Additionally, we can’t just copy it out because – parser combinators working like they do – it’s made up of a bunch of other protected parsers, which are made up of others, etc.

If we want to use it to define a tag signature, we can extend the parser and expose ArgumentList, like this:

public class ExtendedParser : FluidParser
{
  public new Parser<List<FilterArgument>> ArgumentsList => base.ArgumentsList;
}

Now we can use it to register a tag.

var parser = new ExtendedParser();
parser.RegisterParserTag("sql", parser.ArgumentsList, static async (a, w, e, c) =>
{
  // a is a List<FilterArgument>
  // Do whatever you want here
  return Completion.Normal;
});

Now we can call the tag like this:

Liquid

{% sql connectionString:"my_db", sql:"SELECT * FROM table", cssClass:"theme-formal" %}

Note that the values have to be quoted, or else Fluid will interpret them as variables or expressions.