Untemplate documentation

Version 0.1.4-SNAPSHOT

This project documents the untemplate project. For its code, please see swaldman/untemplate.

This documentation is, well, long. You may wish to jump to Quickstart, so you can play around with untemplates as you go.

Table of contents

• Untemplate documentation
  • Table of contents
  • Introduction
  • Some simple untemplates
    • Embedded expressions
    • Repeatable, omittable, blocks
  • Functional untemplates
    • Untemplates are functions
    • Text blocks can be nested functions
    • Naming the top-level untemplate function
    • Untemplates, packages, and imports
    • Metainformation
  • Feature Creep
    • Output transformers
    • Customizers
    • Long delimiters, header notes, and comments
    • Attributes
    • Indexes
    • Side Scala Files
  • Quickstart
  • Cheat sheet
  • FAQs
  • Acknowledgments

Introduction

Every once in a while, I find I need to build a little website for something. I've become a fan of static site generators for that. I've worked with hugo and hexo and paradox, and they've all been great in their way.

But each time, I find myself spending a lot of time in their docs, figuring out each SSG's specific DSLs, their tricks for doing things, what variables are exposed in templates, etc.

I found myself yearning for simplicity. Why can't I just specify my static sites in my language of choice? (For me, Scala.)

Static—and dynamic—site generation is in practice largely about templates. No one enjoys embedding tons of HTML or Markdown or CSS in programming-language string literals, even in modern languages that support multiline literals and interpolated strings.

But templates are a step along the slippery path to DSLs with clever, powerful features that become the idiosyncracies and quirks I'm trying to escape. As much as possible, I want my specification language to be straightforward Scala.

Untemplate is my attempt to create a thin template veneer over vanilla Scala 3. An untemplate is just a text file that optionally includes any of four special delimiters:

Delimiter	Description
<(expression)>	Text-embedded Scala expression
()>	Code / text boundary
<()	Text / code boundary
()[]~()>	Header delimiter

These have the following effects:

• <(expression)> breaks out of plain text and inserts the result into the text
• ()> alone, at the beginning of a line, divides the file into a Scala code region, and a text region. The region above is a Scala code region.
• <() alone, at the beginning of a line, is the inverse of the prior delimiter. It divides the file into a text region and a Scala code region, with text in the region above, and code in the region beneath.
• ()[]~()> is a special header delimiter. Like ()>, it divides the file into a Scala code region above and a text region below. However, import statements in the code region above become top-level imports in the generated file.

💡 Mnemonic
For every construct, whatever an "arrow" — < or >&mdash points at is a text region. Whatever is adjacent to a parenthesis — ( or ) or () — is code.

Back to top ↺

Some simple untemplates

Let's look at an untemplate so simple it seems not to be an untemplate at all.

# Ceci n'est pas...

Well, this is just a regular markdown file, with no
special untemplate constructs. But if we wish, we can treat
it as an unemplate, and it will be immortalized as a scala
function.

It's just a markdown file! But if it's stored in an untemplate source directory as ceci-nest-pas.md.untemplate, it gets compiled to a simple scala function, ceci_nest_pas_md().

(See the little def declaration at the very end.)

// DO NOT HAND EDIT -- Autogenerated from 'ceci-nest-pas.md.untemplate' at 2024-06-18T04:02:12.851534Z
package untemplatedoc

import java.io.{Writer,StringWriter}
import scala.collection.{immutable,mutable}

val Untemplate_ceci_nest_pas_md = new untemplate.Untemplate[immutable.Map[String,Any],Nothing]:
  val UntemplateFunction                    : untemplate.Untemplate[immutable.Map[String,Any],Nothing] = this
  val UntemplateName                        : String = "ceci_nest_pas_md"
  val UntemplatePackage                     : String = "untemplatedoc"
  val UntemplateInputName                   : String = "input"
  val UntemplateInputTypeDeclared           : String = "immutable.Map[String,Any]"
  val UntemplateInputTypeCanonical          : Option[String] = untemplate.Macro.nonEmptyStringOption( untemplate.Macro.recursiveCanonicalName[immutable.Map[String,Any]] )
  val UntemplateInputDefaultArgument        : Option[immutable.Map[String,Any]] = Some(immutable.Map.empty)
  val UntemplateOutputMetadataTypeDeclared  : String = "Nothing"
  val UntemplateOutputMetadataTypeCanonical : Option[String] = untemplate.Macro.nonEmptyStringOption( untemplate.Macro.recursiveCanonicalName[Nothing] )
  val UntemplateHeaderNote                  : String = ""
  val UntemplateLastModified                : Option[Long] = Some(1673166579000L)
  val UntemplateSynthetic                   : Boolean = false

  val UntemplateAttributes : immutable.Map[String,Any] = immutable.Map.empty

  def apply(input : immutable.Map[String,Any] = immutable.Map.empty) : untemplate.Result[Nothing] =
    val writer             : StringWriter = new StringWriter(2030)
    val attrs              : immutable.Map[String,Any] = UntemplateAttributes
    var mbMetadata         : Option[Nothing] = None
    var outputTransformer  : Function1[untemplate.Result[Nothing],untemplate.Result[Nothing]] = identity

      val block0 = new Function0[String]:
        def apply() : String =
          "# Ceci n'est pas...\n\nWell, this is just a regular markdown file, with no\nspecial untemplate constructs. But if we wish, we can treat\nit as an unemplate, and it will be immortalized as a scala\nfunction.\n\n"
      writer.write(block0())

    outputTransformer( untemplate.Result( mbMetadata, writer.toString ) )

  end apply
end Untemplate_ceci_nest_pas_md

def ceci_nest_pas_md(input : immutable.Map[String,Any] = immutable.Map.empty) : untemplate.Result[Nothing] = Untemplate_ceci_nest_pas_md( input )

Embedded expressions

We'd like, of course, for our (un)template library to do a bit more than just spit out unmodified text files though. Let's modify our example just a bit:

# Ceci n'est pas... <(math.random)>

Well, this is _almost_ just a regular markdown file, with no
special untemplate constructs. But if we wish, we can treat
it as an unemplate, and it will be immortalized as a scala
function.

Now, the generated scala would transform the markdown, like this:

# Ceci n'est pas... 0.014777109102756869

Well, this is _almost_ just a regular markdown file, with no
special untemplate constructs. But if we wish, we can treat
it as an unemplate, and it will be immortalized as a scala
function.

The delimiter <( expression )> causes the expression to be evaluated into the text.

Note
This README.md is generated by an untemplate! [current subsection] So how did I slip that delimiter in? Any of the untemplate delimiters — there are only four! — can be escaped with a \ character just prior to them. The \ will be stripped, then the delimiter included in the text unmodified.

Back to top ↺

Repeatable, omittable, blocks

Often you'd like to do more than just embed a few very simple expressions in some text. So, you can break up your text into code blocks and text blocks. Let's do that, and repeat a block of text in a loop.

val num = math.round(math.random * 10).toInt

for (i <- 0 until num)
()>
# Loopy
<()

if (num >= 5)
()>

And we're a winner! (num = <(num)>)
<()
else
()>

It sucks to be us. (num = <(num)>)

Let's get a look at what it produces:

# Loopy
# Loopy
# Loopy

It sucks to be us. (num = 3)

And again!

# Loopy
# Loopy
# Loopy
# Loopy
# Loopy
# Loopy
# Loopy

And we're a winner! (num = 7)

(generated scala)

Back to top ↺

Functional untemplates

Untemplates are functions

Every untemplate defines a Scala function. By default, from a file called awesomeness.md.untemplate, this function would look like...

def awesomeness_md( input : immutable.Map[String,Any] = immutable.Map.empty ) : untemplate.Result[Nothing]

The top-level function accepts a single, author-specifiable input. (immutable.Map[String,Any] is just a default.)

It returns the template output as a simple String, along with any metadata that untemplate chooses to provide.

More specifically, each template returns a

package untemplate

final case class Result[+A](mbMetadata : Option[A], text : String ):
  override def toString() : String = text

Note that the toString() method is overridden, so you can embed Result directly an untemplate expressions. The text will be printed, without metadata.

Untemplate authors may (optionally!) specify the input name and type of the untemplate function, and output metadata type, in the header delimiter:

(sourceMarkdown : String)[immutable.Map[String,String]]~()>

This header causes the generated untemplate function to require a String input, which the template author can work with in the template as sourceMarkdown.

The function will return whatever text it generates, along with an Option[immutable.Map[String,String]].

By default, this returned metadata will be None, but the template can provide Some(metadata) by overwriting the var called mbMetadata.

😊 It's okay!
Ick, it's a var. It's okay! mbMetadata is a strictly local variable, in the single-threaded context of a function call. Your function will remain very functional as long as the input type and output metadata types that you specify are immutable.

💡 Tip!
You can specify a default argument along with your custom untemplate input type, using the usual scala syntax of ( myVar : MyType = DefaultVal )

Back to top ↺

Text blocks can be nested functions

Every text block within an untemplate can be a function.

Ordinarily, text blocks just print themselves automatically into the generated String. However, if you embed a name in the ()> delimiter that begins the block, like (entry)>, then nothing is automatically printed into the String. Instead you will have a function entry() to work with in code blocks.

The block function will return a simple String.

Use writer.write(entry()) to generate text into untemplate output.

Let's try to redo our "Loopy" template making the text block that prints # Loopy into a function.

Instead of beginning our blocks with ()>, we embed a valid scala identifier into the parenthesis, like (loopy)>.

However, doing that carries with it some complications. If we just try that in our loopy markdown file as it was, we'll get compilation errors.

The file...

val num = math.round(math.random * 10).toInt

for (i <- 0 until num)
(loopy)>
# Loopy
<()

if (num >= 5)
()>

And we're a winner! (num = <(num)>)
<()
else
()>

It sucks to be us. (num = <(num)>)

And the ickies...

[info] compiling 7 Scala sources to /Users/swaldman/Dropbox/BaseFolders/development-why/gitproj/untemplate-doc/target/scala-3.2.1/classes ...
[error] -- [E018] Syntax Error: /Users/swaldman/Dropbox/BaseFolders/development-why/gitproj/untemplate-doc/target/scala-3.2.1/src_managed/main/untemplate/untemplatedoc/untemplate_loopy2_bad_md.scala:19:26
[error] 19 |    for (i <- 0 until num)
[error]    |                          ^
[error]    |                          expression expected but val found
[error]    |
[error]    | longer explanation available when compiling with `-explain`
[error] -- [E006] Not Found Error: /Users/swaldman/Dropbox/BaseFolders/development-why/gitproj/untemplate-doc/target/scala-3.2.1/src_managed/main/untemplate/untemplatedoc/untemplate_loopy2_bad_md.scala:23:18
[error] 23 |    def loopy() = block0()
[error]    |                  ^^^^^^
[error]    |                  Not found: block0
[error]    |
[error]    | longer explanation available when compiling with `-explain`
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed

Before things worked, because when we're just printing an expression to output, we indent the call to write in the generated code so that it falls inside of any loops, if expressions, or other language constructs that the prior code block has set up.

If we are going to want to treat the block as a reusable function, then we do not wish to enclose its declaration in a very narrow scope. So, the function declaration provoked by named blocks is not indented, and named blocks do not print by default.

If you want to use a named block, define it before you get to branches in your code flow, then call your named function, which returns a String you can write. Let's fix our Loopy.

val num = math.round(math.random * 10).toInt

// comments in code blocks are fine!
// here is one way to turn text blocks into functions
(loopy)>
# Loopy
<()
for (i <- 0 until num)
  writer.write(loopy()) // you have a java.io.Writer, called writer, to send output to

// below is another, perhaps even simpler way to turn blocks into functions
//
// the indent of the if and else clauses must be lined up,
// the statement that prints becomes indented from that level!
def reportCard() : Unit =
  if (num >= 5)
()>

And we're a winner! (num = <(num)>)
<()
  else
()>

It sucks to be us. (num = <(num)>)
<()
reportCard()

Not the loveliest file. But educational. Here is the output...

# Loopy

It sucks to be us. (num = 1)

(generated scala)

Back to top ↺

Naming the top-level untemplate function

The untemplate app and file-system based tooling in the library will derive a default name for the top-level generated function by transforming its filename. Untemplate are expected to have the suffix .untemplate. The top-level file you are reading is frame-main.md.untemplate, and generates a function like...

package untemplatedoc.readme

def frame_main_md( input: immutable.Map[String,Any] ) : untemplate.Result[Nothing] = ???

Note
Return type untemplate.Result[Nothing] looks intimidating, but it's just a fancy wrapper for a String, as a field called text. The [Nothing] part just means there cannot be metadata attached to this result.

You can override the generated function name in the same way block function names are defined. Header ()[]~(untemplateDoc)> would generate

def untemplateDoc( input: immutable.Map[String,Any] ) : untemplate.Result[Nothing] = ???

Header (pubDate: Instant)[]~(untemplateDoc)> would generate

def untemplateDoc( pubDate: Instant ) : untemplate.Result[Nothing] = ???

Here's an example untemplate. Check out the generated scala code.

import java.time.{Instant, ZoneId}
import java.time.format.DateTimeFormatter

// note that all non-import (and non-package) lines in the header get 
// generated WITHIN the untemplate function, so pubDate is in scope!

val formatted = DateTimeFormatter.RFC_1123_DATE_TIME.format( pubDate.atZone( ZoneId.systemDefault() ) )

(pubDate: Instant)[]~(untemplateDoc)>

# Birthday Post

Happy Birthday to me!

_I was published on <(formatted)>._

Which generates...

# Birthday Post

Happy Birthday to me!

_I was published on Tue, 18 Jun 2024 00:04:28 -0400._

❓ Question
What if you want to override the name of the top level function and use the first text block as a function? You can!

The header ()[]~(mamaFunction.startText)> would override the outer function name with mamaFunction, and turn the first text block into a function startText().

The header ()[]~(.startText)> would turn the first text block into a function called startText(), but leave the top-level function name alone.

Back to top ↺

Untemplates, packages, and imports

Top-level untemplates are top-level functions, declared directly in a Scala package. They are paired with implementations in the form of Function1 objects, which are defined as Untemplate_ prepended to the untemplate function name.

Untemplates are usually generated from a source directory, and the default behavior is for packages to be inferred by the old-school Java convention. The directory hierarchy beneath specified source directory, to the untemplate source file, will be mapped to a package name (or dot-separated path of package names). Untemplate source files placed in the top directory belong to the unnamed "default" package.

However, you can override this default by making an explicit package declaration in the header section of your untemplate (that is, the section before a header delimiter).

If you wish all untemplates to be generated into a single flat directory, regardless of where or how deeply they were found beneath the source directory, you can set the option flatten to true.

Any package declarations or import statements in a header section go at the top-level, outside of the untemplate-generated function.

All other code in the header section gets placed inside the generated function.

This means that whatever input your header accepts is already in scope in the header section, even though its name and type may be declared at the end of the header section, inside the header delimiter.

When generating untemplates, applications may specify a set of default imports that will be inserted into all generated untemplates. So, if a static site generator makes use of a common set of types and utilities, these can be made automatically available to all templates.

Back to top ↺

Metainformation

Within an untemplate, you have access to variables containing metainformation about the generated function.

It may be useful to use UntemplateFunction as a Map key, in order to decorate it with metadata. Beyond that, if this will be useful at all, it will probably be for debugging.

For the untemplate you are reading [generated scala]:

UntemplateFunction:                      untemplate.Untemplate[Int,SubsectionMeta]@untemplatedoc.readme.functionaltemplates.content_metainformation_md
UntemplateName:                         "content_metainformation_md"
UntemplatePackage:                      "untemplatedoc.readme.functionaltemplates"
UntemplateFullyQualifiedName:           "untemplatedoc.readme.functionaltemplates.content_metainformation_md"
UntemplateInputTypeDeclared:            "Int"
UntemplateInputTypeCanonical:            Some(scala.Int)
UntemplateInputDefaultArgument:          None
UntemplateOutputMetadataTypeDeclared:   "SubsectionMeta"
UntemplateOutputMetadataTypeCanonical:   Some(untemplatedoc.SubsectionMeta)
UntemplateHeaderNote:                   "This is a header note."
UntemplateLastModified:                  Some(1718683459518) // milliseconds into UNIX epoch
UntemplateSynthetic:                     false // set to true if you implement by hand the trait untemplate. an untemplate transpiled in the usual way from the source documented here is NOT synthetic.
UntemplateAttributes:                    Map(Tags -> Set(Boring, Useful), PubDate -> 2023-02-12)
UntemplateAttributesLowerCased:          Map(tags -> Set(Boring, Useful), pubdate -> 2023-02-12)

UntemplateFunction is a reference to the Untemplate (which is a subtype of Function1) that implements your untemplate.

"Declared" type values are just String, and names may not be fully qualified.

"Canonical" types are, if possible, resolved to fully qualified type names that look through (non-opaque) aliases. However, for some types such resolution may not be possible, so these are Option[String]

UntemplateInputDefaultArgument is an Option[T] where T is the input type and the value is Some(defaultInputArg) if one was defined or None otherwise.

For more on header notes, see below.

Untemplates can define custom attributes for themselves as an immutable.Map[String,Any].

Note
All untemplates import scala.collection.*, so immutable.Map[String,Any] is a name in scope.

Because it's often convenient to resolve attributes in a case-independent way, untemplates also offer an immutable.Map[LowerCased,Any], where LowerCased is an opaque type representing a lowercased String. For example,

import untemplate.LowerCased

val pubDateKey = LowerCased("PubDate")
val mbPubDate = myUntemplate.UntemplateAttributesLowerCased.get(pubDateKey)

Untemplate metadata is available not only within your untemplates. Each untemplate becomes becomes an untemplate.Untemplate object, on which all the metadata fields are available as public vals. If you have an untemplate called mypkg.hello_md, there will be an untemplate.Untemplate object called mypkg.Untemplate_hello_md, so you can write, e.g.

val helloHeaderNote = mypkg.Untemplate_hello_md.UntemplateHeaderNote

This can be particularly useful in combination with indexing.

Back to top ↺

Feature Creep

Output transformers

In the header or any code section of an untemplate, you can define an OutputTransformer, like this:

outputTransformer = myOutputTransformer

As the name suggests, an output transformer will simply transform the function output.

If you haven't defied a custom output metadata type, then it must be a Function1 that maps untemplate.Result[Nothing] => untemplate.Result[Nothing].

If you have defined an output metadata type, say HttpMetadata, then it must be a function untemplate.Result[HttpMetadata] => untemplate.Result[HttpMetadata].

By default, every untemplate output travels through the identity transformer identity. It's as if you had set:

outputTransformer = myOutputTranformer

But you can set your own, more interesting, transformer.

You can set output transfers as above, "by hand", or you can use an untemplate.Customizer to transform a whole class of untemplates. For example, you could have all untemplates generated from a file like something.md.untemplate pass through an output transformer that converts Markdown to HTML.

The untemplate you are now reading is passed through an output transformer, which embeds the text in a markdown subsection, rendering at the appropriate level something like

### Section Title

Lorem ipsum dolor sit amet, consectetur adipiscing elit,
sed do eiusmod tempor incididunt ut labore et dolore magna
aliqua.

That output transformer is itself defined by an untemplate, untemplatedoc.readme.subsection_content_transformer_md.

Most output transformers will not be untemplates! But an untemplate is just a way to define a function that returns an untemplate.Result, and an output transformer is just that kind of function.

It is as the untemplate for this section includes the line

outputTransformer = untemplatedoc.readme.subsection_content_transformer_md

and in earlier drafts, it did precisely that! But that got repetitive to include for every subsection, so we turn to customizers.

Back to top ↺

Customizers

In generating, for example, a website, one might find oneself repeating the same boilerplate over and over again. Perhaps every *.md file should be piped into a Markdown renderer, and every *.adoc file Asciidoctor. Perhaps many of your untemplates make use of the same suite Scala or Java libraries, for which you would have to repetitively declare import statements.

Customizers to the rescue!

When you generate a collection of untemplates, you can specify a Customizer.Selector, which is just a function Customizer.Key => Customizer.

During the "transpilation" to Scala of each untemplate, a Customizer.Key is generated, like

  final case class Key(inferredPackage      : String, // empty string is the default package
                       resolvedPackage      : String, // empty string is the default package
                       inferredFunctionName : String,
                       resolvedFunctionName : String,
                       outputMetadataType   : String,
                       headerNote           : String,
                       sourceIdentifier     : Option[String])

• inferredPackage is the package that would be inferred for the customizer based on its place in the directory hierarchy being transpiled. It may be overridden by explicit package declarations in the header (but it usually isn't).
• resolvedPackage is usually the same as inferredPackage, unless the untemplate author has overridden that with explicit package declarations, in which case resolvedPackage will be derived from those.
• inferredFunctionName is the function name automatically inferred from the untemplate file name (or equivalent, for nonfile sources of untemplates). It is usually just the file name itself with the .untemplate suffix stripped off, and both - and . characters converted to underscore _.
• resolvedFunctionName may just be inferredFunctionName, but if a user has explicitly defined a function name in the untemplate header delimiter, the explcitly defined name will be used instead.
• outputMetadataType is the output metadata type of the untemplate, Nothing by default but often declared to become something more interesting.
• headerNote will usually be an empty string, but untemplate authors can provide a small note next to the untemplate header definition, and this note will be provided in the Customizer.Key.
• Finally, there may be a sourceIdentifier, which in file-system-based implementations (i.e. the current implementation) will be the original untemplate file name (without '.' or '-' converted to underscores).

For the subsection you are reading, the key would have been:

   Key(inferredPackage      = "untemplatedoc.readme.featurecreep",
       resolvedPackage      = "untemplatedoc.readme.featurecreep",
       inferredFunctionName = "content_customizers_md",
       resolvedFunctionName = "content_customizers_md",
       outputMetadataType   = "SubsectionMeta",
       headerNote           = "",
       sourceIdentifier     = Some("content-customizers.md.untemplate"))

From the key that your selector is provided, you produce an untemplate.Customizer. Ordinarily you begin with Customizer.empty, which looks like this:

final case class Customizer(mbOverrideInferredFunctionName : Option[String]              = None,
                            mbDefaultInputName             : Option[String]              = None,
                            mbDefaultInputTypeDefaultArg   : Option[InputTypeDefaultArg] = None,
                            mbOverrideInferredPackage      : Option[String]              = None, // You can use empty String to override inferred package to the default package
                            mbDefaultMetadataType          : Option[String]              = None,
                            mbDefaultMetadataValue         : Option[String]              = None,
                            mbDefaultOutputTransformer     : Option[String]              = None,
                            extraImports                   : Seq[String]                 = Nil)

This customizer does nothing at all. Note that all of its fields are options or collections, which can be (and so far are) empty.

However, you can use case class copy= to selectively fill some of these values. You can override the default input type for a whole range of untemplates from its usual immutable.Map[String,Any] to, well, anything. Similarly you can override the input name, the function name, the default input argument, the inferred package, the metadata type and its default value, or the default output transformer.

Note that it is always "defaults" that you are overriding.

If an untemplate author, in the untemplate header delimiter, explicitly specifies the input name, type, or input default argument, the function name or output metadata type, if she sets the output metadata value or the output transformer, the author's action will override the customizer provided defaults. The customizer customizes how untemplates are generate what authors do not explicitly specify.

An exception to this rule is extraImports, which will always be provided in addition to any imports the author provides.

Back to top ↺

Long delimiters, header notes, and comments

Long delimiters

Recall from our introduction the four untemplate delimiters:

Delimiter	Description
<(expression)>	Text-embedded Scala expression
()>	Code / text boundary
<()	Text / code boundary
()[]~()>	Header delimiter

The three "line delimiters" — ()>, <(), and ()[]~()> &mdash must be tucked against the left-hand margin. They are small and easy to miss while scanning a file, even though they dramatically switch the meaning of portions of the file from code to text.

Sometimes you may wish these boundaries were more obvious. So they can be! The untemplate transpiler will let you expand ()> and <() by placing arbitrary numbers of - characters, and also the delimiter appropriate > or < between the parentheses and the arrow. So all of the following are equivalent to the untemplate engine:

• ()>
• ()>>>>>>>>>>>>>>>>>>>>>>>>>>>
• ()-------------------------->
• ()------>------>------>----->

Similarly, you can close your textblock, equivalently, with

• <()
• <<<<<<<<<<<<<<<<<<<<<<<<<<<<()
• <---------------------------()
• <-------<-------<-------<---()

The styles don't have to match. The transpiler interprets all of the first list as ()>, and all of the second list as <(), regardless of what style you choose.

In any of its styles, you can enter a name in the parentheses of a start-text block to convert the block following into a named function.

The headers can also be elongated, with the ~ character. So, the following are all equivalent:

• ()[]~()>
• ()[]~~~()>
• ()[]~~~~~~~~~~~~~~~~~~~~~~~~~~~~()>

Again, the meaning of the header, including your ability to optionally include an input name, type, and default argument in the first (), an output metadata type in the [], and a function name in the secon () remain unchanged.

Delimiter comments and header notes

In general, nothing should be placed to the right of ()> and <() delimiters (in their standard or long variants), but if you wish, you may place a comment beginning with a # character. So this is a fine text-bloc-as-function:

(regreet)>--->  # because we love to greet the user, a function we can reuse a lot
       >>>>>>>>> Hello <( name )>!!! <<<<<<<<<
<----------<()

However, for the header delimiter, untemplates support a "header note", at most one per untemplate:

()[]~~~~()> This is a header note. # This is a comment

The header note becomes metainformation, available within your untemplate and published externally. It is also included in Customizer.Key, so you can set-up defaults for whole groups of untemplates based on header notes if you wish. See Customizers above.

Back to top ↺

Attributes

When you write an untemplate, you can associate attributes with it that are accessible within the untemplate as attrs, but also published by your Untemplate as metainformation.

In combination with indexes, this lets you filter in pretty arbitrary ways for untemplates that may be of interest.

In the header section of an untemplate, if you define code lines that begin with '>', those lines will be generated into the body or constructor of the Untemplate subclass rather than within the function body. To define attributes, you just construct an immutable.Map as constructor text.

For example, this untemplate's header includes the following in its header section:

>
> val UntemplateAttributes = immutable.Map (
>   "FavoriteColor" -> "Blue",
>   "FavoriteDay"   -> "Tuesday",
> )
>

Find where that appears in the generated scala.

Now if I write the expression <( attrs("FavoriteColor") )>, it generates: Blue

If I write the expression <( attrs("FavoriteDay") )>, it generates: Tuesday

Suppose we included a "tags" key in untemplate attributes, whose value might be a Set[String]. It would be easy to filter through our index to find untemplates tagged with some value.

Because it's often convenient to resolve attributes in a case-independent way, untemplates also offer an immutable.Map[LowerCased,Any], where LowerCased is an opaque type representing a lowercased String. For example,

import untemplate.LowerCased

val pubDateKey = LowerCased("PubDate")
val mbPubDate = myUntemplate.UntemplateAttributesLowerCased.get(pubDateKey)

Back to top ↺

Indexes

When you generate a collection (usually a directory hierarchy) of untemplates, you can request that the untemplate library generate an index of the untemplates produced that you can work with at runtime.

Here is an example, where I have specified Some("untemplatedoc.Untemplates") as my index, which indexes the untemplates generating this documentation:

// DO NOT HAND EDIT -- Autogenerated at 2024-06-18T04:04:27.482153Z
package untemplatedoc

val Untemplates = scala.collection.immutable.SortedMap[String,untemplate.Untemplate[Nothing,Any]](
  "untemplatedoc.readme.content_cheat_sheet_md" -> untemplatedoc.readme.Untemplate_content_cheat_sheet_md,
  "untemplatedoc.readme.featurecreep.content_main_md" -> untemplatedoc.readme.featurecreep.Untemplate_content_main_md,
  "untemplatedoc.ceci_nest_pas2_md" -> untemplatedoc.Untemplate_ceci_nest_pas2_md,
  "untemplatedoc.readme.featurecreep.content_long_delimiters_etc_md" -> untemplatedoc.readme.featurecreep.Untemplate_content_long_delimiters_etc_md,
  "untemplatedoc.readme.featurecreep.content_indexes_md" -> untemplatedoc.readme.featurecreep.Untemplate_content_indexes_md,
  "untemplatedoc.readme.subsection_content_transformer_md" -> untemplatedoc.readme.Untemplate_subsection_content_transformer_md,
  "untemplatedoc.readme.functionaltemplates.content_untemplates_r_functions_md" -> untemplatedoc.readme.functionaltemplates.Untemplate_content_untemplates_r_functions_md,
  "untemplatedoc.readme.featurecreep.content_output_transformers_md" -> untemplatedoc.readme.featurecreep.Untemplate_content_output_transformers_md,
  "untemplatedoc.readme.content_faqs_md" -> untemplatedoc.readme.Untemplate_content_faqs_md,
  "untemplatedoc.loopy2_md" -> untemplatedoc.Untemplate_loopy2_md,
  "untemplatedoc.readme.featurecreep.content_side_scala_md" -> untemplatedoc.readme.featurecreep.Untemplate_content_side_scala_md,
  "untemplatedoc.readme.functionaltemplates.content_main_md" -> untemplatedoc.readme.functionaltemplates.Untemplate_content_main_md,
  "untemplatedoc.loopy_md" -> untemplatedoc.Untemplate_loopy_md,
  "untemplatedoc.ceci_nest_pas_md" -> untemplatedoc.Untemplate_ceci_nest_pas_md,
  "untemplatedoc.readme.embed_exercise_md" -> untemplatedoc.readme.Untemplate_embed_exercise_md,
  "untemplatedoc.readme.somesimpleuntemplates.content_embeddable_expressions_md" -> untemplatedoc.readme.somesimpleuntemplates.Untemplate_content_embeddable_expressions_md,
  "untemplatedoc.readme.somesimpleuntemplates.content_repeatable_omittable_md" -> untemplatedoc.readme.somesimpleuntemplates.Untemplate_content_repeatable_omittable_md,
  "untemplatedoc.readme.featurecreep.content_attributes_md" -> untemplatedoc.readme.featurecreep.Untemplate_content_attributes_md,
  "untemplatedoc.readme.content_acknowledgments_md" -> untemplatedoc.readme.Untemplate_content_acknowledgments_md,
  "untemplatedoc.readme.functionaltemplates.content_text_blocks_as_functions_md" -> untemplatedoc.readme.functionaltemplates.Untemplate_content_text_blocks_as_functions_md,
  "untemplatedoc.readme.content_quickstart_md" -> untemplatedoc.readme.Untemplate_content_quickstart_md,
  "untemplatedoc.readme.somesimpleuntemplates.content_main_md" -> untemplatedoc.readme.somesimpleuntemplates.Untemplate_content_main_md,
  "untemplatedoc.readme.functionaltemplates.content_naming_the_top_level_md" -> untemplatedoc.readme.functionaltemplates.Untemplate_content_naming_the_top_level_md,
  "untemplatedoc.readme.frame_main_md" -> untemplatedoc.readme.Untemplate_frame_main_md,
  "untemplatedoc.readme.introduction.content_main_md" -> untemplatedoc.readme.introduction.Untemplate_content_main_md,
  "untemplatedoc.readme.featurecreep.content_customizers_md" -> untemplatedoc.readme.featurecreep.Untemplate_content_customizers_md,
  "untemplatedoc.untemplateDoc" -> untemplatedoc.Untemplate_untemplateDoc,
  "untemplatedoc.readme.functionaltemplates.content_metainformation_md" -> untemplatedoc.readme.functionaltemplates.Untemplate_content_metainformation_md,
  "untemplatedoc.readme.functionaltemplates.content_untemplates_packages_imports_md" -> untemplatedoc.readme.functionaltemplates.Untemplate_content_untemplates_packages_imports_md,
)

You'll note that, a bit uselessly, the untemplates in indexes are typed very generically as accepting Nothing for their input and generating an Option[Any] as their metadata. In order to call the indexed untemplate functions, you must cast them first to untemplate.Untemplate[<InputType>,<OutputMetadataType>] or more generically Function1[<InputType>,<OutputMetadataType>].

Usually you will know, by conventions you've adopted, what kinds of functions untemplates are likely to be. Perhaps all the functions under a package called posts accept a PageInfo type and return EntryMetadata. You might then try something like

  def hasHappyTag( ut : Untemplate[Nothing,Any] ) : Boolean =
    val mbTags = ut.UntemplateAttributes.get("tags")
    mbTags match
      case Some(tags) if tags.isInstanceOf[Set[String]] =>
        tags.asInstanceOf[Set[String]].contains("happy")
      case _ => false

  val happyBlogPosts =
    Untemplates // my generated index
      .filter( _(0).indexOf(".posts.") >= 0 )
      .filter( tup => hasHappyTag(tup(1)) )
      .map( tup => (tup(0), tup(1).asInstanceOf[untemplate.Untemplate[PageInfo,EntryMetadata]]) )
      .to(immutable.SortedMap)

Now you have an index of functions you can call, like

val myResult = blogPosts("poopyblog.posts.firstPost_md")( myPageInfo )

Your result will be typed as an untemplate.Result[EntryMetadata].

If you do not know the types of your untemplates, even the generic, uncallably typed untemplates held by the default index can be queried for all their metainformation, including input and output metadata types.

Back to top ↺

Side Scala Files

For small utilities to be used only by untemplates within a particular package, it's natural to just write a whatever.scala file next to your untemplates, and then access your utilities directly.

This is now supported (as of v0.1.2).

These "side scala" files will be placed in the same package as untemplates you defined in the same directory. Your untemplates will be able to access your utilities without imports or fully-qualified names.

Note
All of this assumes that you've placed no explicit package declarations in your untemplate or side scala files, and that you haven't overridden the default package location with a Customizer)

Back to top ↺

Quickstart

Prerequisite

You'll need something that can extract a giter8 template from github to get started. We'll use sbt new, but mill init or g8 should also be fine.

Let's do it!

In a place where you are happy to have a new project directory created, run:

% sbt new swaldman/untemplate-seed.g8
[info] welcome to sbt 1.8.2 (Oracle Corporation Java 17.0.5)
[info] loading settings for project global-plugins from dependency-graph.sbt,gpg.sbt,metals.sbt ...
[info] loading global plugins from /Users/swaldman/.sbt/1.0/plugins
[info] set current project to new (in build file:/private/var/folders/by/35mx6ty94jng67n4kh2ps9tc0000gn/T/sbt_16d4daf/new/)

You'll be prompted to "fill in the blanks" of the giter8 template. You can do want you want, but the defaults will be fine for now. Just hit return a bunch of times.

name [untemplateplay]:
module [untemplateplay]:
package [example]:
untemplate_version [0.1.4-SNAPSHOT]:
mill_version [0.10.10]:

Template applied in /Users/swaldman/tmp/./untemplateplay

Cool. Now go into your new directory, and run ./mill untemplateplay:

% cd untemplateplay
% ./mill untemplateplay
Compiling /Users/swaldman/tmp/untemplateplay/build.sc
[38/51] untemplateplay.compile
[info] compiling 3 Scala sources to /Users/swaldman/tmp/untemplateplay/out/untemplateplay/compile.dest/classes ...
[info] done compiling
[51/51] untemplateplay.run

Following this (but no spoilers here!), you should see the output of your first untemplate! Hooray!

Check out the file untemplateplay/src/example/core.scala:

package example

@main def hello() = println( hello_txt() )

Pretty simple! This is just an ordinary Scala 3 file. hello_text() is a function defined by an untemplate, which you can check out in untemplateplay/untemplate/example/hello.txt.untemplate.

---

Exercise 1: Supply untemplate input

The function hello_text actually accepts name : String as input. We are just using a default argument. Try modifying untemplateplay/src/example/core.scala so the function call is like hello_txt( "<your name>" ), then rerun ./mill untemplateplay

---

Shocking, right?

Building your untemplate has caused two Scala source files to be generated into the directory out/untemplateplay/untemplateGenerateScala.dest/example/. Check those out!

---

Exercise 2: Write your own!

Using the information and examples in the documentation above, write your own untemplate! Modify the @main method to invoke it instead of (or in addition to) hello_txt().

As long as you drop your untemplate in the package directory example under untemplateplay/untemplate, your untemplate can call any scala code you add to core.scala, or any other source file in example. (Of course, you can also create Scala code in other packages, and access it from your untemplate with import statements.)

Your untemplates have seamless access to your Scala code. To your Scala code, each untemplates is just an ordinary function.

---

Before we go, check out the build.sc file that was generated for you. In particular, note this section:

  override def untemplateSelectCustomizer: untemplate.Customizer.Selector = { key =>
    var out = untemplate.Customizer.empty

    // to customize, examine key and modify the customer
    // with out = out.copy=...
    //
    // e.g. out = out.copy(extraImports=Seq("untemplateplay.*"))

    out
  }

When you have a large body of untemplates integrating with an application, often collections of them will share common imports, input and output types, etc. It's tedious to repetitively specify all of this within each untemplate. Here is where you might use customizers to override defaults that are not explicitly specified for subsets of your untemplates that you select. Right now, for all untemplates, we are returning Customizer.empty, meaning no customizations, we are using untemplate library defaults. But here you can selectively override these!

And that, for now, will suffice for our quick start!

Back to top ↺

Cheat sheet

Untemplate Cheat Sheet

Line Delimiters (start of line):
  ()>       -- text start (code end)
  <()       -- text end   (code start)
  ()[]~()>  -- header delimiter (at most one, begins first text block of file)

Embedded expression:
  <( expression )>

Function generated:
  Given a file hello.md.untemplate with
    no header or
    default header ()[]~()> generates
       def hello_md(input:immutable.Map[String,Any]=immutable.Map.empty): untemplate.Result[Nothing]
    (i:Int=0)[]~()> generates
       def hello_md(i:Int=0): untemplate.Result[Nothing]
    ()[Instant]~()> generates
       def hello_md(input:immutable.Map[String,Any]=immutable.Map.empty): untemplate.Result[Instant]
    ()[]~(hi)> generates
       def hi(input:immutable.Map[String,Any]=immutable.Map.empty): untemplate.Result[Nothing]
  Fill in any or all of the header fields. Mix or match.

  Output type:
    final case class Result[+A](mbMetadata : Option[A], text : String ):
      override def toString() : String = text

  Function object:
    def hello_md(input : In) : untemplate.Result[Out] <==> val Untemplate_hello_md : untemplateUntemplate[In,Out] <: Function1[In,Out]

Inside template:
  val writer            <: java.io.Writer
  val attrs              : immutable.Map[String,Any]
  var mbMetadata         : Option[Out] = None
  var outputTransformer  : Function1[untemplate.Result[out],untemplate.Result[out]] = identity

  Textblocks automatically write to writer, unless defined as block functions. (Below)
  Code blocks write to writer, can set mbMetadata and/or outputTransformer

Block functions:
  Text blocks began as
   (myFunction)>
     - are not automatically written into output
     - take no arguments and return strings
     - so, e.g.
         for (i <- 0 until 3) writer.write(myFunction().toUpperCase)

Metainformation:
  Available within untemplates and as public vals from Untemplate function objects
    val UntemplateFunction                    : untemplate.Untemplate[In,Out]
    val UntemplateName                        : String
    val UntemplatePackage                     : String
    val UntemplateInputName                   : String
    val UntemplateInputTypeDeclared           : String
    val UntemplateInputTypeCanonical          : Option[String]
    val UntemplateInputDefaultArgument        : Option[In]
    val UntemplateOutputMetadataTypeDeclared  : String
    val UntemplateOutputMetadataTypeCanonical : Option[String]
    val UntemplateHeaderNote                  : String
    val UntemplateAttributes                  : immutable.Map[String,Any]

Customizers:
  Bulk customization of templates to avoid having to write repetative headers or code
  - You provide Customizer.Selector: (key : Customizer.Key) => Customizer
  - For each untemplate, selector will be provided with key

      final case class Key(inferredPackage      : String, // empty string is the default package
                           resolvedPackage      : String, // empty string is the default package
                           inferredFunctionName : String,
                           resolvedFunctionName : String,
                           outputMetadataType   : String,
                           headerNote           : String,
                           sourceIdentifier     : Option[String])

    selector returns customizer

      final case class Customizer(mbOverrideInferredFunctionName : Option[String]              = None,
                                  mbDefaultInputName             : Option[String]              = None,
                                  mbDefaultInputTypeDefaultArg   : Option[InputTypeDefaultArg] = None,
                                  mbOverrideInferredPackage      : Option[String]              = None, // You can use empty String to override inferred package to the default package
                                  mbDefaultMetadataType          : Option[String]              = None,
                                  mbDefaultMetadataValue         : Option[String]              = None,
                                  mbDefaultOutputTransformer     : Option[String]              = None,
                                  extraImports                   : Seq[String]                 = Nil)

    overriding defaults, ie whatever untemplate author does not explicitly specify

Delimiter variations:
  Long delimiters:
  ()> equivalents
     ()>>>>>>>>>>> # Comments beginning '#' are permissible
     ()---------->
     ()->-->>->--> # any combination of '>' and '-' between '()' and terminal '>'
  <() equivalents
     <<<<<<<<<<<<() # Comments beginning '#' are permissible
     <-----------()
     <<--<-<<<--<() # any combination of '<' and '-' between the first '<' and terminal '()'
  ()[]~()> equivalents
     ()[]~~~()> header note # Comments beginning '#' are permissible. Also "header notes"
     ()[]~~~~~~~()>         # Any number at least one of `~` characters are permissible

Attributes:
  You can associate an immutable.Map[String,Any] which can be queried from
  inside or outside of your untemplate function. You can thus "tag" untemplates
  in ways that may be useful to applications that will autogenerate text.

Indexes:
  All the untemplates your project generates can be collected into an index
  of type immutable.Map[String,Untemplate[Nothing,Any]]. Either by convention
  or by examining type metainformation (see above), you'll have to downcast
  these to more specific types before you can call them. But if you will
  be autogenerating text, you can filter through this index based on name
  and/or metainformation to organize what gets generated where.

Back to top ↺

FAQs

How does the untemplate library transform untemplate source file names?

Transforming untemplate source file to Scala function names

Untemplate function names by default derive from the name of their encloding source file. Untemplate source files end with the suffix .untemplate. That suffix is stripped, and then any special characters that wouldn't be legal in a scala function name, including any - or . characters, are replaced with underscore (_).

Transforming untemplate source file to Scala source file names

Although usually you are not interested in the Scala source code files generated from untemplates, but occasionally you may want tools to see or inspect them, or documentation to expose them. Generated source code files originally just took the full untemplate source file name (including the .untemplate suffix) + a .scala suffix, but some Scala tooling has trouble with multiply dotted source files. So, currently generated source files take the full untemplate source file name, then convert any . chars to -, and then append .scala

Back to top ↺

Acknowledgments

This project owes a debt to Java Server Pages (JSPs), and the special place they will always have in my heart.

The mill plugin I am currently working on owes a debt to Twirl's plugin and ScalaXB's plugin, from which I am gently (and much less sophisticatedly) cribbing.

Back to top ↺