Scala

A functional approach towards cross-platform HTML/XML manipulation

Pine is a functional library for parsing, manipulating and rendering HTML/XML. Its development started several years ago, originally emerging out of a commercial project. Since then, we used it in many other projects (including this website). Over time, we experimented with different concepts and finally converged to a simple, yet versatile API.

As with most of our open-source software, cross-platform support was paramount. Pine has full support for all three Scala back-ends: JavaScript, LLVM and JVM. It is available for Scala 2.11 and 2.12. The JVM and LLVM back-ends have zero dependencies, JavaScript only relies on scala-js-dom.

The result of our continuous development effort is a library that can be used across platforms and therefore covers many use cases. In this article, we will present some of the features along with code examples.

Parsing and Tree Construction

You can use Pine to write inline, statically-checked HTML/XML using the html or xml string interpolator:

import pine._

val url  = "http://github.com/"
val node = html"<a href=$url>GitHub</a>"

As Scala's native XML literals will be dropped in a future version, these interpolators can be used in order to retain the benefits of compile-time checks. This feature relies on macros to convert the specified string to a Pine tree. Within the literal you can refer to external variables, e.g. to embed another node or set an attribute's value (as above).

Furthermore, HTML strings can also be parsed dynamically:

val html  = """<a href="http://github.com/">GitHub</a>"""
val node2 = HtmlParser.fromString(html)

A guiding principle in the design of Pine was to leverage type-safety. We generated the HTML5 bindings from MDN. This allows to construct the tree using a more idomatic interface:

val node3: Tag[tag.A] = tag.A.href(url).set("GitHub")

Note that all operations so far have been immutable and all three nodes are structurally equivalent.

tag.A looks like a regular type, but it is in fact a literal-based singleton type which is a new Scala language feature. If you look up the definition of tag.A, you will find this:

type A = "a"
val A = Tag("a")

Literal-based singleton types allowed us to greatly simplify the library and its footprint without compromising on expressiveness. It has been recently merged into Scala 2.13. As literal types are quite a significant feature for the entire Scala ecosystem, we will dedicate a separate article to it and explore some real-world use cases.

Manipulation

On tags, we offer typical functions you would expect such as map(), flatMap() and filter(). There are also tree-specific functions like prepend(), append(), replace() etc.

But sometimes you would like to update a deeply-nested node. Although possible with the combinators above, it is not convenient. Oftentimes, you receive templates from a front-end developer and need to populate these template files directly with content. Porting the template HTML files to Scala, even when using the inline HTML interpolator, would be duplicated effort and requires you to synchronise any future changes manually. In Pine, nodes can be identified by their ID, type or class name:

val spanAge  = TagRef[tag.Span]("age")  // Look up node by ID "age"
val spanName = TagRef[tag.Span]("name")

def render()(implicit ctx: RenderContext): Unit = {
  spanAge  := 42
  spanName := "Joe"
}

We encapsulate changes as Diffs. := sets the content of the referenced node. It takes an implicit RenderContext on which it calls the render() function. It is then up to the RenderContext to implement an efficient strategy for applying those Diffs. It could apply individual changes immediately, or enqueue them and apply the changes in a batch.

You may find it surprising that we did not have to specify any root node. This is due to the cross-platform nature of Pine. If you would like to update a regular tree node, you could use the update() operation on a Tag:

val node = html"""
  <div>
    <span id="age"></span>
    <span id="name"></span>
  </div>
"""

val result = node.update(implicit ctx => render())

Its RenderContext creates a queue of the changes which iterates the tree only once and applies the Diffs on-the-fly.

However, if the node is located in the browser's document DOM, we could use the same rendering logic. A local copy of the DOM node is not needed:

import pine.dom._
val result = DOM.render(implicit ctx => render())

Rendering

You can render a node to a string as follows:

println(root.toHtml)  // <a href="http://github.com/">GitHub</a>

HTML has slightly different rendering semantics than XML. You can call toXml which will produce valid XML content.

On the JavaScript platform, another function is available: toDom. It instantiates a node which can be directly added to the DOM:

val n = root.toDom
println(n)  // [object HTMLAnchorElement]

import org.scalajs.dom
dom.document.body.appendChild(n)

As you can see, the instantiation and manipulation logic is the same across platforms and only the final rendering call is different, i.e. toDom or DOM.render. You can easily share the core logic of your application by using sbt-crossproject.

Conclusion

We only gave a short account of some of the key features of Pine. If you would like to adopt Pine, please also read our extensive documentation. There is a sample project that implements cross-platform rendering. It performs server-side rendering for the initial page request. Page changes are handled by the browser which reduces the server load and allows for instant rendering.

We consider Pine ready for production environments. In future versions, we are going to add support for parsing HTML/XML streams as well as incremental DOM updates. Pine will always stick to its core principle of simplicity and cross-platform support. It will not become a full-fledged web framework, but we encourage you to experiment with reactive features, template loading, bindings for UI frameworks, form validation etc. which could be implemented as separate libraries.

If you need any commercial support with Pine, please get in touch with us.