Making objects queryable in Scala

Defining a Queryable trait

In a running program its often nice to have a way of querying and updating the various components of the application. In this example I’ll demonstrate one way you could do this in Scala by defining a Queryable trait. By inheriting this trait, an object can make available properties for reads and updates, and enable traversal to other objects. The data-format used for read results and updates is xml.

The query uses syntax similar to xpath. A few example usages could be:

  • Read a property by config.cache.dir
  • Filter on collection of objects by app.tasks[status='idle'].id
  • Write to a single-valued property by config.screen.width = <width>800</width>
  • Write to a structural-valued property by config.audio.format = <format chan="2" rate="48.0kHz" res="24bit"/>
  • Apply a method by app.playlist.queue = <file type="mp3" loc="http://tunes.com/tune.mp3"/>

Queryable objects can choose between the following kinds of properties:

  • QueryVal for read-only access
  • QueryVar for read-write access
  • QueryDef for an applicable property (similar to write)
  • QueryPath for traversal to named object(s)

The complete sourcecode for the trait is defined as follows.

import scala.util.matching.Regex import scala.util.parsing.combinator._ /** * Query Syntax * query ::= {query-path `.`} query-leaf query-value? * query-path ::= name-path filters? * query-leaf ::= name-leaf * query-value ::= `=` <a xml="" element=""> * name-path ::= ([a-zA-Z][-a-zA-Z0-9]*) * name-leaf ::= ([a-zA-Z][-a-zA-Z0-9]*)|[?] * filters ::= `[` {filter `,`} `]` * filter ::= filter-key `=` `'` filter-value `'` * filter-key ::= [a-zA-Z][-a-zA-Z0-9]* * filter-value ::= `'` `'` */ trait Queryable { import Queryable._ private val queryKeys: () => Seq[xml.Elem] = () => { for (item <- queryMap.keys.toSeq) yield { } } protected var queryMap = Map[String,(String,QueryNode)]( "?" -> ( "Lists all queryable keys." -> QueryVal( read = queryKeys ) ) ) final def query(expr: String): Either[String,xml.Elem] = { try { val parser = new QueryParser parser.parseAll(parser.queryPath, expr) match { case parser.Success((path,value), remaining) => Right({ query(path,value) }) case parser.Error(msg, remaining) => Left("error while parsing query: " + msg) case parser.Failure(msg, remaining) => Left("failure while parsing query: " + msg) case err => Left("unknown error while parsing query: " + err) } } catch { case exp: Exception => Left("exception caught during query: " + exp) } } private def query(path: Seq[Prop], value: Option[xml.Elem]): Seq[xml.Elem] = { path.toList match { case (key,filters) :: Nil if queryMap contains key => (queryMap(key)._2,value) match { case (QueryVal(read),None) => read() case (QueryVar(read,_),None) => read() case (QueryVar(_,write),Some(value)) => write(value) case (QueryDef(apply),Some(value)) => apply(value) case _ => Nil } case (key,filters) :: tail if queryMap contains key => queryMap(key)._2 match { case QueryPath(next) => next() .filter(node => filters.forall(item => contains(node,item))) .flatMap(node => node.query(tail,value)) case _ => Nil } case _ => Nil } } private def contains(node: Queryable, check: (String,String)): Boolean = { if (node.queryMap.contains(check._1) == false) { false } else { val regex = new Regex(check._2) node.queryMap(check._1)._2 match { case QueryVal(read) => read().headOption.exists( item => regex.findPrefixOf(item.text).isDefined ) case QueryVar(read,_) => read().headOption.exists( item => regex.findPrefixOf(item.text).isDefined ) case _ => false } } } } object Queryable { import xml.XML.{loadString => toXml} private type Prop = (String,Map[String,String]) sealed trait QueryNode { def typeId(): String } case class QueryVal(read: () => Seq[xml.Elem]) extends QueryNode { def typeId(): String = "val" } case class QueryVar(read: () => Seq[xml.Elem], write: xml.Elem => Seq[xml.Elem]) extends QueryNode { def typeId(): String = "var" } case class QueryDef(apply: xml.Elem => Seq[xml.Elem]) extends QueryNode { def typeId(): String = "def" } case class QueryPath(next: () => Seq[Queryable]) extends QueryNode { def typeId(): String = "path" } private class QueryParser extends JavaTokenParsers { def queryPath: Parser[(Seq[Prop],Option[xml.Elem])] = repsep(query, ".") ~ opt(white ~ "=" ~> white ~> queryValue) ^^ { case path ~ value => (path,value) } def query: Parser[Prop] = queryName ~ opt(queryFilters) ^^ { case name ~ Some(filters) => (name,filters) case name ~ None => (name,Map()) } def queryName: Parser[String] = """([a-zA-Z][-a-zA-Z]*)|[\?]""".r def queryFilters: Parser[Map[String,String]] = "[" ~ white ~> repsep(queryFilter, white ~> "," <~ white) <~ "]" ^^ { (Map() ++ _) } def queryFilter: Parser[(String,String)] = filterKey ~ "=" ~ filterValue ^^ { case key ~ "=" ~ value => (key,value) } def queryValue(): Parser[xml.Elem] = new Parser[xml.Elem] { def apply(in: Input): ParseResult[xml.Elem] = { // manually parse xml element from text val (begin,end) = (in.offset,in.source.length) if (begin >= end) { Error("Empty value", in) } else { val elem = toXml(in.source.subSequence(begin,end).toString) Success(elem, in.drop(end - begin)) } } } def filterKey: Parser[String] = """[a-zA-Z][-a-zA-Z]*""".r def filterValue: Parser[String] = "'" ~> """[^']+""".r <~ "'" ^^ { (value: String) => value } def white: Parser[Option[String]] = opt("""[ \t]+""".r) } }
Code language: Scala (scala)

An example usage

For the demonstration, lets define a few classes that implements the Queryable trait.

case class Song(title: String, composer: String, duration: String) extends Queryable { import Queryable._ queryMap += ( "title" -> ("The song's title." -> QueryVal( read = () => { :: Nil} )), "composer" -> ("The song's composer." -> QueryVal( read = () => {{composer} :: Nil} )), "xml" -> ("Transform into xml." -> QueryVal( read = () => { :: Nil} )) ) } class Jukebox extends Queryable { import Queryable._ private var status = "idle" private var songs = List.empty[Song] def queue(song: Song) { songs = song :: songs } queryMap += ( "xml" -> ("Transform into xml." -> QueryVal( read = () => { :: Nil} )), "status" -> ("Status of jukebox." -> QueryVal( read = () => {{status} :: Nil} )), "songs" -> ("Traverse to songs." -> QueryPath( next = () => {songs} )), "queue" -> ("Queues a song from xml." -> QueryDef( apply = (elem: xml.Elem) => { val song = Song( title = (elem \ "@title").text, composer = (elem \ "@composer").text, duration = (elem \ "@duration").text ) songs = song :: songs done :: Nil } )) ) } class App extends Queryable { import Queryable._ var status = "ready" val jukebox = new Jukebox queryMap += ( "xml" -> ("Transform into xml." -> QueryVal( read = () => { :: Nil} )), "jukebox" -> ("Traverse to jukebox." -> QueryPath( next = () => {jukebox :: Nil} )) ) }
Code language: Scala (scala)

Before demonstrating the Queryable trait using the App, Jukebox, and Song classes, lets also create an utility function for printing out the Either values that Queryable.query() returns.

def print(value: Either[String,xml.Elem]) { val printer = new xml.PrettyPrinter(width=80,step=2) value match { case Right(result) => println(printer format result) case Left(error) => println(error) } }
Code language: Scala (scala)

Lets start by creating the app and query which properties it and the jukebox has.

val app = new App print(app query "?") print(app query "jukebox.?")
Code language: Scala (scala)
<query expr="?"> <key doc="Lists all queryable keys." type="val" name="?"></key> <key doc="Transform into xml." type="val" name="xml"></key> <key doc="Traverse to jukebox." type="path" name="jukebox"></key> </query> <query expr="jukebox.?"> <key doc="Queues a song from xml." type="def" name="queue"></key> <key doc="Traverse to songs." type="path" name="songs"></key> <key doc="Transform into xml." type="val" name="xml"></key> <key doc="Status of jukebox." type="val" name="status"></key> <key doc="Lists all queryable keys." type="val" name="?"></key> </query>
Code language: HTML, XML (xml)

Now, lets add some songs programatically and query the jukebox for status and songs.

app.jukebox.queue(Song(title="Mars", composer="Gustav Holst", duration="7m05s")) app.jukebox.queue(Song(title="Jupiter", composer="Gustav Holst", duration="7m51s")) print(app query "jukebox.status") print(app query "jukebox.songs.xml")
Code language: Scala (scala)

And a narrow search against specific songs using filtering (note that filter values are treated as regexes).

print(app query "jukebox.songs[title='Ma',composer='Gustav'].xml")
Code language: Scala (scala)
<query expr="jukebox.songs[title='Ma',composer='Gustav'].xml"> <song composer="Gustav Holst" title="Mars" duration="7m05s"></song> </query>
Code language: HTML, XML (xml)

Lets queue a new song on the jukebox by specifying its data in xml.

print(app query ("jukebox.queue = " + ))
Code language: Scala (scala)
<query expr="jukebox.status"> <status>idle</status> </query> <query expr="jukebox.songs.xml"> <song composer="Gustav Holst" title="Jupiter" duration="7m51s"></song> <song composer="Gustav Holst" title="Mars" duration="7m05s"></song> </query>
Code language: HTML, XML (xml)

And finally, searching for the included song.

print(app query "jukebox.songs[composer='Claude'].xml")
Code language: Scala (scala)
<query expr="jukebox.songs[title='Ma',composer='Gustav'].xml"> <song composer="Gustav Holst" title="Mars" duration="7m05s"></song> </query>
Code language: HTML, XML (xml)

In summary

Fairly little code was necessary to integrate a querying mechanism into objects. Both the parser combinators and the collection library were of great help. Now, the parser combinators aren’t know to be high performance but that is hardly a problem with the small chunks of text being parsed. There are still some places where this solution is lacking though, like filtering being restricted to single-valued properties, lack of validation, and the dependence on a specific data-format like xml. Maybe the trait should be more asynchronous? This is still work in progress and might receive some updates later on.

If you have any suggestions please drop a comment!

Leave a Reply

Your email address will not be published. Required fields are marked *