Making objects queryable in Scala

Posted on by Trond

Defining a Queryable trait

In a running program its often nice to have a way of querying and updating the various components of the application. In this example I’ll demonstrate one way you could do this in Scala by defining a Queryable trait. By inheriting this trait, an object can make available properties for reads and updates, and enable traversal to other objects. The data-format used for read results and updates is xml.

The query uses syntax similar to xpath. A few example usages could be:

  • Read a property by config.cache.dir
  • Filter on collection of objects by app.tasks[status='idle'].id
  • Write to a single-valued property by config.screen.width = <width>800</width>
  • Write to a structural-valued property by config.audio.format = <format chan="2" rate="48.0kHz" res="24bit"/>
  • Apply a method by app.playlist.queue = <file type="mp3" loc="http://tunes.com/tune.mp3"/>

Queryable objects can choose between the following kinds of properties:

  • QueryVal for read-only access
  • QueryVar for read-write access
  • QueryDef for an applicable property (similar to write)
  • QueryPath for traversal to named object(s)

The complete sourcecode for the trait is defined as follows.

import scala.util.matching.Regex
import scala.util.parsing.combinator._
 
/**
 * Query Syntax
 *   query         ::= {query-path `.`} query-leaf query-value?
 *   query-path    ::= name-path filters?
 *   query-leaf    ::= name-leaf
 *   query-value   ::= `=` <a xml element>
 *   name-path     ::= ([a-zA-Z][-a-zA-Z0-9]*)
 *   name-leaf     ::= ([a-zA-Z][-a-zA-Z0-9]*)|[?]
 *   filters       ::= `[` {filter `,`} `]`
 *   filter        ::= filter-key `=` `'` filter-value `'`
 *   filter-key    ::= [a-zA-Z][-a-zA-Z0-9]*
 *   filter-value  ::= `'` <an escaped string> `'`
 */
trait Queryable {
  import Queryable._
 
  private val queryKeys: () => Seq[xml.Elem] =
    () => {
      for (item <- queryMap.keys.toSeq) yield {
        <key name={item} doc={queryMap(item)._1} type={queryMap(item)._2.typeId}/>
      }
    }
 
  protected var queryMap = Map[String,(String,QueryNode)](
    "?" -> (
      "Lists all queryable keys." -> QueryVal(
        read = queryKeys
      )
    )
  )
 
  final def query(expr: String): Either[String,xml.Elem] = {
    try {
      val parser = new QueryParser
      parser.parseAll(parser.queryPath, expr) match {
        case parser.Success((path,value), remaining) =>
          Right(<query expr={expr}>{ query(path,value) }</query>)
        case parser.Error(msg, remaining) =>
          Left("error while parsing query: " + msg)
        case parser.Failure(msg, remaining) =>
          Left("failure while parsing query: " + msg)
        case err =>
          Left("unknown error while parsing query: " + err)
      }
    }
    catch {
      case exp: Exception => Left("exception caught during query: " + exp)
    }
  }
 
  private def query(path: Seq[Prop], value: Option[xml.Elem]): Seq[xml.Elem] = {
    path.toList match {
      case (key,filters) :: Nil if queryMap contains key =>
        (queryMap(key)._2,value) match {
          case (QueryVal(read),None)           => read()
          case (QueryVar(read,_),None)         => read()
          case (QueryVar(_,write),Some(value)) => write(value)
          case (QueryDef(apply),Some(value))   => apply(value)
          case _                               => Nil
        }
      case (key,filters) :: tail if queryMap contains key =>
        queryMap(key)._2 match {
          case QueryPath(next) =>
            next()
              .filter(node => filters.forall(item => contains(node,item)))
              .flatMap(node => node.query(tail,value))
          case _ => Nil
        }
      case _ => Nil
    }
  }
 
  private def contains(node: Queryable, check: (String,String)): Boolean = {
    if (node.queryMap.contains(check._1) == false) {
      false
    }
    else {
      val regex = new Regex(check._2)
      node.queryMap(check._1)._2 match {
        case QueryVal(read) =>
          read().headOption.exists(
            item => regex.findPrefixOf(item.text).isDefined
          )
        case QueryVar(read,_) =>
          read().headOption.exists(
            item => regex.findPrefixOf(item.text).isDefined
          )
        case _ => false
      }
    }
  }
}
 
 
object Queryable {
  import xml.XML.{loadString => toXml}
 
  private type Prop = (String,Map[String,String])
 
  sealed trait QueryNode {
    def typeId(): String
  }
  case class QueryVal(read: () => Seq[xml.Elem]) extends QueryNode {
    def typeId(): String = "val"
  }
  case class QueryVar(read: () => Seq[xml.Elem],
                      write: xml.Elem => Seq[xml.Elem]) extends QueryNode {
    def typeId(): String = "var"
  }
  case class QueryDef(apply: xml.Elem => Seq[xml.Elem]) extends QueryNode {
    def typeId(): String = "def"
  }
  case class QueryPath(next: () => Seq[Queryable]) extends QueryNode {
    def typeId(): String = "path"
  }
 
  private class QueryParser extends JavaTokenParsers {
    def queryPath: Parser[(Seq[Prop],Option[xml.Elem])] =
      repsep(query, ".") ~ opt(white ~ "=" ~> white ~> queryValue) ^^ {
        case path ~ value => (path,value)
      }
 
    def query: Parser[Prop] =
      queryName ~ opt(queryFilters) ^^ {
        case name ~ Some(filters) => (name,filters)
        case name ~ None          => (name,Map())
      }
 
    def queryName: Parser[String] = """([a-zA-Z][-a-zA-Z]*)|[\?]""".r
 
    def queryFilters: Parser[Map[String,String]] =
      "[" ~ white ~> repsep(queryFilter, white ~> "," <~ white) <~ "]" ^^ {
        (Map() ++ _)
      }
 
    def queryFilter: Parser[(String,String)] =
      filterKey ~ "=" ~ filterValue ^^ {
        case key ~ "=" ~ value => (key,value)
      }
 
    def queryValue(): Parser[xml.Elem] =
      new Parser[xml.Elem] {
        def apply(in: Input): ParseResult[xml.Elem] = {
          // manually parse xml element from text
          val (begin,end) = (in.offset,in.source.length)
          if (begin >= end) {
            Error("Empty value", in)
          }
          else {
            val elem = toXml(in.source.subSequence(begin,end).toString)
            Success(elem, in.drop(end - begin))
          }
        }
      }
 
    def filterKey: Parser[String] = """[a-zA-Z][-a-zA-Z]*""".r
 
    def filterValue: Parser[String] =
      "'" ~> """[^']+""".r <~ "'" ^^ {
        (value: String) => value
      }
 
    def white: Parser[Option[String]] = opt("""[ \t]+""".r)
  }
}

An example usage

For the demonstration, lets define a few classes that implements the Queryable trait.

case class Song(title: String, composer: String, duration: String)
  extends Queryable {
  import Queryable._
 
  queryMap += (
    "title" -> ("The song's title." -> QueryVal(
      read = () => {<title>{title}</title> :: Nil}
    )),
    "composer" -> ("The song's composer." -> QueryVal(
      read = () => {<composer>{composer}</composer> :: Nil}
    )),
    "xml" -> ("Transform into xml." -> QueryVal(
      read = () => {<song title={title} composer={composer} duration={duration}/> :: Nil}
    ))
  )
}
 
class Jukebox extends Queryable {
  import Queryable._
 
  private var status = "idle"
  private var songs  = List.empty[Song]
 
  def queue(song: Song) { songs = song :: songs }
 
  queryMap += (
    "xml" -> ("Transform into xml." -> QueryVal(
      read = () => {<jukebox status={status} size={songs.length.toString}/> :: Nil}
    )),
    "status" -> ("Status of jukebox." -> QueryVal(
      read = () => {<status>{status}</status> :: Nil}
    )),
    "songs" -> ("Traverse to songs." -> QueryPath(
      next = () => {songs}
    )),
    "queue" -> ("Queues a song from xml." -> QueryDef(
       apply = (elem: xml.Elem) => {
         val song = Song(
           title    = (elem \ "@title").text,
           composer = (elem \ "@composer").text,
           duration = (elem \ "@duration").text
         )
         songs = song :: songs
         <queue>done</queue> :: Nil
       }
    ))
  )
}
 
class App extends Queryable {
  import Queryable._
 
  var status = "ready"
  val jukebox = new Jukebox
 
  queryMap += (
    "xml" -> ("Transform into xml." -> QueryVal(
      read = () => {<app status={status}/> :: Nil}
    )),
    "jukebox" -> ("Traverse to jukebox." -> QueryPath(
      next = () => {jukebox :: Nil}
    ))
  )
}

Before demonstrating the Queryable trait using the App, Jukebox, and Song classes, lets also create an utility function for printing out the Either values that Queryable.query() returns.

def print(value: Either[String,xml.Elem]) {
  val printer = new xml.PrettyPrinter(width=80,step=2)
  value match {
    case Right(result) => println(printer format result)
    case Left(error)   => println(error)
  }
}

Lets start by creating the app and query which properties it and the jukebox has.

val app = new App
print(app query "?")
print(app query "jukebox.?")
<query expr="?">
  <key doc="Lists all queryable keys." type="val" name="?"></key>
  <key doc="Transform into xml." type="val" name="xml"></key>
  <key doc="Traverse to jukebox." type="path" name="jukebox"></key>
</query>
<query expr="jukebox.?">
  <key doc="Queues a song from xml." type="def" name="queue"></key>
  <key doc="Traverse to songs." type="path" name="songs"></key>
  <key doc="Transform into xml." type="val" name="xml"></key>
  <key doc="Status of jukebox." type="val" name="status"></key>
  <key doc="Lists all queryable keys." type="val" name="?"></key>
</query>

Now, lets add some songs programatically and query the jukebox for status and songs.

app.jukebox.queue(Song(title="Mars", composer="Gustav Holst", duration="7m05s"))
app.jukebox.queue(Song(title="Jupiter", composer="Gustav Holst", duration="7m51s"))
print(app query "jukebox.status")
print(app query "jukebox.songs.xml")
<query expr="jukebox.status">
  <status>idle</status>
</query>
<query expr="jukebox.songs.xml">
  <song composer="Gustav Holst" title="Jupiter" duration="7m51s"></song>
  <song composer="Gustav Holst" title="Mars" duration="7m05s"></song>
</query>

And a narrow search against specific songs using filtering (note that filter values are treated as regexes).

print(app query "jukebox.songs[title='Ma',composer='Gustav'].xml")
<query expr="jukebox.songs[title='Ma',composer='Gustav'].xml">
  <song composer="Gustav Holst" title="Mars" duration="7m05s"></song>
</query>

Lets queue a new song on the jukebox by specifying its data in xml.

print(app query ("jukebox.queue = " + <song title="Nocturnes III" composer="Claude Debussy" duration="10m40s"/>))
<query
expr="jukebox.queue = &lt;song composer=&quot;Claude Debussy&quot; title=&quot;Nocturnes III&quot; duration=&quot;10m40s
&quot;&gt;&lt;/song&gt;">
  <queue>done</queue>
</query>

And finally, searching for the included song.

print(app query "jukebox.songs[composer='Claude'].xml")
<query expr="jukebox.songs[composer='Claude'].xml">
  <song composer="Claude Debussy" title="Nocturnes III" duration="10m40s"></song>
</query>

In summary

Fairly little code was necessary to integrate a querying mechanism into objects. Both the parser combinators and the collection library were of great help. Now, the parser combinators aren’t know to be high performance but that is hardly a problem with the small chunks of text being parsed. There are still some places where this solution is lacking though, like filtering being restricted to single-valued properties, lack of validation, and the dependence on a specific data-format like xml. Maybe the trait should be more asynchronous? This is still work in progress and might receive some updates later on.

If you have any suggestions please drop a comment!

Leave a Reply

Your email address will not be published. Required fields are marked *