logji

The magical Specs/ScalaCheck defaultPrettyParams configuration implicit

2011-08-17T15:04:00.001-06:00

When using Specs with ScalaCheck, I have occasionally found myself annoyed that, when a function under test threw an exception, ScalaCheck didn't show me enough of a stack trace to actually figure out what was going on. There is a means to configure this, but to the best of my knowledge it's essentially not documented anywhere. In any case, to get full stack traces in your spec, all that you need is this:


import org.specs._

import org.scalacheck._



class MySpec extends Specification with ScalaCheck {

  override val defaultPrettyParams = Pretty.Params(2)



  "your scalachecky specs here" should {

     //...

  }

}

The magical '2' in Pretty.Params(2) tells Scalacheck to spit out the full stack trace instead of just the error message from the exception.

Code Retreat Boulder: Conway's Life

2011-02-27T09:17:00.002-07:00

I had a ton of fun yesterday at Code Retreat Boulder, even though I ended up as the Scala guy and thus didn't get to play in other languages as much as I would have liked.Anyway, this morning I decided to recreate my favorite of the solutions. Small, simple, pure, and relatively flexible:


import scala.annotation.tailrec

trait Life {
  type Entity
  type Population = Set[Entity]

  def generation(pop: Population) = pop.flatMap(neighborhood).filter {
    entity => willLive(pop.contains(entity), neighbors(entity).count(pop.contains))
  }

  def neighbors(entity: Entity): Seq[Entity]
  def neighborhood(entity: Entity) = neighbors(entity) :+ entity

  def willLive(now: Boolean, n: Int): Boolean
  def render(pop: Population): Unit = println(pop)

  @tailrec final def run(pop: Population): Unit = { 
    render(pop)
    if (!pop.isEmpty) run(generation(pop))
  }
}

class Life2D extends Life {
  type Entity = (Int, Int)

  override def willLive(now: Boolean, n: Int) = ((now && n == 2) || n == 3)

  override def neighbors(entity: Entity) = entity match {
    case (x, y) => for (i <- x-1 to x+1; j <- y-1 to y+1 if !(i == x && j == y)) yield (i, j)
  }
}

object Blinker extends Life2D {
  def main(argv: Array[String]) = run(Set((1, 0), (1, 1), (1, 2)))
}

A monad for failure-tolerant computations.

2010-07-29T16:47:00.005-06:00

I was working on a problem today where some of the computations could fail, but degrade gracefully while providing information about how exactly they failed so that clients could choose whether or not to use the result. This is what I came up with:


/**
 * A monad used to entrain computations on a value that might fail.
 * This is distinct from Either in that it allows computations to continue
 * in the presence of errors, with the possibility of a degraded result.
 * The user of the result can then decide its suitability for use based upon
 * any errors that might be returned. Errors must form a semigroup.
 */
sealed trait Attempt[E, +A] {
  def value: A
  def map[B](f: A => B): Attempt[E, B]
  def flatMap[B](f: A => Attempt[E, B]): Attempt[E, B]

  def error: Option[E]
  def either: Either[E, A]
}

case class Success[E, +A](value: A) extends Attempt[E, A] {
  def map[B](f: A => B): Attempt[E, B] = Success(f(value))
  def flatMap[B](f: A => Attempt[E, B]): Attempt[E, B] = f(value)
  def error = None
  def either = Right(value)
}

case class Failure[E, +A](err: E, value: A)(implicit append: (E, E) => E) extends Attempt[E, A] {
  def map[B](f: A => B): Attempt[E, B] = Failure(err, f(value))
  def flatMap[B](f: A => Attempt[E, B]): Attempt[E, B] = f(value) match {
    case Failure(e, b) => Failure(append(e, err), b)
    case Success(b)   => Failure(err, b)
  }

  def error = Some(err)
  def either = Left(err)
}

Pretty trivial, but maybe occasionally useful. Here's how it looks in my code:


  def mergeComponents(f: String => JSONReport): Attempt[List[String], JSONReport] = {
    components.map(f).map(_.mergeComponents(f)).foldLeft[Attempt[List[String], JSONReport]](Success(this)) {
      (result, comp) => for {
        cur <- result
        jr  <- comp
        properties     <- mergeProperties(cur, jr)
        queries        <- mergeQueries(cur, jr)
        dataAliases    <- mergeAliases(cur, jr, queries.values)
        dataTransforms <- mergeTransforms(cur, jr, dataAliases.values)
        dataMailers    <- mergeMailers(cur, jr, dataTransforms.values)
      } yield {
        JSONReport(
          cur.reportId, cur.active, cur.version, properties,
          queries, dataAliases, dataTransforms, dataMailers,
          cur.dataRange, Nil
        )
      }
    }

Here each of the various mergeX methods return an Attempt[List[String], X] where X is something needed to build the ultimate report. Here I'm just aggregating lists of strings describing the errors, but of course any type for which (E, E) => E is defined would work.

Attempt. For all those times where you want a monad that says, "Hey, I maybe couldn't get exactly what you asked for. Maybe it's little broken, maybe it won't work right, but this the best I could do. You decide."

EDIT:

After a bit of thinking, I realized that this monad is really more general than being simply related to success or failure - it simply models a function that may or may not produce some additional metadata about its result. Then a lightbulb went off, and quick google search confirmed... yup, I just reinvented the writer monad. It's not *exactly* like Writer, because it just requires a semigroup for E instead of a monoid, and the presence of a "log" is optional, so maybe it's better suited than Writer for a few instances.

The one really nice thing about rediscovering well-known concepts is that doing the derivation for yourself, in the context of some real problem, gives you a much more complete understanding of where the thing you reintevented is applicable!

Inspired by michid's example in the comments below, here's a simpler example that demonstrates some utility his doesn't quite capture.


implicit val append = (l1: List[String], l2: List[String]) => l1 ::: l2

def succ(s: String) = Success[List[String], String]("result: success")
val fail: String => Attempt[List[String], String] = { 
  var count = 0 
  (s: String) => {
    count += 1
    Failure(List("failure " + count), s + " then failure")
  }
}

val r = for {
  x <- succ("here we go!")
  y <- fail(x)
  z <- fail(y)
} yield z

println(r)

This results in:


Failure(List(failure 2, failure 1),result: success then failure then failure)

An unfortunate situation.

2010-07-15T13:22:00.000-06:00

I've been working a bit with Hadoop lately. As many will know, there are essentially two versions of Hadoop that coexist within the Hadoop API; a deprecated legacy API and a newer one that is not nearly so feature-complete. It's a source of confusion to say the least.

Earlier today, I asked in the #hadoop channel on freenode, why were the breaking changes not simply made on a separate branch, the major version number incremented, and so forth such that everyone could move forward? The answers that I got back illuminated a troublesome trend that I think is worth considering for those projects (like my beloved Scala) that are moving towards professional support models of funding.

During the discussion, I commented that there is an unfortunate reality that the largest companies, who may provide the greatest support any given open-source project are also those who will likely be most conservative with respect to change. So as soon as open-source projects grow large enough that companies selling commercial support for those projects survive, there is an immediate disincentive for the support companies to make breaking changes. And stagnation ensues.

The response was startling. People who had been involved with numerous prominent businesses and projects immediately replied with stories of their own projects suffering such a fate, sometimes to the point that the project died entirely.

I think that this "success effect" can be traced to a single, simple cause, with a single, simple solution. Companies (and the individuals that make them up) need to stop thinking of mission-critical software as something that is ever complete. Marketing is never complete; sales is never done with their job, purchasing and HR are never done, nor is accounting. Software is no different - as engineers, we do not (and should not purport to) produce solutions to the problems that companies have; instead, we automate and support the operation of businesses so that businesses can do more, serve more customers, and in the end create more wealth. A project is finished when nobody is using it; never before then.

Trick of the day

2010-02-05T14:35:00.002-07:00

James Iry just mentioned something I'd never noticed before in #scala:


scala> def f(i: Int, x: Int) = i match { case `x` => println("x!"); case _ => println("nope") }
f: (i: Int,x: Int)Unit

scala> f(1, 1)
x!

scala> f(1, 2)
nope

That's one of those things that I'd idly wondered how to do, but hadn't had a use case so hadn't really gone looking. Good to know it's possible and simple... and more or less makes sense.More or less. Actually, I guess I'm kind of glad that the following doesn't compile:


scala> def f(i: Int) = i match { case `yield` => println(`yield`) }                           
<console>:4: error: not found: value yield
       def f(i: Int) = i match { case `yield` => println(`yield`) }

I can live with the restriction as not being able to use a reserved word for a variable on the lhs of a pattern match, though it does make for a weird inconsistency:


scala> val (`yield`, x) = (1, 1)
<console>:4: error: not found: value yield
       val (`yield`, x) = (1, 1)
            ^

scala> val `yield` = 1
yield: Int = 1

Chalk up one more to the box of "pain points caused by using pattern matching for parallel assignment."

On Extraction

2010-02-03T14:57:00.003-07:00

Scala case classes are great; they give you a tremendous leg up when it comes to implementing pattern matching. However, there's a drawback: the unapply method of case class companion objects take arguments of the type of the case class, and return Option[TupleN[...]] where N is the number of variables to the case class constructor. What's more, it's not possible to overload or override the unapply method to deconstruct other types of by creating your own companion object on the side, as shown in this REPL session:


scala> case class  Foo(val i: Int, val s: String)
defined class Foo

scala> object Foo {
     |   def unapply(s: String): Option[Foo] = Some(new Foo(s.substring(0, 2).toInt, s.substring(2)))
     | }
defined module Foo

scala> "12abcd" match {
     |   case Foo(i, s) => println(s, i)
     | }
<console>:10: error: wrong number of arguments for object Foo
         case Foo(i, s) => println(s, i)
              ^
<console>:10: error: wrong number of arguments for method println: (Any)Unit
         case Foo(i, s) => println(s, i)

Sadly, it doesn't even seem to be possible to move this extractor to another class and get the desired result:


scala> object SFoo {                                                                                 
     |   def unapply(s: String): Option[Foo] = Some(new Foo(s.substring(0, 2).toInt, s.substring(2)))
     | }
defined module SFoo

scala> "12abcd" match {                 
     |   case SFoo(i, s) => println(s, i)
     | }
<console>:9: error: wrong number of arguments for object SFoo
         case SFoo(i, s) => println(s, i)
              ^
<console>:9: error: wrong number of arguments for method println: (Any)Unit
         case SFoo(i, s) => println(s, i)

Fortunately, there is a way around this issue: simply define your own class that extends ProductN! Then, you can define as many extractors as you want in different pseudo-companion objects and use them idiomatically:


scala> class Foo(val i: Int, val s: String) extends Product2[Int, String] {
     |   val (_1, _2) = (i, s)
     | }
defined class Foo

scala> object Foo {
     |   def unapply(f: Foo): Option[(Int, String)] = Some((f.i, f.s))
     | }
defined module Foo

scala> object SFoo {
     |   def unapply(s: String): Option[Foo] = Some(new Foo(s.substring(0, 2).toInt, s.substring(2)))
     | }
defined module SFoo

scala> "12abcd" match {
     |   case SFoo(i, s) => println(s, i)
     | }
(abcd,12)

scala> new Foo(1, "hi") match {
     |   case Foo(i, s) => println(s, i)
     | }
(hi,1)

Also, naturally, the trivial extractor (Foo => Foo) can supplant the Foo => Tuple2 extractor above:


scala> object Foo {
     |   def unapply(f: Foo): Option[Foo] = Some(f)
     | }

scala> new Foo(1, "hi") match {                  
     |   case Foo(i, s) => println(s, i)         
     | }                                
(hi,1)

A monadic Visitor

2009-12-08T11:15:00.004-07:00

This Reader-esque monad emerged from my code this morning:


trait A {
  def accept[T](v: V[T]): T
}

class B(val s: String) extends A {
  var i = 0
  override def accept[T](v: V[T]): T = v.visit(this)
}

class C(val s: String) extends A {
  var i = 0
  override def accept[T](v: V[T]): T = v.visit(this)
}

class D(val s: String) extends A {
  var i = 0
  override def accept[T](v: V[T]): T = v.visit(this)
}

trait V[T] {
  outer =>
  def visit(b: B): T
  def visit(c: C): T
  def visit(d: D): T

  def map[U](f: T => U): V[U] = new V[U] {
    def visit(b: B): U = f(outer.visit(b))
    def visit(c: C): U = f(outer.visit(c))
    def visit(d: D): U = f(outer.visit(d))
  }

  def flatMap[U](f: T => V[U]): V[U]= new V[U] {
    def visit(b: B): U = f(outer.visit(b)).visit(b)
    def visit(c: C): U = f(outer.visit(c)).visit(c)
    def visit(d: D): U = f(outer.visit(d)).visit(d)
  }
}

object V {
  def pure[T](t: T): V[T] = new V[T] {
    def visit(b: B) = t
    def visit(c: C) = t
    def visit(d: D) = t
  }
}

And the test:


object Test {
  def main(argv: Array[String]) {
    class VI(i: Int) extends V[Int] {
      def visit(b: B) = error("not supported")

      def visit(c: C) = {
        println("vi: " + c.s + " = " + c.i)
        c.i = c.i + i
        c.i
      }

      def visit(d: D) = {
        println("vi: " + d.s + " = " + d.i)
        d.i = d.i + i
        d.i
      }
    }

    def f(i: Int): V[Int] = {
      println("f: " + i)
      new VI(i)
    }

    def g(i: Int): V[Int] = {
      println("g: " + i)
      new VI(i)
    }

    val c1: A = new C("c1")
    val c2: A = new C("c2")
    val d1: A = new D("d1")
    val v = new VI(1)
    println(c1.accept(v.flatMap(f).flatMap(g)))
    println(c2.accept(v.flatMap(x => f(x).flatMap(g))))
    println(d1.accept(v.flatMap(f).flatMap(g).flatMap(f).flatMap(g)))
  }
}

As far as I can tell, this satisfies the monad laws. Am I missing anything? If not, this is the first monad I've "found in the wild!"

So much from so little

2009-12-02T21:30:00.003-07:00

My solution to Tony's catamorphism challenge:


trait MyOption[+A] {
  import MyOption._

  def cata[X](some: A => X, none: => X): X
  def map[B](f: A => B): MyOption[B] = cata(a => some(f(a)), none[B])
  def flatMap[B](f: A => MyOption[B]): MyOption[B] = cata(f, none[B])
  def getOrElse[AA >: A](e: => AA): AA = cata(a => a, e)
  def filter(p: A => Boolean): MyOption[A] = if (cata(p, false)) cata(a => some(a), none[A]) else none[A]
  def foreach(f: A => Unit): Unit = cata(f, ())
  def isDefined: Boolean = cata(_ => true, false)
  def isEmpty: Boolean = cata(_ => false, true)
  def get: A = cata(a => a, error("get on none"))
  def orElse[AA >: A](o: MyOption[AA]): MyOption[AA] = cata(a => some(a), o)
  def toLeft[X](right: => X): Either[A, X] = cata(a => Left(a), Right(right))
  def toRight[X](left: => X): Either[X, A] = cata(a => Right(a), Left(left))
  def toList: List[A] = cata(a => List(a), Nil)
  def iterator: Iterator[A] = toList.elements
}

object MyOption {
  def none[A] = new MyOption[A] {
    def cata[X](s: A => X, n: => X) = n
  }

  def some[A](a: A) = new MyOption[A] {
    def cata[X](s: A => X, n: => X) = s(a)
  }
}

Composable Bindings in Lift, part II

2009-11-07T15:02:00.003-07:00

In my previous post I presented the DataBinding[T] type (T => NodeSeq => NodeSeq) and illustrated how classes inhabiting this type could be used along with an implicit binder function to create modular, reusable rendering logic for objects. In this post, I will discuss how to assemble these components and strategies for handling rendering over inheritance hierarchies.

I left off last time with this DataBinding trait:


trait OrderBinding extends DataBinding[Order] {
  implicit val userBinding: DataBinding[User]
  implicit val addressBinding: DataBinding[Address]
  implicit val lineBinding: DataBinding[OrderLine]
  implicit val txnBinding: DataBinding[Transaction]

  def apply(order: Order): Binding = (xhtml: NodeSeq) => {
    val itemTemplate = chooseTemplate("order", "lineItem", xhtml)
    val txnTemplate = chooseTemplate("order", "transaction", xhtml)

    bind("order", xhtml,
      "user" -> order.user.bind("templates-hidden/user/order-display".path),
      "shipTo" -> order.shipTo.bind("templates-hidden/address/order-display".path),
      "lineItems" -> order.items.flatMap(_.bind(itemTemplate)).toSeq,
      "transactions" -> order.transactions.flatMap(_.bind(txnTemplate)).toSeq
    )
  }
}

This DataBinding[Order] is a trait, not an object like the previous binding instances. In addition, the other DataBinding instances that it delegates to are abstract implicit vals - meaning that I will need to create a concrete implementation of this trait to actually use it. The interesting thing here is that these vals are declared implicit within the scope of the trait, but do not need to be declared implicit when the trait is extended and made concrete. I can then compose what I want my binding to look like at the use site (say an ordinary snippet) like this:


import Bindings._

class MySnippet {
  private implicit object ShowOrderBinding extends OrderBinding { 
    val userBinding = UserBinding
    val addressBinding = MailingAddressBinding
    val lineBinding = OrderLineBinding
    val txnBinding = TransactionBinding //we'll implement this in a minute
  }

  def showOrder(xhtml: NodeSeq): NodeSeq = {
    val order: Order = //get an order from somewhere
    order.bind(xhtml)
  }
}

The ShowOrderBinding selects the bindings I want for the components of the order in a declarative fashion, and since it is an implicit object in scope, it is selected as the DataBinding instance to render the order when order.bind is called. If I decide that I want to use the AdminAddressBinding from the previous post instead, I simply substitute it for MailingAddressBinding at the use site and I'm done!

There's one more thing that needs to be done in the above code - to implement TransactionBinding. This is a little tricky because there are several subclasses of Transaction that may be in the list of transactions for an order, so I'm going to need to recover type information about those instances at runtime in order to be able to choose the correct binding for a given Transaction. Furthermore, there's some information in the base Transaction type that I don't want to have to repeat bindings for in each subtype binding, so I'm going to use function composition to "add together" two different bindings.

There are two different approaches I'm going to show for how one might go about doing this: first, I'll try using a straightforward Scala match statement to figure out the object's type. That looks like this:


trait TransactionBinding extends DataBinding[Transaction] {
  implicit val authBinding: DataBinding[AuthTransaction]
  implicit val captureBinding: DataBinding[CaptureTransaction]
  implicit val shipBinig: DataBinding[ShipTransaction]

  def apply(txn: Transaction): Binding = {
    val base: Binding = bind("txn", _,
      "scheduleDate" -> Text(txn.scheduledAt.toString),
      "completionDate" -> Text(txn.completedAt.getOrElse("(incomplete)").toString),
      "state" -> Text(
        txn.state match {
          case COMPLETE => "complete"
          case _ => "weird"
        }
      )
    )

    base compose (
      txn match {
        case auth: AuthTransaction => auth.binding
        case capture: CaptureTransaction => capture.binding
        case ship: ShipTransaction => ship.binding
        case _ => error("Unrecognized transaction type!")
      }
    )
  }
}

Here, I've created a base binding that defines some simple behavior for how to bind elements that are common to all transactions. I then compose that generated binding with a binding for the appropriate subclass. The determination of the subclass binding is of necessity a bit boilerplate-laden, but that's what happens when one throws away compile-time type information. The alternative, of course, would be to store each type of transaction in a separate collection associated with the order, but that's not the way I've gone for now.

There is, however, a hidden bug that may lurk in the pattern match above, depending upon your environment. Let's supposed for a minute that our model classes are persisted by Hibernate - and that the list of Transactions is annotated such that it is lazily, instead of eagerly, fetched from the database. In this case, the transactions returned may be translucently wrapped in runtime proxies. I say translucently instead of transparently because there is one critical respect in which a proxy does not behave in the same manner as the class being proxied - how it responds to the instanceof check that underlies the Scala pattern match. In this case, we will need to use a Visitor to correctly get the type for which we want to delegate binding. To do this, we'll modify our transaction classes as so:


trait TransactionFunc[T] {
  def apply(auth: AuthTransaction): T
  def apply(capture: CaptureTransaction): T
  def apply(ship: ShipTransaction): T
}

abstract class Transaction(val scheduledAt: DateTime, val completedAt: Option[DateTime], state: State) {
  def accept[T](f: TransactionFunc[T]): T
}

case class AuthTransaction(cc: CreditCard, amount: BigDecimal, scheduledAt: DateTime, completedAt: Option[DateTime], state: State) extends Transaction(scheduledAt, completedAt) {
  def accept[T](f: TransactionFunc[T]): T = f.apply(this)
}

case class CaptureTransaction(auth: AuthTransaction, ...) extends Transaction(...)  {
  def accept[T](f: TransactionFunc[T]): T = f.apply(this)
}

case class ShipTransaction(items: List[OrderLine], ...) extends Transaction(...)  {
  def accept[T](f: TransactionFunc[T]): T = f.apply(this)
}

Then, the implementation of our transaction binding becomes this:


trait TransactionBinding extends DataBinding[Transaction] {
  implicit val authBinding: DataBinding[AuthTransaction]
  implicit val captureBinding: DataBinding[CaptureTransaction]
  implicit val shipBinig: DataBinding[ShipTransaction]

  def apply(txn: Transaction): Binding = {
    val base: Binding = //same as before...

    base compose txn.accept(
      new TransactionFunc[Binding] {
        def apply(auth: AuthTransaction) = auth.binding
        def apply(capture: CaptureTransaction) = capture.binding
        def apply(ship: ShipTransaction) = ship.binding
      }
    )
  }
}

Since the correct overloading of TransactionFunc.apply will be statically determined, there is no possibility of a proxy getting in the way of our binding.

So, now we have a set of tools for binding composition. Here are some additional ideas of where this might be able to go, given the appropriate changes to Lift.

In my example of the showOrder snippet above, the implementation is trivial... so trivial, in fact, that it almost doesn't need to be there. If the notion of a binding was fully integrated into Lift, we could make Loc somewhat more powerful. Loc is one of the primary classes in Lift used to generate the SiteMap, which declares what paths are accessible within a Lift application. A path in a Loc typically corresponds to the path to an XML template that will specify the snippets and markup for a request. An important thing to note though is that Loc is not simply Loc - it is in fact Loc[T], where by default T is NullLocParams, a meaningless object essentially equivalent to Unit.

When one looks at making T something other than Unit, Loc starts to become significantly more interesting - first, it declares a rewrite method that can be overriden to populate the Loc with an instance of T as part of its processing, and secondly, Loc can specify explicitly the template to be used to render the Loc, instead of using the one that corresponds to the path. Wait... we have an instance, and we have a template... what if we could specify a DataBinding[T] as well? In this case, we might not need for there to be a snippet at all!

Composable bindings in Lift

2009-09-29T17:53:00.010-06:00

I've been using Lift as the framework behind a small internal web application at my company for the past year or so. The experience has been generally positive, despite a few quibbles I have with how state is typically maintained and passed around within an application.

Lift takes a "view first" approach to building web applications, as opposed to the commonly favored MVC pattern. In practice, "view first" boils down to essentially this: when you create an XHTML template for a page, that template will contain one or more XML tags where the name of the tag refers to a method to be called with the body of the tag as an argument. Such a method will return the XHTML to be substituted in place of the original tag and its contents. In Lift parlance such a method is called a "snippet," and has the type NodeSeq => NodeSeq. In order to more fully understand the rest of this article, I recommend you read more about snippets here if you're not already familiar with them.

The interesting thing about this signature is that functions with this type can easily be composed. What's more, the method most frequently used to implement snippets, BindHelpers.bind, can be partially applied to its arguments to give a closure with the same signature.

When I'm designing a website, most of the time rendering a page (or even a snippet within a page) isn't about displaying the data from a single object - instead, I often want to render a whole object graph. Ideally, I want to be able to choose a rendering for each type of object I'm concerned with, then compose these elements to produce the final page. Templates for objects should be reusable and modular, and for any given page I should be able to pick between multiple renderings of any given object, in some places displaying a concise summary and elsewhere a detailed exposition of all the available data.

Furthermore, while I believe that "the meaning should belong with the bytes" I also think that the rendering should be as decoupled from the bytes as possible to allow for maximum flexibility. Hence, I don't want my model classes to have any sort of "render" method - the display is something that should be layered on above the semantics provided by the names and types of the members of the object.

With these goals and composition in mind, I started with the following tiny bit of code:


import net.liftweb.http.TemplateFinder.findAnyTemplate
import net.liftweb.util.{Box,Full,Empty,Failure,Log}
import scala.xml._

object Bindings {
   type Binding = NodeSeq => NodeSeq

   type DataBinding[T] = T => NodeSeq => NodeSeq

   //adds a bind() function to an object if an implicit DataBinding is available for that object
   implicit def binder[T](t: T)(implicit binding: DataBinding[T]): Binder = Binder(binding(t))
   implicit def binder(binding: Binding): Binder = Binder(binding)

   //decorator for a binding function that allows it to be called as bind() rather than apply()
   //and also provides facilities for binding to a specific template
   case class Binder(val binding: Binding) {
       def bind(xhtml: NodeSeq): NodeSeq = binding.apply(xhtml)
       def bind(templatePath: List[String]): NodeSeq = {
           findAnyTemplate(templatePath) map binding match {
               case Full(xhtml) => xhtml
               case Failure(msg, ex, _) => Text(ex.map(_.getMessage).openOr(msg))
               case Empty => Text("Unable to find template with path " + templatePath.mkString("/", "/", ""))
           }
       }
   }

   object NullBinding extends Binding {
       override def apply(xhtml : NodeSeq) : NodeSeq = NodeSeq.Empty
   }

   object StringBinding extends DataBinding[String] {
       override def apply(msg: String) = (xhtml: NodeSeq) => Text(msg)
   }

   implicit def path2list(s: String) = new {
     def path: List[String] = s.split("/").toList
   }
}

The principle idea is that in my application, I will create one or more DataBinding instances for each of my model types, and that then by simply declaring an implicit val containing the instance I want in the narrowest scope possible, I can call .bind on any object for which an implicit DataBinding is available, or .binding if I want to compose multiple such binding functions and close over the model instance. Furthermore, this gives me good ways to compose both over composition and inheritance hierarchies, as I'll show below.

Consider that I have the following simplified, store-oriented model objects. In Lift, these would probably be implemented using Mapper or JPA - in the app that I've been working on, these are Java classes persisted by Hibernate - certainly not things that I want to change in order to support rendering by Scala.


case class User(fname: String, lname: String, email: String, joinedOn: DateTime)
case class Address(
  recipient: String, 
  addr1: String, 
  addr2: String, 
  city: String, 
  state: String, 
  country: String, 
  postalCode: String
)
case class Product(name: String, basePrice: BigDecimal, shippingCost: Option[BigDecimal])
case class OrderLine(product: Product, quantity: Int)
class Order(
  val user: User, 
  val placedOn: DateTime, 
  val shipTo: Address, 
  var items: List[OrderLine], 
  var transactions: List[Transaction]
)

sealed trait State
object PENDING extends State
object IN_PROGRESS extends State
object COMPLETE extends State
object FAILED extends State
object CANCELED extends State

abstract class Transaction(val scheduledAt: DateTime, val completedAt: Option[DateTime], state: State)
case class AuthTransaction(
  cc: CreditCard, 
  amount: BigDecimal, 
  scheduledAt: DateTime, 
  completedAt: Option[DateTime], 
  state: State
) extends Transaction(scheduledAt, completedAt)
case class CaptureTransaction(auth: AuthTransaction, ...) extends Transaction(...)
case class ShipTransaction(items: List[OrderLine], ...) extends Transaction(...)

With this set of models, I have a few different ways that they will be rendered - the display to the users will not be the same as the display to site administrators, I may (or may not) want to reuse the same rendering code for a line item in an order as in a shipping transaction, and so forth. Let's take a look at how this ends up.

First, I want to create some DataBinding objects for my classes that are built just on primitive types. Let's start with User.


import net.liftweb.util.Helpers._ //this is where BindHelpers.bind comes from
import scala.xml._
import Bindings._

object UserBinding extends DataBinding[User] {
  def apply(user: User): Binding = bind("user", _,
    "name" -> Text(user.fname+" "+user.lname),
    "email" -> Text(user.email)
  )
}

object MailingAddressBinding extends DataBinding[Address] {
  def apply(addr: Address): Binding = bind("address", _,
    "line1" -> Text(addr.recipient),
    "line2" -> Text(addr.addr1),
    "line3" -> Text(addr.addr2),
    "line4" -> Text(addr.city+", "+addr.state+" "+addr.postalCode+" "+addr.country)
  )
}

object AdminAddressBinding extends DataBinding[Address] {
  def apply(addr: Address): Binding = bind("address", _,
    "recipient" -> Text(addr.recipient),
    // imagine more literal bindings like the previous one here
  )
}

object OrderLineBinding extends DataBinding[OrderLine] {
  //more of the same
}

These implementations take advantage of partial application and the Scala type inferencer to turn the call to bind(...) into a closure with the type Binding, which is NodeSeq => NodeSeq. At this point, we're not taking advantage of the fact that these functions can compose, so we'll do so now a we build bindings for our more complex objects.


//same imports as before

trait OrderBinding extends DataBinding[Order] {
  implicit val userBinding: DataBinding[User]
  implicit val addressBinding: DataBinding[Address]
  implicit val lineBinding: DataBinding[OrderLine]
  implicit val txnBinding: DataBinding[Transaction]

  def apply(order: Order): Binding = (xhtml: NodeSeq) => {
    val itemTemplate = chooseTemplate("order", "lineItem", xhtml)
    val txnTemplate = chooseTemplate("order", "transaction", xhtml)

    bind("order", xhtml,
      "user" -> order.user.bind("templates-hidden/user/order-display".path),
      "shipTo" -> order.shipTo.bind("templates-hidden/address/order-display".path),
      "lineItems" -> order.items.flatMap(_.bind(itemTemplate)).toSeq,
      "transactions" -> order.transactions.flatMap(_.bind(txnTemplate)).toSeq
    )
  }
}

There's a lot more going on in this one. First off, we define our DataBinding as a trait rather than a concrete object, in order to take advantage of abstract implicit val declarations. When we go to instantiate this trait, we will provide concrete implementations of the various bindings that we want applied to the objects from which an Order instance is composed. The compiler will then be able to apply the "binder" conversion to get a Binder instance which closes over the instance with the appropriate DataBinding so that we can simply call instance.bind and pass either a path to a template, or a chunk of XHTML extracted from the input with chooseTemplate to get our final value.

To be continued...

Paulp speaks great truth

2009-04-24T09:54:00.002-06:00

"After you have spent a certain amount of time in the compiler, you will
come to feel angry resentment at every comment you encounter, because it
might be trying to mislead you! I have modified my editor not to show me
any comments in scala sources so they cannot tempt me with their siren
songs of reasons and explanations. AH THE LIMITLESS SERENITY"

Seems to me that this technique is much more widely applicable than just to scala internals...

Deconstruct(or)ing Scala

2009-04-21T21:12:00.010-06:00

The question of whether to use a constructor or a factory method for object construction is not new; we've had this discussion for years in the Java community. Scala's approach to object construction has a few features that will undoubtedly reignite this debate.

On the one hand, ordinary object construction is significantly more powerful than in Java - first, the all the ordinary Java boilerplate of assigning constructor parameters to member variables is abolished. What in Java would look like this:


public class Foo {
  final int myInt;
  final String myString;

  public Foo(int myInt, String myString) {
    this.myInt = myInt;
    this.myString = myString;
  }
}

becomes the concise and sensible


class Foo(val myInt: Int, val myString: String)

More significantly, Scala's trait mechanism allows for extension of a class definition at the use site; for example, if I have a trait that adds rendering logic to a class, I can mix it in only when I need it.


trait FooRenderer extends Foo {
  def render: String = "I'm a Foo! My int is "+myInt+" and my string is "+myString
}

val f1 = new Foo(1, "hi") // normal Foo
val f2 = new Foo(2, "well, hello!") with FooRenderer // renderable Foo

So Scala's constructors are really powerful and you really want to use them, right? But wait...

It turns out that there are some subtle issues that arise if you start adding more logic to constructors in Scala. The logic of a Scala constructor goes directly in the body of the class, and here's the tricky bit: this is also where other member variables that aren't just blindly assigned as constructor parameters are declared and assigned. In Java, any intermediate variables that you used within a constructor were unavoiably local; in Scala they can easily (and will, if care is not taken) become a permanent part of the object.

Let's look at something a little more complex. Consider a class that, as part of its construction, finds the most common element in a list and assigns both that element and the number of occurrences to member variables.


class Common[T](l: Iterable[T]) {
  val (value, count) = l.foldLeft(Map.empty[T,Int]) {(m, v) =>
    m + (v -> (m.getOrElse(v, 0) + 1))
  }.reduceLeft {(a, b) =>
    if (a._2 > b._2) a else b
  }
}

Now, there may be ways to implement this that avoid constructing and decomposing a tuple, but this is the most straightforward and efficient implementation I could come up with. A peek at the generated bytecode reveals something interesting, however:


   93: putfield #83; //Field x$1:Lscala/Tuple2;
   96: aload_0
   97: aload_0
   98: getfield #83; //Field x$1:Lscala/Tuple2;
   101: invokevirtual #73; //Method scala/Tuple2._1:()Ljava/lang/Object;
   104: putfield #85; //Field value:Ljava/lang/Object;
   107: aload_0
   108: aload_0
   109: getfield #83; //Field x$1:Lscala/Tuple2;
   112: invokevirtual #76; //Method scala/Tuple2._2:()Ljava/lang/Object;
   115: invokestatic #91; //Method scala/runtime/BoxesRunTime.unboxToInt:(Ljava/lang/Object;)I
   118: putfield #93; //Field count:I

What's up with line 93? I didn't want that spurious Tuple2 to hang around as a member field - I just needed it as an intermediate value in the construction of the object!

As it turns out, this problem is not restricted to tuples; if you use intermediate variables in the construction of your objects, they will become permanent residents. This may not be a real problem in most circumstances, but it feels messy. Now, the standard thing to do in this situation would to be to create a factory method on the companion object:


object Common {
  def apply[T](l: Iterable[T]): Common[T] = {
    val (value, count) = l.foldLeft(Map.empty[T,Int]) {(m, v) =>
      m + (v -> (m.getOrElse(v, 0) + 1))
    }.reduceLeft {(a, b) =>
      if (a._2 > b._2) a else b
    }

    new Common(value, count)
  }
}

class Common[T](val value: T, val count: Int)

Now there's no intermediate variable stored in the bytecode for Common, but we have a new problem: we can no longer construct the object from an iterable while mixing in an additional trait at the instantiation site!

Scala supports the use of auxiliary constructors, with the caveat (similar to that present in Java) that the first statement in an auxiliary constructor must be either a call to the primary constructor, or another auxiliary constructor. Because of this constraint, we can't simply use the contents of the factory method above in an auxiliary constructor. We can, however, evaluate a method within the chained call, and that gives us a workable, if somewhat boilerplate-laden solution.


class Common[T](val value: T, val count: Int) {

  private def this(t: Tuple2[T, Int]) = this(t._1, t._2)

  def this(l: Iterable[T]) = this(
    l.foldLeft(Map.empty[T,Int]) {(m, v) =>
      m + (v -> (m.getOrElse(v, 0) + 1))
    }.reduceLeft {(a, b) =>
      if (a._2 > b._2) a else b
    }
  )
}

By threading the decomposition through a private constructor that takes a tuple, we can now avoid the spurious intermediate values getting incorporated into the class, and still enjoy the benefits of instantiation-site mix-ins. What's more, there has been a bunch of talk on the Scala mailing list about a future unification of tuples with method (and hopefully constructor) parameters - in which event the extra private constructor could disappear entirely!

A Scala Koan

2009-04-15T21:38:00.003-06:00

I just came across this post from James Iry on the Scala list.

"It's not part of the language so you write your own and decide if it's too confusing to be useful."


class LazyVar[T](init : => T) {
  private var thunk : (() => T) = {() => {thunk = {() => init}; thunk()}}
   
  def ! = synchronized { thunk() }
  def :=(newVal : => T) = synchronized { thunk = {() => {thunk = {() => newVal}; thunk()}}}
   
  def morph(f : T => T) = synchronized { val oldThunk = thunk ; thunk = { () => { thunk = {() => f(oldThunk())}; thunk()}}}                        
}

object LazyVar {
  def apply[T](init : => T) = new LazyVar({init})
}

scala> val x = LazyVar(1)
x: LazyVar[Int] = LazyVar@5354a

scala> x := {println("assigning 2"); 2}

scala> x := {println("assigning 3"); 3}

scala> x!
assigning 3
res16: Int = 3

It's a Scala koan!

2007-12-05T16:08:00.000-07:00

In looking around the web a bit, I've had a hard time finding any capistrano plugins that handle deployment of SVN tags and branches well. So, I wrote a little one of my own.



Capistrano.configuration(:must_exist).load do

  set :svn_trunk_dir, 'trunk'
  set :svn_tag_dir, 'tags'
  set :svn_branch_dir, 'branches'
  
  before "deploy" do
    if rails_env == 'production'
      abort "ERROR: Production deploys must be from a tag or branch." unless repository =~ /#{svn_tag_dir}|#{svn_branch_dir}/
    end
  end
  
  desc <<-DESC 
  Deploy from a tag instead of from the trunk. This will prompt if no TAG value 
  is specified on the command line. No sanity checks for whether the tag actually
  exists are performed.
  DESC
  task :deploy_tag do
    set :repository, get_svn_url(svn_tag_dir, 'TAG')
    run_deploy
  end
  
  desc "Create the specified tag and deploy from that tag."
  task :deploy_new_tag do
    trunk = "#{repository[/(.*?)(?:\/trunk)?$/, 1]}/#{svn_trunk_dir}"
    set :repository, get_svn_url(svn_tag_dir, 'TAG')
    puts "Creating new tagged release of trunk as #{ENV['TAG']}."
    system "svn cp #{trunk} #{repository}"
    run_deploy
  end
  
  desc "Deploy from a branch instead of a tag; same caviats as for :deploy_tag."
  task :deploy_branch do
    set :repository, get_svn_url(svn_branch_dir, 'BRANCH')
    run_deploy
  end
  
  desc "Create a branch from the specified tag and deploy from that branch."
  task :deploy_new_branch do
    tag_url = get_svn_url(svn_tag_dir, 'TAG')
    branch_url = get_svn_url(svn_branch_dir, 'BRANCH')
    puts "Creating new branch #{ENV['BRANCH']} from tag #{ENV['TAG']}."
    system "svn cp #{tag_url} #{branch_url}"
    set :repository, branch_url
    run_deploy    
  end
  
  # Executes the ordinary deploy task
  def run_deploy
    puts "Deploying using repository url #{repository}"
    execute_task(find_task("deploy"))
  end
  
  # Prompts the user for a tag and updates the repository url based upon that
  # tag and the svn conventions.
  def get_svn_url(tag_dir, env_var = 'TAG')
    while ENV[env_var].nil? || ENV[env_var].strip.empty?
      ENV[env_var] = Capistrano::CLI.ui.ask("Please specify #{env_var.downcase} name: ")
      ENV[env_var].strip!
    end
    
    repo_root = repository[/(.*?)(?:\/trunk)?$/, 1]
    "#{repo_root}/#{tag_dir}/#{ENV[env_var]}"
  end
end

I just stuck all of this in lib/cap/tag_deployer.rb, then added


require 'lib/cap/tag_deployer.rb'

to my deploy.rb file, and away I go!

XML is a social tool

2006-12-08T20:04:00.000-07:00

Yesterday, I was listening to a presentation where XML was presented as a solution to some technical problem. Now, don't get me wrong, I like XML, but I'd like to make one thing perfectly clear.

XML is a solution to exactly 0 technical problems. It is, however, a reasonable answer to many social problems of software development.

But not any techical ones. Period.