Tuesday, March 5, 2013

Scala Either Example - Responding to Failures



A lot of Scala developers I know regularly use Option to write concise code in situations where something may or may not be returned by a given function. By getting familiar with the functions available on the Option class, you can chain many operations together without a lot of boilerplate or defensive protection against nonexistent values. Something like:

getAnOption
  .flatMap(transformIntoAnotherOption)
  .map(changeThing)
  .filter(thingIsTheRightType)

Assuming in this contrived bit of pseudo-code that getAnOption and transformIntoAnotherOption are functions that return Option types, this chain of function calls will short-circuit if any call returns a None. For example, if transformIntoAnotherOption returns None, then the expression evaluation is complete without ever running changeThing.

But what if I want to know which part of a complex expression like the one above returned None? It's tricky and less elegant when using an Option expression which simply evaluates to Some or None.

Let's say I'm writing a database application that requires a two-phase lookup, either phase of which may fail to find the requested record. I want to be able to:
  • Look up an id based on a user-provided String
  • If we found an id, look up the Record
  • If we found a Record, validate it, process it, display it in an app, etc
The driver I am using returns Option[Record] from find calls. For this simple example, I mocked up a call with a frequent random failure that looks like this:

def databaseCall[A](value: A) = 
  if (nextInt % 5 == 0) None else Some(value)

def nextInt = math.abs(scala.util.Random.nextInt)

And two wrapper calls to represent our two phases of lookup:

def lookupId(key: String): Option[Int] = 
  databaseCall(nextInt)

def lookupRecord(id: Int): Option[Record] =
  databaseCall(Record(id))

Chaining Option calls is a natural fit if we just need to find a record and return it (or not):

def findRecordByKey(key: String): Option[Record] =
  databaseCall(key)
  .flatMap(databaseCall)
  .filter(validateIt)
  .map(processIt)
  .foreach(displayInApp)

The code above will sometimes call displayInApp with a value, and sometimes it will not. It is concise and avoids cluttering the logic with explicit conditionals and error handling. But it does not allow us to provide detailed feedback about what part of the process failed. Say we want to log whenever a databaseCall fails. We have to break up our nice single expression with calls to isDefined and error logging code. Not exactly elegant functional monadic wizardry that will get you invited to the cool kids' parties.

Either FTW


The scala.Either class can be used in ways very similar to Option. Either represents an instance of either Left or Right. By convention, Left is the "bad" or failure result and Right is the "good" one. In our example, that means Right is a value we found in the database and Left is a richer version of None where we can put a sensible message, an Exception or whatever we like. Profit!

First thing, let's change our database functions so they return Either instead of Option. Conveniently, Option has the functions toLeft and toRight for just this purpose. We'll use toRight in our example, which returns a Right containing the value of the Option it is called on if it is a Some. toRight() takes an argument that will be used to construct a Left if the Option is None. In our case, we'll provide a straightforward error message to use in case of failures:

def lookupId(key: String): Either[String, Int] = 
  databaseCall(nextInt).toRight("failed to find id based on key")

def lookupRecord(id: Int): Either[String, Record] =
  databaseCall(Record(id)).toRight("failed to find record based on id")

Easy enough. Here's how we can chain the calls to these functions:

lookupId("key").right
  .flatMap(lookupRecord(_))
Pretty similar to how we use Options. Calling right() on the result of lookupId("key") projects the Either as a Right. The contained value is the passed to lookupRecord and so on. If the Either is a Left, lookupRecord is never called. So just as with Option, we can chain our calls, evaluating to an Either in the end.

So we end up with either a Right(someRecordInstance) or a Left(someErrorMessage). We can use pattern matching to process this result, or you can use the fold() function, which takes two function arguments. The first is run in the case of a Left, the second in the case of a Right. So that's just super for us. Here's the whole thing:

lookupId("key").right
  .flatMap(lookupRecord(_)).fold(
    s => println("ERROR - " + s),
    s => println("looked up record " + s))

Running this code a thousand or so times prints a bunch of lines like the following:
looked up record Record(2073762079)
ERROR - failed to find id based on key
ERROR - failed to find record based on id
ERROR - failed to find record based on id
looked up record Record(1377010886)
looked up record Record(1665383097)
looked up record Record(646489647)

We now know when we failed to resolve an id to a key. We know if the record itself was not found. And assuming all went well, we get our Record result returned, validated, processed and displayed in our app in one neat expression. And we get invited to the cool kids' parties.

Download a buildable version of this code from github, and enjoy using Either in your projects.

Sunday, November 20, 2011

Introduction to Casbah - The Scala MongoDB Driver

I've been meaning to point out the sheer coolness of the Scala MongoDB driver - Casbah. Written and maintained by 10Gen's own Brendan McAdamas, it is a truly elegant way to interact with MongoDB. If you've been using the Java driver, this might be a fun way to try out some Scala.

Here are some examples, none of which will be too surprising, since they look so much like interacting with Mongo via the shell client. Really that's the nice thing about Casbah - it's much closer to using the Python driver or JavaScript shell than the Java map-style driver.

I'll demonstrate some more advanced usage in the future, but this will get you started.


//connect to a MongoDB server
val mongo = MongoConnection("anduin")

//create/connect to a collection
val coll = mongo("casbah_examples")("movies")

//drop collection
coll.dropCollection

//create some documents
val pf = MongoDBObject("title" -> "Pulp Fiction",
"director" -> "Quentin Tarantino",
"foo" -> "bar",
"actors" -> List("John Travolta", "Samuel L Jackson"),
"year" -> 1994)

val sw = MongoDBObject("title" -> "Star Wars",
"director" -> "George Lucas",
"cast" -> List("Harrison Ford", "Carrie Fisher"),
"year" -> 1977)

val fc = MongoDBObject("title" -> "Fight Club",
"director" -> "David Fincher",
"cast" -> List("Brad Pitt", "Edward Norton"),
"year" -> 1999)

//add some documents
coll += pf
coll += sw
coll += fc
val pfid = MongoDBObject("_id" -> pf.get("_id"))

//update a document
//increment a value (or set it if the value does not exist)
coll.update(pfid, $inc("likes" -> 1))

//set a value
coll.update(pfid, $set("year" -> 1994))

//remove a field
coll.update(pfid, $unset("foo"))

//add a value to an array - create it if need be
coll.update(pfid, $push("comebackKids" -> "John Travolta"))

//add a bunch of values to an array
coll.update(pfid, $pushAll("comebackKids" -> ("Bruce Willis", "Uma Thurman")))

//add to a set (only adds values that were not there already)
coll.update(pfid, $addToSet("actors") $each("John Travolta","Groucho Marx","Harpo Marx","Alyssa Milano","Bruce Willis","Uma Thurman", "Jim Carrey"))

//remove last element of an array - use -1 to remove the first element
coll.update(pfid, $pop("actors" -> 1))

//remove matching elements from a field
coll.update(pfid, $pull("actors" -> "Alyssa Milano"))

//remove more than one matching element from a field
coll.update(pfid, $pullAll("actors" -> ("Groucho Marx", "Harpo Marx")))

//rename a field
coll.update(pfid, $rename("actors" -> "cast"))

//query documents
//find one by an exact field match
coll.findOne(MongoDBObject("title" -> "Star Wars"))

//find by regex
coll.find(MongoDBObject("title" -> ".*F".r))

//leave out the ids
coll.find(MongoDBObject(),MongoDBObject("_id" -> 0))

//include only certain fields without ids
coll.find(MongoDBObject(), MongoDBObject("title" -> 1, "cast" -> 1, "_id" -> 0))

//find most recent
coll.find(MongoDBObject()).sort(MongoDBObject("year" -> -1)).limit(1)

//remove
coll.remove(MongoDBObject("title" -> "Fight Club"))

Saturday, September 24, 2011

Scalatra MongoDB Executable Jar Template

Just a quick note that I have updated my giter8 template for a simple Scalatra project using MongoDB. It now uses library versions that are available for Scala 2.9.1 and updates some dependency versions. These dependencies/plugins target SBT 0.10.1 and probably won't work well with other versions. I also added the assembly plugin to make it easy to build an executable jar. I didn't bother to branch since the old one was broken due to some dependency issues.

To start a new project, do the following:


  1. Install giter8

  2. g8 JanxSpirit/scalatra-mongodb

  3. cd *project-name*

  4. sbt

  5. jetty-run

  6. browse to http://localhost:8080/test

  7. If you want an executable jar, run 'assembly' at the SBT prompt

  8. If you want to create a standalone shell command wrapping that jar, run something like 'cat src/main/resources/execute_jar.sh target/scalatra-mongodb-project-assembly-1.0.jar > target/scalatra_mongo'



Hopefully this template is helpful. Cheers!

Sunday, June 12, 2011

Scalatra MongoDB template project using SBT 0.10.x

I spent some time this weekend getting a basic Scalatra/Casbah/MongoDB project (much like the one I covered in this post) set up using SBT 0.10.0.

If you have looked at the newest version of SBT, you know that a lot has changed. SBT 0.7.x projects will not build under the newer versions without a bit of migration. I ran into some hiccups along the way, so I set up a template project to avoid having to figure it out again. It uses Nathen Hamblen's fantastic giter8(g8) tool to allow you to set up a new project in seconds. Hopefully other g8 users find this template useful.

Changes in SBT 0.10.x

I just want to note some of the changes I ran into in setting up this very simple project:

  • build.sbt file - SBT 0.10.x offers an alternative to the /project/build/Project.scala configuration file - you can put a *.sbt file in the root of your project and configure your project there using the new SBT DSL. You can still configure your project using Scala, but the API and file locations have changed. See the SBT documentation for details.

  • "jetty" context for the Jetty dependency - this one confused me as I didn't see it documented anywhere. The Jetty dependency in build.sbt should not use the "test" scope as it did with SBT 0.7.x but rather the "jetty" scope. Using "test" will cause the jetty-run command to silently fail in a most unhelpful way.

  • Jetty support no longer built-in - Unlike SBT 0.7.x which provided commands like jetty-run, jetty-stop and prepare-webapp for any Project extending DefaultWebProject, SBT 0.10.x requires a plugin to support web applications. See the project's homepage for setup and my template project for an example.


Template Project

You'll need to install SBT 0.10.x - README

You'll also need to install g8

Both tools are very easy one-time installs and worth having. Once they are installed, you should be able to run:


g8 JanxSpirit/scalatra-mongodb
cd *name-of-app*
sbt
> update
> jetty-run


You'll then be able to browse to the test resource to make sure it's working.

What's inside

The servlet provides two resources:
  1. 'test' - a 'Hello World' type resource just to make sure everything is working.

  2. 'msgs' - GET msgs to see all records currently in the database and a form to add another record - the form issues a POST to the same resource to add records

The project includes a few basic tests to get you started. Run 'sbt test' to see the test output.

The template allows you to configure where your MongoDB server is running, the name of your servlet and other parameters.

All source code is here: https://github.com/JanxSpirit/scalatra-mongodb.g8 I'd welcome any comments or suggestions or pull requests. Happy coding!

Sunday, June 5, 2011

Java Without Semicolons: An Introduction to Scala - Part 1: Iteration

This series of posts was inspired by a suggestion from Martin Oedersky at Scala Days 2011 as well as something I have heard Dick Wall say on several occasions. To paraphrase Dick, you can get started with Scala by converting Java code to Scala with few changes beyond getting rid of the semicolons. It may not take advantage of many of Scala's more powerful features, but it's a great way to get started and an excellent jumping off point to learning Scala's richer language features.

Scala is a deep and flexible language that allows an enormous degree of freedom to library developers and advanced users, but those features can be daunting to new developers trying to take them all in. So, what areas should a Java programmer getting started with Scala start with? This blog series will cover Scala features that Java programmers should be able to put to immediate use. In other words, I'll suggest some of the first things a newcomer to Scala can try after getting rid of the semicolons.

Setup

You're encouraged to follow along with these examples in the Scala REPL and use them as a jumping off point for your own exploration. Installing Scala is easy - if you already have Java set up, just unzip the Scala distribution, set SCALA_HOME and add $SCALA_HOME/bin to your path. You can run the REPL by simply typing 'scala' in a shell. Further instructions at scala-lang.org or typesafe.com. I'll be using Scala 2.9.0.1 which is the latest version at the time of this writing.

Iteration

As a longtime Java developer, I was immediately drawn to Scala's alternatives to for-loop iteration. Java loops are easy enough to write, but can be buggy and confusing to read and comprehend.

This post will take a look at some functions Scala Collections provide to iterate that are as easy to read and understand as they are to write.

Function Literals

Assuming that Java is the only language you are familiar with, there's one important Scala language feature you'll need to understand in these examples, and that is function literals.

A function literal is analogous to String or int literals in Java in that it is a value that can be provided directly where it is used.

In Scala, you can define a function inline - that is pass a bit of functionality right where it is needed. This is what a simple function literal looks like:

x => x + 1

You can read this as "a function that take one argument x and returns the result of adding 1 to the value of x".

Scala's collections have special functions that take functions as arguments to be applied to the members of the collection. So unlike a Java for loop, where you get a handle on each member in the collection one at a time and do something to it, in Scala you can tell the collection to apply some functionality to all of its members and the actual iteration over elements is handled internally. Building on our example:

val someInts = List(4, 8, 15, 16, 23, 42)
val incrementedInts = someInts.map(x => x + 1)

The map function takes a single-argument function as its argument. It applies that function to every element in the List and returns a new List whose contents are the result of applying the function argument. In the example above, each element of the list is incremented by 1 and the result looks like this:

List(5, 9, 16, 17, 24, 43)

Note that the original list still exists unmodified.

In Scala code you will often see a shorter form of function literals. Scala allows us to substitute a _ character for the function parameters in many cases, so the above map call could be written as:

val incrementedInts = someInts.map(_ + 1)

The two are functionally identical, and it's up to you which you want to use.

In some cases the function literal can be even shorter. When the function passed as an argument takes a single argument itself, Scala will automatically pass each element of the collection to it without even needing the _. The following examples are functionally equivalent and use the foreach function, which is executed only for side effects and does not return a value (it returns Unit, equivalent to Java's void return type).

someInts.foreach(x => println(x))
someInts.foreach(println(_))
someInts.foreach(println)

Now let's say we only want the subset of our list that is odd numbers. The filter function is useful for such operations. It takes a function argument that returns a boolean to determine whether an element should be included in the output:

someInts.filter(x => x % 2 == 1)
//someInts.filter(_ % 2 == 1)

Maybe we want the first number in the list that is greater than 20. We can use find:

someInts.find(x => x > 20)
//someInts.find(_ > 20)

If you want to discover whether all elements of a collection satisfy some requirement, use forall, which takes a function that returns boolean and itself returns a boolean:

someInts.forall(x => x > 0)
//someInts.forall(_ > 0)

You can get the first n elements that meet a certain requirement by using takeWhile. The following example returns a new list containing all of the words up to the first word that begins with a vowel:

val words = List("the","quick","brown","fox","jumped","over","the","lazy","dog")
words.takeWhile(w => !"aeiou".contains(w(0)))
// returns List(the, quick, brown, fox, jumped)

If you wanted to retain the remainder of the list separately, you can pass the same function to span, which returns a tuple of two Lists:

words.span(w => !"aeiou".contains(w(0)))
// returns (List(the, quick, brown, fox, jumped),List(over, the, lazy, dog))

The last example of these sort of collection iteration functions I'll demonstrate is foldLeft. It's a bit different in that it does not return a collection, but rather a single value as a result of applying a two-argument function to each element of a list and the result of the last function execution. You need to provide it a seed value, and the first iteration will execute the function with the seed value and first element of the list. In this example we'll get the sum of all numbers with more than one digit:

someInts.foldLeft(0)((a,b) => if (b.toString.length > 1) a+b else a)

In the example, foldLeft is seeded with a 0. The function is then called with 0 and 4, returning 0. Then 0 and 8 returning 0. Then 0 and 15 returning 15, and so on. Folding operations take a bit of getting used to, but are a powerful functional programming paradigm.

For Comprehension

Scala does have a for loop structure that may look more familiar to java developers. In Scala it's called a for comprehension, and it can be used in very powerful ways. It can also be used to iterate over collections in a more imperative way:

for (word <- words) {
println(word)
}

But, often the functions on the various Scala collection classes allow you to perform common iteration tasks without using loops. This change can result in fewer more readable lines of code. In Java, even the contrived examples above would necessetate loops, temporary variables, temporary lists, and, in general, more boilerplate code not relating to business logic. The Scala collection libraries allow us to focus on the code that performs the actions we're really interested in. And it makes for some nifty one-liners:

//find all numbers from 1 to a million that are evenly divisible by 3 and 5,
//remove all instances of the digit '9' and sum the resulting numbers
(1 to 1000000).filter(i => i % 3 == 0 && i % 5 == 0).map(_.toString.replace("9","").toInt).sum

That's it for this installment. Hopefully someone getting started with Scala finds these examples helpful as a jumping off point.

Sunday, February 13, 2011

Sneaking Scala Through the Alley - Test Java Maven Web Services with Scala Specs

It's been said before, but a great way to introduce Scala at work is in writing tests. In my case, that means writing functional tests for web services built with Maven. I've found the Specs library excellent for this purpose, and after sorting out a bit of setup, it's integrated into my build well. Some things I like about this approach:
  • Tests are concise and quick to write
  • Specs can be written in natural language, user stories, or however you normally describe software functionality
  • Output is natural language, so identifying the source of any bugs, regressions, etc is easy
  • Test code is generally such that developers who are unfamiliar with Scala can understand it quickly
  • Scala and Specs make it trivial to write focused, single-assertion tests with little boilerplate
  • Write empty specs as requirements emerge and bugs are found then implement them when you have time
  • You get to use Scala at work

This example will use Maven and a very simple Java REST service built with Jersey. In the real world you'd probably be using Spring and a lot of other libraries, but I've tried to distill this example down to the barest essentials in order to focus on Specs and how it plays with Maven.

Envision a beer service. It accepts requests for beer by type and size in ounces. It should return a 404 Not Found response if the beer is not a type it knows (Corona, Guiness or Fat Tire). If someone tries to order a beer smaller than a pint, the service should respond with a 400 Bad Request response. And if the service is able to fulfill the request, it should return a Json Beer representation.

As you work with your design team to specify these behaviors, you can write a first pass at your specs. The great thing about specs is how well they translate from the plain English description above:

"Calling GET /beer" should {
"when called with" in {
"a sizeOunces argument of 4" in {
"return status 400" >> { }
}

"a beer name 'Budweiser'" in {
"return status 404" >> { }
}

"beer name 'Guiness' and sizeOunces 16" in {
"return status 200" >> { }
"return a Beer representation" in {
"with" in {
"name 'Guiness'" >> {}
"sizeOunces 16" >> {}
}
}
}
}
}

You can read each spec from its outermost part to inner, for example: "Calling GET /beer should when called with beer name 'Guiness' and sizeOunces 16 return status 200". You can leave these specs empty until you implement the corresponding functionality. Some build environments like Hudson display the unimplemented tests separately from the passing or failing tests. These unimplemented tests are a great aspect of using Specs. They convey to others what the expected functionality is, though it may not yet be implemented, and it serves as a checklist for you the developer. As new requirements emerge, you can immediately write empty specs that reflect them.

Setting up pom.xml
In order to use Scala in a Maven project, you'll need to set up a few plugins, dependencies. etc. I'll highlight the ones that may be unfamiliar to Java developers, and the entire pom.xml is shown below.

Repositories
I'm not going to get into repository configuration much, since everyone's setup is different. One repository to be aware of when doing Scala development is Scala Tools. It's built into SBT, but you'll need to add it to your Maven repository or project pom to use it in a Maven project.

<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</repository>

Dependencies
Scala
This example uses version 2.8.0 of the Scala library.

<!-- Scala -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.8.0</version>
</dependency>

Specs
Specs is a library for Behavior Driven Development in Scala. It integrates easily with mock libraries, ScalaTest for generating test data, and various IDEs and build tools. Scala Specs can be run as JUnit tests, allowing integration into Java development environments like Eclipse, IntelliJ and NetBeans as well as CI systems like Hudson and Bamboo. It has an excellent lightweight syntax that is easy to read and write. I'll only be scratching the surface of Specs' functionality here, but you should check out the excellent documentation for more examples.

<!-- Specs -->
<dependency>
<groupId>org.scala-tools.testing</groupId>
<artifactId>specs_2.8.0</artifactId>
<version>1.6.5</version>
</dependency>

Dispatch
Dispatch is a wrapper around HttpClient that provides a very terse DSL syntax and the ability to make Http calls in a Scalactic way. The overloaded operators throw some people off so if you prefer to use HttpClient or any other Http library, that works too. I have found that once I got familiar with Dispatch syntax, I quite like it, and it meshes well with Specs to allow me to quickly bang out a lot of test calls.

<!-- Dispatch -->
<dependency>
<groupId>net.databinder</groupId>
<artifactId>dispatch-http_2.8.0</artifactId>
<version>0.7.7</version>
<scope>test</scope>
</dependency>

Lift-Json
Dispatch provides support for Json through the lift-json library, which is a part of the Lift framework. It provides straightforward serialization and deserialization of Scala case classes as well as clear XPath-like parsing of Json.

<dependency>
<groupId>net.databinder</groupId>
<artifactId>dispatch-lift-json_2.8.0</artifactId>
<version>0.7.7</version>
<scope>test</scope>
</dependency>

And the rest...
You'll also need to add JUnit and Jersey for this example project. See the complete pom.xml below for specifics.

Build
You'll need to tell Maven to look for Scala tests.

<testSourceDirectory>${project.basedir}/src/test/scala</testSourceDirectory>

Compile the Scala Tests
Configure the Scala compiler plugin so that your Scala tests get built like so:

<!-- Compile Scala -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>

Surefire
Clarify that you would indeed like to run the Scala tests by telling the Maven Surefire plugin when and what to run. In this example we flout JUnit convention and run things that end in Spec. Observant types will notice that the configuration says **/*Spec.java even though we're writing tests in Scala. Just go with it.

<!-- Tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<useSystemClassLoader>false</useSystemClassLoader>
<argLine>-Xmx512m</argLine>
<includes>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>

Start and Stop Jetty
This configuration will run Jetty for the duration of the build. The sophisticated among you may wish to start and stop the server at different phases of the build. Determining an appropriate strategy is left as an exercise to the reader.

<!-- Run Jetty -->
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.24</version>
<configuration>
<connectors>
<connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
<port>2342</port>
</connector>
</connectors>
<stopKey>stop</stopKey>
<stopPort>9999</stopPort>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>process-classes</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>

Beer Service
With the Maven setup behind us, we'll throw together a simple Jersey service so there's something to test. Here's a Beer JavaBean and a service class to serve some Beers.

/src/main/java/com/janxspirit/Beer.java

package com.janxspirit;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
public class Beer {

private String name;
private int sizeOunces;

public Beer() {}

public Beer(String name, int sizeOunces) {
this.name = name;
this.sizeOunces = sizeOunces;
}

@XmlElement(name = "name")
public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

@XmlElement(name = "sizeOunces")
public int getSizeOunces() {
return sizeOunces;
}

public void setSizeOunces(int sizeOunces) {
this.sizeOunces = sizeOunces;
}
}

/src/main/java/com/janxspirit/BeerResource.java

package com.janxspirit;

import java.util.ArrayList;
import java.util.List;
import javax.ws.rs.Path;
import javax.ws.rs.GET;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.Status;

@Path("/")
public class RestResource {

private List beerList;

public RestResource() {
beerList = new ArrayList();
beerList.add("Corona");
beerList.add("Guiness");
beerList.add("Fat Tire");
}

@GET
@Produces("application/json")
@Path("beers/{name}")
public Response getBeer(@PathParam("name") String name, @QueryParam("sizeOunces") int sizeOunces) {
if (name == null || !beerList.contains(name)) {
return Response.status(Status.NOT_FOUND).entity(String.format("Sorry we don't have any %s", name)).build();
}
if (sizeOunces < 8) {
return Response.status(Status.BAD_REQUEST).entity("Sorry no kid-size beers.").build();
}
Beer beer = new Beer(name, sizeOunces);
System.out.println(String.format("Pouring %s ounces of %s", sizeOunces, name));
return Response.ok(beer).build();
}
}

/src/main/java/JsonContextResolver.java
In order to get the default Json library to render the responses correctly, you need to add a class like the following. There are other ways, but in general its unfortunate that Jersey doesn't do the right thing out of the box at least presenting integer attributes unquoted in Json. We explicitly set the sizeOunces property to be non-String below.

package com.janxspirit;

import com.sun.jersey.api.json.JSONConfiguration;
import com.sun.jersey.api.json.JSONJAXBContext;
import javax.ws.rs.ext.ContextResolver;
import javax.ws.rs.ext.Provider;
import javax.xml.bind.JAXBContext;

@Provider
public class JsonContextResolver implements ContextResolver<JAXBContext> {

private JAXBContext context;
private Class[] types = {Beer.class};

public JsonContextResolver() throws Exception {
this.context =
new JSONJAXBContext(
JSONConfiguration.mapped().nonStrings("sizeOunces").build(), types);
}

public JAXBContext getContext(Class<?> objectType) {
for (Class type : types) {
if (type == objectType) {
return context;
}
}
return null;
}
}

web.xml
A last bit of setup is to configure your web.xml to fire up the Jersey servlet. This goes in the usual place - src/main/webapp/WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">
<display-name>scala_test_jersey</display-name>
<servlet>
<servlet-name>RestResourcer</servlet-name>
<servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
<init-param>
<param-name>com.sun.jersey.config.property.packages</param-name>
<param-value>com.janxspirit</param-value>
</init-param>
</servlet>
<servlet-mapping>
<servlet-name>RestResourcer</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
</web-app>

Writing the Specs
Now the fun part! This is actually much less involved than all the boilerplate above, but hopefully the setup is helpful in getting you to the good stuff more quickly.

The first thing I did was create a simple case class and a function to convert the Dispatch responses into a simple object with a status code and body. I'd love to hear any better approaches to this, but in any event, this does work.

def toResponse(req: Request): StatusCodeBody = {
val http = new Http
http x (req as_str) {
case (status, _, ent, _) => StatusCodeBody(status, EntityUtils.toString(ent.get))
}
}

case class StatusCodeBody (status: Int, body: String)

Now we fill in the empty spec bodies. In most cases that's a one-liner to make the Http call and then a simple equality test against the response object. Here's what a simple call to the service looks like:

val app = "scala_test_jersey"
val getBeerRes = toResponse(:/("localhost", 2342) / app / "beers" / "Budweiser")
"return status 404" >> { getBeerRes.status must_== 404 }

As you can see, assembling an Http call with dispatch is very straightforward. If you don't like the overloaded operators, I suggest you use trusty old HttpClient.

Here's all you need to do to add a query param to a request:

val getBeerRes = toResponse(:/("localhost", 2342) / app / "beers" / "Corona" <<? Map("sizeOunces" -> 4))
"return status 400" >> { getBeerRes.status must_== 400 }

These examples also show one of the Specs matchers 'must_==' The Specs library provides a lot of matchers and if you need some new ones, it's very easy to create them.

For the last two specs we'll do another simple check of the Http response code as well as deserialize the Json response body and inspect the result. To do that, I've created a simple case class. Lift-Json takes care of the rest. Here's what that looks like:

case class JsonBeer(name: String, sizeOunces: Int)


val getBeerRes = toResponse(:/("localhost", 2342) / app / "beers" / "Guiness" <<? Map("sizeOunces" -> 16))
"return status 200" >> { getBeerRes.status must_== 200 }
"return a Beer representation" in {
val beer = parse(getBeerRes.body).extract[JsonBeer]
"with" in {
"name 'Guiness'" >> {beer.name mustEqual "Guiness"}
"sizeOunces 16" >> {beer.sizeOunces must_== 16}
}
}


Pretty slick.

And now the moment of truth. You can run

mvn clean test

or get fancier and run it in your IDE. This screenshot shows output in Netbeans.


There's a bit more setup for the spec class. I've set these tests to share variables and run sequentially. The complete code for BeerSpec.scala and pom.xml is below.

I hope this setup guide has been helpful and encourages some folks to dip their toe in with Scala testing. Happy coding!

/src/test/scala/BeerSpec.scala

import org.specs._
import dispatch._
import net.liftweb.json.JsonParser._
import net.liftweb.json.JsonAST._
import net.liftweb.json.JsonDSL._
import org.apache.http.util.EntityUtils

class BeerSpec extends SpecificationWithJUnit {

implicit val formats = net.liftweb.json.DefaultFormats // Brings in default date formats etc.
val app = "scala_test_jersey"
setSequential()
shareVariables()

"Calling GET /beers" should {
"when called with" in {

"name 'Budweiser'" in {
val getBeerRes = toResponse(:/("localhost", 2342) / app / "beers" / "Budweiser")
"return status 404" >> { getBeerRes.status must_== 404 }
}

"name 'Corona' and sizeOunces 4" in {
val getBeerRes = toResponse(:/("localhost", 2342) / app / "beers" / "Corona" <<? Map("sizeOunces" -> 4))
"return status 400" >> { getBeerRes.status must_== 400 }
}

"beer name 'Guiness' and sizeOunces 16" in {
val getBeerRes = toResponse(:/("localhost", 2342) / app / "beers" / "Guiness" <<? Map("sizeOunces" -> 16))
"return status 200" >> { getBeerRes.status must_== 200 }
"return a Beer representation" in {
val beer = parse(getBeerRes.body).extract[JsonBeer]
"with" in {
"name 'Guiness'" >> {beer.name mustEqual "Guiness"}
"sizeOunces 16" >> {beer.sizeOunces must_== 16}
}
}
}
}
}

def toResponse(req: Request): StatusCodeBody = {
val http = new Http
http x (req as_str) {
case (status, _, ent, _) => StatusCodeBody(status, EntityUtils.toString(ent.get))
}
}
}

case class JsonBeer(name: String, sizeOunces: Int)
case class StatusCodeBody (status: Int, body: String)


pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.janxspirit</groupId>
<artifactId>scala_test_jersey</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>scala_test_jersey</name>
<url>http://maven.apache.org</url>

<repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</repository>
</repositories>

<dependencies>

<!-- Scala -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.8.0</version>
</dependency>

<!-- Dispatch -->
<dependency>
<groupId>net.databinder</groupId>
<artifactId>dispatch-http_2.8.0</artifactId>
<version>0.7.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.databinder</groupId>
<artifactId>dispatch-lift-json_2.8.0</artifactId>
<version>0.7.7</version>
<scope>test</scope>
</dependency>

<!-- Specs -->
<dependency>
<groupId>org.scala-tools.testing</groupId>
<artifactId>specs_2.8.0</artifactId>
<version>1.6.5</version>
</dependency>

<!-- JUnit -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.1</version>
</dependency>

<!-- Jersey -->
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-json</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<version>1.3</version>
</dependency>

</dependencies>

<build>
<testSourceDirectory>${project.basedir}/src/test/scala</testSourceDirectory>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.9.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>

<!-- Compile Scala -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>

<!-- Compile Java -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>

<!-- Tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<useSystemClassLoader>false</useSystemClassLoader>
<argLine>-Xmx512m</argLine>
<includes>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>

<!-- Run Jetty -->
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.24</version>
<configuration>
<connectors>
<connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
<port>2342</port>
</connector>
</connectors>
<stopKey>stop</stopKey>
<stopPort>9999</stopPort>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>process-classes</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

Thursday, January 27, 2011

Create an Executable Scala Jar with SBT

A friend was asking about building an executable Jar with Scala and SBT. This ended up working for me.

In a fresh SBT project, create the following files:

project/build/AssemblyProject.scala
(this file is lifted in its entirety from here - thanks to Nathan Hamblen)

import sbt._

trait AssemblyProject extends BasicScalaProject {
def assemblyExclude(base: PathFinder) = base / "META-INF" ** "*"
def assemblyOutputPath = outputPath / assemblyJarName
def assemblyJarName = artifactID + "-assembly-" + version + ".jar"
def assemblyTemporaryPath = outputPath / "assembly-libs"
def assemblyClasspath = runClasspath
def assemblyExtraJars = mainDependencies.scalaJars
def assemblyPaths(tempDir: Path, classpath: PathFinder, extraJars: PathFinder, exclude: PathFinder => PathFinder) = {
val (libs, directories) = classpath.get.toList.partition(ClasspathUtilities.isArchive)
for(jar <- extraJars.get ++ libs) FileUtilities.unzip(jar, tempDir, log).left.foreach(error)
val base = (Path.lazyPathFinder(tempDir :: directories) ##) (descendents(base, "*") --- exclude(base)).get }
lazy val assembly = assemblyTask(assemblyTemporaryPath, assemblyClasspath, assemblyExtraJars, assemblyExclude) dependsOn(compile)
def assemblyTask(tempDir: Path, classpath: PathFinder, extraJars: PathFinder, exclude: PathFinder => PathFinder) =
packageTask(Path.lazyPathFinder(assemblyPaths(tempDir, classpath, extraJars, exclude)), assemblyOutputPath, packageOptions)
}
project/build/Project.scala

import sbt._

class ExecutableProject(info: ProjectInfo) extends DefaultProject(info) with AssemblyProject {
override def mainClass = Some("RunMe")
}
src/main/scala/RunMe.scala

object RunMe {
def main(args: Array[String]) = {
println("I run.")
}
}
Now in your sbt shell, run:
reload
compile
assembly
That will create an executable jar in the target/scala_*version* directory. You can run it with something like:
java -jar target/scala_2.8.1/runme_2.8.1-assembly-1.0.jar
I run.
Thanks to Nathan Hamblen for the AssemblyProject trait.