Onward and upward

Form submission and validation in Playframework with any Javascript framework

Since we’ve migrated GIVE.asia from Ruby on Rails to Playframework about a year ago, we have discovered a few of new techniques around Playframework. In this blog, we would like to share how we build form submission and validation that is cleanly integrated with any Javascript framework. This blog serves as a tutorial as well.

The article, Handling form submission in the official documentation, focuses on the traditional way of form submission, which revolves around <form action="post">. The traditional way isn’t suitable when using a Javascript framework. A more suitable way is to send JSON-encoded data using AJAX.

There are 2 important points that we should keep in mind when building a form submission and validation:

  1. A clean way to set default values when rendering inputs to users.
  2. A clean way to transform between end-user representation and internal representation. For example, when users input an amount (of dollars), what users interact with is 49.99 (two-decimal floating point), but what we use internally is 4999L (cents), whose type is Long. We want a clean mechanism to transform these 2 values back and forth.

Let’s start…

First, let’s define our case class to represent the internal data:

case class Product(itemName: String, price: Long)

Our Play’s form is defined as shown below:

import play.api.data.{Form, FormError, Forms}
import play.api.data.format.Formatter
import play.api.data.Forms._

val amount = Forms.of(new Formatter[Long] {
  def bind(key: String, data: Map[String, String]): Either[scala.Seq[FormError], Long] = {
     data.getOrElse(key, "") match {
       case "^([0-9]+).([0-9][0-9])$".r(whole, decimals) => Right(whole.toLong * 100 + decimals.toLong)
       case _ => Left(Seq(FormError(key, "error.number")))
     }
  }

  def unbind(key: String, cents: Long) = Map(key -> s"${cents / 100}.${(cents % 100).formatted("%02d")}")
})

val PRODUCT_FORM = Form(
  mapping(
    "name" -> text,
    "price" -> amount
  )(Product.apply)(Product.unapply)
)

From the code above, we define the amount mapping that converts, for example, 49.99 to 4999L and vice versa. This is great because we can convert our internal representation (which is Long) back to the end-user representation.

We can draw a line in our controller which converts between our internal representation (case class) and the end-user representation (JSON). Here is a snippet in the controller:

def showForm = Action { implicit request =>
  Ok(views.html.form(
    params = Json.toJson(PRODUCT_FORM.fill(Product("iphone", 59999L)).data)
  ))
}

def submit = Action(parse.json) { implicit request =>
  form.bindFromRequest().fold(
    hasErrors = { ... } // handle errors
    success = { data =>
      // data is an instance of Product.
    }
  )
}

As you can see above, the controller logic only operates on the case class Product.

Now let’s look at the view side. Let’s assume we use Vue. Our view looks like below:

@(params: play.api.libs.json.JsValue)

<html>
  <body>
    <div id="app">
      <input type="text" v-model="params.name">
      <input type="text" v-model="params.price">
      <button v-on:click="submit()">Submit</button>
    </div>

    <script>
      var app = new Vue({
        el: '#app',
        data: {
          params: @Html(params.toString) // in practice, we should encode JSON in Base64 to avoid cross-site scripting.
        },
        methods: {
          submit: function() {
            // We simply submit `this.params` using AJAX to the controller.
          }
        }
      })
    </script>
  </body>
</html>

In Vue, we don’t do anything with params. We simply use params in textboxes. Notice that the default values are set from our controller. For example, 59999L is converted to 599.99 when we invoke PRODUCT_FORM.fill(Product("iphone", 59999L)).data, and 599.99 is what users see.

We like this approach because:

  • There is a clear boundary. Our view operates on JSON; Our controller immediately converts JSON to Product and only operates on Product.
  • The structure of params is defined in one place, which is in PRODUCT_FORM.
  • The validation and transformation logic is also defined in one place, which is in PRODUCT_FORM.

We had been happy with this approach and play.api.data.Form … until we started using a Seq in our form like below:

case class ComplexProduct(itemName: String, price: Long, images: Seq[String])

val COMPLEX_PRODUCT_FORM = Form(
  mapping(
    "name" -> text,
    "price" -> amount,
    "images" -> seq(text)
  )(ComplexProduct.apply)(ComplexProduct.unapply)
)

A Seq in Play’s form doesn’t work well with JSON. For example, with the explained approach above, ComplexOrder("iphone", 59999L, Seq("iphone.png", "iphone2.png")) would be converted to:

{
  "name": "iphone",
  "price": "599.99",
  "images[0]": "iphone.png",
  "images[1]": "iphone2.png"
}

And our Vue code can’t operate on this kind of array encoding. It’s also tricky and hacky to work with this kind of array encoding. This problem has led us to build our own Play’s form library that is fully compatible with JSON.

Well, I hope this is helpful for anyone looking to implement form submission and validation on Playframework with a Javascript framework.

The working example can be founded here. Though our play-json-form is used in the example, the architecture follows what is explained in this blog.

If you have any question or would like to discuss about anything, please feel free to open a Github issue here. We are always looking for ways to improve.

Instantiate case class with arbitrary values to reduce verbosity in tests

As we are using Playframework with Slick, we have 20+ case classes that directly correspond to tables in Postgresql. We have 10+ case classes that are used for Playframework’s forms and for higher-level representations (e.g. encapsulating two case classes within a single case class). With a large number of case classes, writing tests, specifically faking these case classes, becomes tedious and verbose.

More than often, I don’t care much what the values are. I just want sensible values (e.g. not empty, positive values, not too long). I’ve asked around a few times on Reddit and on Stackoverflow. One good suggestion is to use scalacheck-shapeless.

Scalacheck-shapeless works but isn’t a good fit. The main problem is that scalacheck-shapeless generates an extremely wild value. For example, a generated string might be empty or use Japanese characters. A generated integer might be a large negative number. Because our tests aim to verify normal use cases of our application. We don’t want to perform some sort of fuzzing tests here. We ended up having to define some values in our case class instances (using .copy) to avoid flaky tests. So, the code was still a bit verbose.

Moreover, Scalacheck-shapeless is a little bit too verbose for my taste, especially when we have a special type, say, com.twitter.util.Time, we need to bring into scope an implicit Arbitrary for com.twitter.util.Time. For example:

// import statements are omitted.

case class User(name: String, createdAt: com.twitter.util.Time)

implicit val implicitTime = Arbitrary.apply(Gen.const(Time.now))

val user = arbitrary[User].sample.get

Note that you might think we can encapsulate these lines in a method. We can’t because scalacheck-shapeless requires the type to be known (because of Macro expansion); We can encapsulate these lines using Macros though. In any case, using Macros comes with its own burden which will be explained in the next paragraph.

Then, I turned to Macros for generating a case class instance with arbitrary values. However, the main problem with using Macros is that our compile time (in sbt test:compile) takes 2x longer (from 2-3 minutes to 6 minutes). Ouch. Besides, Macros, whose advantage is to provide compilation safety, isn’t really needed here because the generation is only used in tests. We would have caught an error fairly quickly anyway if there is one.

Eventually, I’ve decided to go with the good old Scala/Java reflection for instantiating case classes with arbitrary values. Here’s what the code looks like:

import scala.reflect.api
import scala.reflect.api.{TypeCreator, Universe}
import scala.reflect.runtime.universe._

object Maker {
  val mirror = runtimeMirror(getClass.getClassLoader)

  var makerRunNumber = 1

  def apply[T: TypeTag]: T = {
    val method = typeOf[T].companion.decl(TermName("apply")).asMethod
    val params = method.paramLists.head
    val args = params.map { param =>
      makerRunNumber += 1
      param.info match {
        case t if t <:< typeOf[Enumeration#Value] => chooseEnumValue(convert(t).asInstanceOf[TypeTag[_ <: Enumeration]])
        case t if t =:= typeOf[Int] => makerRunNumber
        case t if t =:= typeOf[Long] => makerRunNumber
        case t if t =:= typeOf[Date] => new Date(Time.now.inMillis)
        case t if t <:< typeOf[Option[_]] => None
        case t if t =:= typeOf[String] && param.name.decodedName.toString.toLowerCase.contains("email") => s"[email protected]"
        case t if t =:= typeOf[String] => s"arbitrary-$makerRunNumber"
        case t if t =:= typeOf[Boolean] => false
        case t if t <:< typeOf[Seq[_]] => List.empty
        case t if t <:< typeOf[Map[_, _]] => Map.empty
        // Add more special cases here.
        case t if isCaseClass(t) => apply(convert(t))
        case t => throw new Exception(s"Maker doesn't support generating $t")
      }
    }

    val obj = mirror.reflectModule(typeOf[T].typeSymbol.companion.asModule).instance
    mirror.reflect(obj).reflectMethod(method)(args:_*).asInstanceOf[T]
  }

  def chooseEnumValue[E <: Enumeration: TypeTag]: E#Value = {
    val parentType = typeOf[E].asInstanceOf[TypeRef].pre
    val valuesMethod = parentType.baseType(typeOf[Enumeration].typeSymbol).decl(TermName("values")).asMethod
    val obj = mirror.reflectModule(parentType.termSymbol.asModule).instance

    mirror.reflect(obj).reflectMethod(valuesMethod)().asInstanceOf[E#ValueSet].head
  }

  def convert(tpe: Type): TypeTag[_] = {
    TypeTag.apply(
      runtimeMirror(getClass.getClassLoader),
      new TypeCreator {
        override def apply[U <: Universe with Singleton](m: api.Mirror[U]) = {
          tpe.asInstanceOf[U # Type]
        }
      }
    )
  }

  def isCaseClass(t: Type) = {
    t.companion.decls.exists(_.name.decodedName.toString == "apply") &&
      t.decls.exists(_.name.decodedName.toString == "copy")
  }
}

And here’s how we can use it:

val user = Maker[User]
val user2 = Maker[User].copy(email = "[email protected]")

Using reflection is pretty satisfactory because the compile time is still fast and the API is concise (more concise than using scalacheck-shapeless). Here are some other advantages of the code above:

  • You can override the values you care about using the .copy method of a case class.
  • You can customise the generated values in any way you’d like. For example, if the field has the word email in it, then we generate a valid random email.
  • You can easily extend it to support some other types that are not case classes.
  • All values are uniquely generated. This is good because using duplicated values might not catch a certain kind of bugs (e.g. passing wrong values to wrong parameters).
  • It works with Enum. It’ll always choose the first enum value.

I hope this code will be useful for your codebase!

Compilation safety on Playframework's i18n with Scala Macros

Or how Scala Times describes it: “Leverage macros to make compilation fail for invalid key or invalid arguments.”

Playframework’s i18n mechanism isn’t very safe. Well, it’s not as safe as I’d like as a person who prefers Scala because of its safeness. A translation is defined in messages, for example, page.index.hello = Hello {0}. Then, somewhere in a template, we invoke @request.messages("page.index.helllo", "Tanin").

With the above way of doing things, we can easily make a few mistakes. We could misspell the key. We could forget to invoke the method with an appropriate number of arguments. To make the matter worse, Playframework will compile it successfully. You will only see the mistake(s) when you render the page. Your message will look like page.index.helllo (did you notice the three ls?) because the misspelled key isn’t defined. Avoiding mistakes when there are thousands of translations in a project is painful.

I made this exact mistake, and I’ve realised afterwards that there is a way to catch this kind of mistakes with Scala Macros!

Read more

i18n with Playframework and Vue

Playframework has its own internationalization mechanism, which use conf/messages for storing translations. The mechanism is great for server-side rendering. On Vue’s side, there are multiple libraries for internationalization in Vue.js, and it seems vue-i18n is the winner.

Now the question is: How do we use it with Vue.js?

It would be great if (1) vue-i18n uses the Playframework’s conf/messages and (2) the internationalization works well with local development (e.g. hot-reloading).

Read more

Vue.js with Playframework

We’ve finally extracted out our Vue.js plugin for Playframework, sbt-vuefy, and shared it with the world.

It has been quite an effort to make Vue.js work seamlessly well with Playframework. This is especially true when we want to avoid using Single-page application (SPA).

Read more