Having become quite interested in languages that support deployment on both Java and .NET, then diving into Scala to find it’s a good language, the obvious next step is to try using Scala on .NET. I found a little time over the weekend to try that out with good success. Notably, I haven’t tried this in a real project yet so there may yet be some significant catches, but so far it looks really quite promising.

Simple .NET Exe

First things first, let’s write some Scala then compile it to a .exe file that uses .NET. Our scala class is the simple Hello World:

import System.Console
object test extends Application {
  def main(args: Array[String]) = Console.println(“Hello .NET World!”)
}

Note that I put this in a file called test.scala as part of my fiddling to make this work – I don’t think that should be required but it certainly doesn’t hurt.

Compilation to .NET requires two steps. Both are best performed by opening the “SDK Command Prompt”1. This looks very much like the normal Command Prompt but it defines a bunch of important environment variables so that all the .NET tools are on your path and working correctly.

1. If you haven’t already, as the Scala docs say, copy the mscorlib.dll file from the .NET SDK (I had to use find to locate it then picked one of the results at random) to the %SCALA_HOME%\share\scala\lib directory
2. Again, if you haven’t already, install the scala-msil support by running:

   sbaz install scala-msil
   

3. Then compile to the MSIL intermediary format with scalac-net2:

   scalac-net test.scala
   

Now you should have a test.msil file alongside the test.scala file. The next step is to convert the file to an actual EXE file with the ilasm tool3:

ilasm test.msil

Now you should have test.exe alongside your test.msil and test.scala files. Before you run it though, you’ll need to find the predef.dll and scalaruntime.dll files in the Scala distribution and copy them to the current directory4.

Finally, you can now run test.exe and see “Hello .NET World!” printed.

Creating DLLs for Visual Studio Projects

The same tool setup can be used to create DLLs for use with Visual Studio projects. I created a C# web application project in Visual Studio, then modified my test scala class slightly:

import System.Console
class test extends Application {
  def getStuff) = “Hello .NET World!”
}

Then recompile it with scalac-net as before. Since we want a DLL instead of an EXE file we add a /dll parameter to the ilasm invocation:

ilasm /dll test.msil

Now we’ve got a DLL ready to go. I copied the three DLLs, test.dll, predef.dll and scalaruntime.dll into my Visual Studio project to make them easy to access. Then:

1. Right click on the project in Visual Studio
2. Select “Add Reference…”
3. Click the “Browse” tab
4. Select the three DLLs you copied in.
5. Click “OK”

You can now access your Scala class from C# quite easily. I edited Default.aspx and added:

<%= new test().getStuff() %>

Code completion even worked. Running the project correctly shows “Hello .NET World!”.

Stuff Yet To Discover

There’s a lot of stuff I don’t know about yet, largely because I’m starting from pretty much no knowledge of .NET programming, but also a few Scala related things:

• Unlike Ruby and Python, Scala doesn’t have a class library that’s designed to stand alone – it somewhat assumes the Java libraries will be there. Things like IO are left to the Java classes so they’d need to be abstracted away with separate Java and .NET implementations. It remains to be seen how limiting that could be.
  • Worse, I’ve seen some blog posts mention that not all of the Scala library is actually available on .NET. The XML libraries directly reference a bunch of Java IO stuff for example.
• You can combine multiple classes in one DLL. When I still had test as an object, I got ‘test’ and ‘test$’ put into the DLL. When I have multiple .scala files to compile I’m not sure at what point or how they all get combined into one DLL. I’m guessing somewhere along the line putting all the files as arguments should do the trick. Otherwise it will get pretty unwieldy.
• I’ve got no idea what actual deployment of this stuff is like. Last time I touched .NET that was a complete nightmare even without Scala. Hopefully if the project references all the Scala DLL files they will get bundled up in whatever way Visual Studio would normally do when preparing to distribute.

1 – I believe this was installed along with the .NET SDK 2.0 package I grabbed – I didn’t notice it being installed with just Visual Studio but I could be wrong. ↩

2 – scalac-net seems to be a handy little wrapper around scalac that sets -target=msil and adds the right DLLs to the classpath. ↩

3 – this is the part that really requires the SDK Command Prompt and the environment variables it sets up.↩

4 – I think under Windows DLL loading rules they can actually be anywhere on the PATH. ↩

When I was in Australia visiting Ephox’s awesome engineering team recently, it was suggested that we should be seriously looking at using Scala for some or all of the server side development we have coming up. The key benefit that was described to me was based on productivity – I certainly think that has merit but I’m somewhat more interested in it being a potential solution to the new cross platform problem1. Options like JRuby and Jython are also a possibility in this space, but Scala fits more nicely into a Java programmers mind-set and is designed with the JVM in mind, so while it pushes a lot of boundaries, ultimately it fits with the JVM and Java very nicely2.

So I read through the O’Reilly Programming Scala book on the plane ride home3, then jumped in and built a small but real-world project with Scala and integrated it into a Java servlet project. I think it’s given me a pretty good grounding in what Scala can do, the benefits it brings and the general state of play.

The Language

The Scala language is fantastic. Easy to learn if you have a solid programming background and just a smidgeon of functional programming knowledge4. I’ve always been on the fence between compile time and runtime typing, but I think Scala strikes the perfect balance. It’s strictly typed at compile time, but uses a lot of inference so you don’t have to constantly declare types. I quickly came to find that there are two places where it’s incredibly handy to always specify types:

1. Parameters. Scala requires it anyway, but it is vital documentation for anyone who wants to call that method.
2. Return types. Not usually required by Scala, but by being explicit you can catch quite a lot of weird errors that crop up and isolate them to the right method. If you leave the return type as inferred, the compile error you get usually comes out on the caller of a method and can be somewhat confusing. The most common error here is effectively not returning something or returning two different types by accident (usually because you expected a different return type from a method you called). This isn’t hard and fast – short, simple methods are usually best left inferred.

Having closures is definitely handy, but the benefit isn’t really in closures themselves. What really makes a difference is the libraries designed to be used with closures5, and the functional programming style that comes naturally in Scala. Looking back at the code I’ve written, the structure really looks like something that would make sense as a mathematical equation – each method does one thing, consistently and without side-effects. More importantly though, the breakdown of methods that results winds up being quite logical and helps reusability a lot. I find that getting the split points for methods right in Java is much harder and requires more conscious thought.

Traits – wow do I ever love these things. The first key benefit I found for them was unit testing. In Java, one class would tend to be responsible for loading and processing it’s data. In Scala, traits make it so easy for the class to declare an abstract variable for the data and just provide the processing. Traits then step in to provide the loading part and provide a concrete implementation of the variable. It becomes obvious to have a LoadFromDatabase trait, a HardCodedString trait or a LoadFromFile trait – now where the data comes from is clearly separated. In tests you just use a HardCodedString trait and there’s no need for Mocks or Stubs.

Interestingly, there is a wonderful synergy between traits and JNDI. Traits can’t accept constructor arguments and even if they could it’s ideal to just declare that you want a new Widget() with DatabaseLoading and have it just work. JNDI gives you that ability – the DatabaseLoading trait just grabs the datasource from JNDI and there’s no need to tunnel parameters through to it like you would otherwise.

I’m also a big fan of the uniform access principle that Scala uses. Essentially, from the caller’s perspective, there is no visible difference between accessing a public variable and accessing a public method. So you can have a base class that declares an abstract variable, then implement it with a concrete method in a subclass and visa-versa. It’s an incredibly simple concept that gives a lot of flexibility and avoids a ton of boilerplate code.

The Tools

This is the weak point of Scala, but it shows some promise.

Testing Libraries

There are actually quite a few testing libraries in Scala – all of them somewhat difficult to get working inside an IDE or Ant build, but all quite solid and stable if you use their test runners. The xUnit equivalent is ScalaTest, I never got it working with Ant or Eclipse so gave up and moved on to Specs. I love it6. The Aussie dev team recently switched from CamelCase test method names to using underscores7 for readability, but Specs takes that a few steps further by letting you use strings. For example:

“Administrator users” should {
    “have a password at least 8 characters long” in { …code goes here… }
    “be in the admin group” in { …code goes here… }
}

The test output is really easy to read using the Specs runner, but it gets much less readable when using JUnit to run the tests via ant. You can avoid the outer ‘should’ block which helps readability in JUnit but it’s still not great. Fortunately, IntelliJ has support for Specs directly and lays things out pretty well, plus the normal runner isn’t bad.

Standard Toolchain

The Scala distribution comes with a good set of tools. A scala command for executing scripts and scalac for compiling classes are the main two. Both work really nicely and the scala tool is particular good for experimenting with things – especially when you’re trying to learn the type inference system.

The single biggest drawback is that scalac can’t compile Java code, so you can either have Scala code depending on Java code or Java code depending on Scala code, but can’t do both easily. This isn’t all bad, as it really encourages you to break the project down into independent libraries and reduce coupling but it would be nice to more easily mix languages.

IDEs

Wow, these are terrible. The Eclipse plugin is incredibly buggy and is practically unusable. NetBeans seemed to work well but is very short on features. IntelliJ has a good set of features but has overall stability issues since you need to use an early access build of the IDE – the plugin itself seems quite stable. IntelliJ also has some support for Specs and ScalaTest rather than having to run them through JUnit which is quite nice, but it didn’t seem able to just find all the tests in my project and run them – it was limited to one package at a time. Getting Scala working in IntelliJ is an absolute nightmare and you’ll spend forever searching Google for answers. There are magic libraries you need to put in place and very cryptic error messages if you don’t have them – at least it’s a once off. Once the Scala plugin works with a released version of IntelliJ I think that will be the winner, otherwise I may have to deal with NetBeans8.

Java Libraries

They work really nicely, except that Scala doesn’t use the normal java.util Collections framework. Scala has it’s own collections framework with equivalents and they are dramatically better but it does make it difficult to inter-operate with Java libraries that use Java Collections. Fortunately, Scala provides wrappers for the Java Collection types to make them act like Scala collections, but it doesn’t have wrappers for going the other way. It’s definitely not the end of the world, but does make you think a bit more about where the Java to Scala leap should be – sometimes writing a few key classes in Java can make things dramatically easier. Judicious use of implicit conversions, a scary but powerful feature of Scala, can be a big help too.

I do like the ability to specify values in Scala as @BeanProperty and have the compiler automatically generate JavaBean compliant getters and setters (or just getters if it’s a read-only value as is common in Scala).

There can also be problems with tools that manipulate bytecode since the bytecode Scala generates is compliant with the JVM specs, but often quite different to what javac would product.

The Performance

I can’t tell the difference between Java code and Scala code. That’s roughly what other people are saying too. There are an awful lot of classes that come out of Scala code because of all the closures and traits and the like so it’s not going to work if you are tight on space such as in applets. Server side where processes are long-running and JAR size isn’t a critical concern, it will work perfectly. I would guess9, that there are a very large number of short-lived objects being created and discarded with Scala due to the use of implicit conversions and all those closures, but garbage collectors are so heavily optimised for that use case that I can’t see any evidence that it makes a difference and the performance testing I’ve read about bears that out.

The Scala compiler implements tail-recursive optimisation and even supports an annotation so that you can mark a method as tail-recursive and get a compile error if it’s not.

In terms of scalability – Scala advocates claim that functional programming and immutable data structures that are heavily used in Scala are a key tool, if not the solution, for scalability. I don’t know and don’t really care – the applications I’m likely to write will fit nicely into the “scale by handling many requests at once” model rather than having to break up the work of each individual request over multiple threads. The problem will come in the database or web server level, not my application code. I’m watching Tim Bray’s Concur.next series with interest though.

Overall

Scala is very much a tool I’m glad to now have in my toolkit. It’s not yet ready for prime time purely because of the poor IDE support, but it is excruciatingly close. I can see me advocating it’s use in Ephox quite heavily if it deploys to .NET smoothly as the cross-deployment is a massive benefit and would be worth the pain of sorting out the tooling, but it would be hard to justify if that isn’t viable yet – especially given the potential for Ruby or Python to work on both J2EE and .NET.

I don’t think it will be long before the Scala toolsets catch up and it becomes an extremely compelling option for Java server-side development.

1 – it remains to be seen how stable/useful the .NET version of Scala is – I’ve been focussed on the Java side of things for now ↩

2 – this adds to the scepticism around the .NET version until I find time to really test it out though ↩

3 – yay for power points in cattle-class ↩

4 – a few months of Haskell in uni counts if you paid attention. In fact, Haskell is a particularly useful background to have as it is a very pure functional language and uses a similar static typing with inference system as Scala does. ↩

5 – thus I question how useful closures would be in Java ↩

6 – Actually, I think ScalaTest can do the same style of tests but I got Specs working first. ↩

7 – sacrilege! ↩

8 – I love the concept of it’s ant based build system, yet somehow the implementation drives me nuts every time I try to use it ↩

9 – I’m too lazy to profile this at the moment ↩