Evaluating languages for building internal DSLs

time to read 5 min | 969 words

Michael Dorfman presents an interesting question about building internal DSLs.

[You should] point out for those of us who have not (yet) tasted the Boo koolaid exactly what makes this language more suitable to DSLs than VB9, C#3, F#, IronPython or IronRuby.
It's not enough to say "You can't build DSLs in C# / VB, they aren't flexible enough (at all)", you need to mention which language features are missing.

So, why am did I say that C#/VB/Java/etc are not flexible enough to build usable DSLs out of?

Let us consider what we want to achieve when we are thinking about a DSL. The first thing that comes to mind is that is should be about the domain, so I want to have as few programming related stuff as possible, and the syntax should be simple and clear to read.

Statically typed languages are historically rigid in their structure, and force you to follow specific syntax rules. Dynamic languages are far more relaxed in that manner. Boo is a statically typed language that masquerade as a dynamic language, so it sits with the dynamic languages on this matter.

I am not familiar enough with F# to give an answer on that, but both ruby & python can certainly build DSLs. DSLs are very common in ruby, and I think that python can do much the same, although I do wonder about the white space sensitivity thing regarding DSLs.

But let us explore for a moment the idea of build a DSL in a statically typed language. We want to define a DSL for executing tasks in a build process. We want to execute a process and copy some files. C# (for java, just change the using to imports):

using My.Imaginary.Build.Framework;
using My.Imaginary.Build.Framework.IO;

public class MyCustomTask : TaskBase
{
	public MyCustomerTask() : TaskBase("Execute process & copy some files")
	{	
	}

	public override void Execute()
	{
		Execute("msbuild.exe", "myProject.sln");
		CopyFiles("bin\debug\**.*", "ftp://staging-srv/myApp/");
	}
}

And in VB.Net:

Imports My.Imaginary.Build.Framework
Imports My.Imaginary.Build.Framework.IO
 
Public Class MyCustomTask Inherits TaskBase

	Public Sub New()
		MyBase.New("Execute process & copy some files")
	End Function
 
	Public Overrides  Sub Execute()
		Execute("msbuild.exe", "myProject.sln")
		CopyFiles("bin\debug\**.*", "ftp://staging-srv/myApp/")
	End Sub
	
End Class

There is a lot of syntactic noise here that we need just to make the compiler happy. We aren't working with a DSL, at best, we are working with a fluent interface (more on that later).

We could probably do this, though:

Description("Execute process & copy some files");
Execute("msbuild.exe", "myProject.sln");
CopyFiles("bin\debug\**.*", "ftp://staging-srv/myApp/");

And then pre-process the file in order to put all the rest of the noise around it, but that is going to be bite us if we need to do things such extending it further (how do we add a using statement here? how do we define a method?), or have anything even slightly more complex than a series of sequential operations?

Sure, I can think of solutions for all of the above, but they #include stuff that is (a) hard, (b) unpleasant, (c) complex and (d) leaky.

When you are building DSLs, you want as few constraints as possible from the language, and that is not possible to do in static languages. But, I can hear you say, we have Linq and extension methods and fluent interfaces now, surely that is a solved problem.

Not really, I believe that I have pushed C# 2.0 as far as it is possible to go in regards to fluent interfaces, and I have looked very closely at what I can do with C# 3.0 new syntax. The answer is that it is great for programmers, but not really useful for developing DSLs. You are still going to have to deal with making the compiler happy, and your ability to modify the syntax of the language itself to fit the ideas that you want to express is nil.

In comparison, here is how you can express the same idea in Boo:

Task "Exexcute process & copy some files:
	Execute "msbuild.exe", "myProject.sln"
	CopyFiles "bin\debug\*.*", "ftp://staging-srv/myApp"	

There is nothing here that is here to make the compiler happy, we are focused on what we want to do, not on the syntax.

Beyond that, we have the Boo compiler itself, which allows you to go far beyond that. This goes a bit beyond the scope of DSL, but this is valid Boo code:

receive:
	message msg as ChangeAddressMessage if msg.AddressId is null:
		transaction:
			address = Address.FromMessage(msg)
			address.Create()		
	message msg as ChangeAddressMessage:
		transaction:
			address = LoadAddress(msg.AddressId)
			Address.FromMessage(address, msg)
			address.Save()
	message other:
		raise MessageNotUnderstood()

Boo doesn't have the receive / message primitives, nor a transaction keyword, but it is trivial to add them (well, trivial to add the keywords, the messaging infrastructure is not trivial).

This expose an interesting concept that I have yet to talk about, Boo allows me to do code transformation at compile time. This means that even if the way that we want to express something is not the way we would like it to execute, we can have the best of both worlds, we can express it clearly, and translate it to the way it should be executed.

That is not something that you can really do in most other languages, certainly not in statically typed ones. Yeah, I know about assembly weaving, that is now what I call a DSL. I think that Ruby can do something similar, using ParseTree and eval(str) {don't think Ruby has a way to translate expression list to code, but again}, but I am not sure.

So, anyway, those are my reasoning for using Boo.