Evaluating languages for building internal DSLs
Michael Dorfman presents an interesting question about building internal DSLs.
[You should] point out for those of us who have not (yet) tasted the Boo koolaid exactly what makes this language more suitable to DSLs than VB9, C#3, F#, IronPython or IronRuby.
It's not enough to say "You can't build DSLs in C# / VB, they aren't flexible enough (at all)", you need to mention which language features are missing.
So, why am did I say that C#/VB/Java/etc are not flexible enough to build usable DSLs out of?
Let us consider what we want to achieve when we are thinking about a DSL. The first thing that comes to mind is that is should be about the domain, so I want to have as few programming related stuff as possible, and the syntax should be simple and clear to read.
Statically typed languages are historically rigid in their structure, and force you to follow specific syntax rules. Dynamic languages are far more relaxed in that manner. Boo is a statically typed language that masquerade as a dynamic language, so it sits with the dynamic languages on this matter.
I am not familiar enough with F# to give an answer on that, but both ruby & python can certainly build DSLs. DSLs are very common in ruby, and I think that python can do much the same, although I do wonder about the white space sensitivity thing regarding DSLs.
But let us explore for a moment the idea of build a DSL in a statically typed language. We want to define a DSL for executing tasks in a build process. We want to execute a process and copy some files. C# (for java, just change the using to imports):
using My.Imaginary.Build.Framework; using My.Imaginary.Build.Framework.IO; public class MyCustomTask : TaskBase { public MyCustomerTask() : TaskBase("Execute process & copy some files") { } public override void Execute() { Execute("msbuild.exe", "myProject.sln"); CopyFiles("bin\debug\**.*", "ftp://staging-srv/myApp/"); } }
And in VB.Net:
Imports My.Imaginary.Build.Framework Imports My.Imaginary.Build.Framework.IO Public Class MyCustomTask Inherits TaskBase Public Sub New() MyBase.New("Execute process & copy some files") End Function Public Overrides Sub Execute() Execute("msbuild.exe", "myProject.sln") CopyFiles("bin\debug\**.*", "ftp://staging-srv/myApp/") End Sub End Class
There is a lot of syntactic noise here that we need just to make the compiler happy. We aren't working with a DSL, at best, we are working with a fluent interface (more on that later).
We could probably do this, though:
Description("Execute process & copy some files"); Execute("msbuild.exe", "myProject.sln"); CopyFiles("bin\debug\**.*", "ftp://staging-srv/myApp/");
And then pre-process the file in order to put all the rest of the noise around it, but that is going to be bite us if we need to do things such extending it further (how do we add a using statement here? how do we define a method?), or have anything even slightly more complex than a series of sequential operations?
Sure, I can think of solutions for all of the above, but they #include stuff that is (a) hard, (b) unpleasant, (c) complex and (d) leaky.
When you are building DSLs, you want as few constraints as possible from the language, and that is not possible to do in static languages. But, I can hear you say, we have Linq and extension methods and fluent interfaces now, surely that is a solved problem.
Not really, I believe that I have pushed C# 2.0 as far as it is possible to go in regards to fluent interfaces, and I have looked very closely at what I can do with C# 3.0 new syntax. The answer is that it is great for programmers, but not really useful for developing DSLs. You are still going to have to deal with making the compiler happy, and your ability to modify the syntax of the language itself to fit the ideas that you want to express is nil.
In comparison, here is how you can express the same idea in Boo:
Task "Exexcute process & copy some files: Execute "msbuild.exe", "myProject.sln" CopyFiles "bin\debug\*.*", "ftp://staging-srv/myApp"
There is nothing here that is here to make the compiler happy, we are focused on what we want to do, not on the syntax.
Beyond that, we have the Boo compiler itself, which allows you to go far beyond that. This goes a bit beyond the scope of DSL, but this is valid Boo code:
receive: message msg as ChangeAddressMessage if msg.AddressId is null: transaction: address = Address.FromMessage(msg) address.Create() message msg as ChangeAddressMessage: transaction: address = LoadAddress(msg.AddressId) Address.FromMessage(address, msg) address.Save() message other: raise MessageNotUnderstood()
Boo doesn't have the receive / message primitives, nor a transaction keyword, but it is trivial to add them (well, trivial to add the keywords, the messaging infrastructure is not trivial).
This expose an interesting concept that I have yet to talk about, Boo allows me to do code transformation at compile time. This means that even if the way that we want to express something is not the way we would like it to execute, we can have the best of both worlds, we can express it clearly, and translate it to the way it should be executed.
That is not something that you can really do in most other languages, certainly not in statically typed ones. Yeah, I know about assembly weaving, that is now what I call a DSL. I think that Ruby can do something similar, using ParseTree and eval(str) {don't think Ruby has a way to translate expression list to code, but again}, but I am not sure.
So, anyway, those are my reasoning for using Boo.
Comments
Both F# and Nemerle have macros (code transformation at compile time).
I used to be a huge fan of the lispy/schemey languages. Both have a very good DSL story - essentially, that's the main model in those languages for getting things done – iteratively build a tower of increasingly domain specific languages using the very powerful sexp based macro systems that those languages have. They also coalsece a fairly large body of software practice and experience to the extent that Greenspun's Tenth Rule of Programming cautions us: "Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified bug-ridden slow implementation of half of Common Lisp".
My heart belongs to Haskell now, rather than lisp and has done so for the last few years. A great deal of research into DSL’s has gone on in the Haskell community. Since the Haskell community is pretty strong within MS I’ve not been so surprised to see a glimmer of this work reappear c# 3.0 in the guise of Linq (thanks to Erik Meijer, who is continuing his good deeds with Volta).
Anyway, the point of this longish rambling post, such as it has one, is to encourage you to look at Haskell (and perhaps lisp or scheme) with a view to seeing how these communities tackle DSLs. The downside of looking so far outside the box is that the box begins to seem very, very small.
Alexey,
do you have a link to the F# macros?
I am familiar with Nemerle
David,
Thanks, I'll be certain to do so.
Ayende,
Here's a link to F# macro:
http://www.strangelights.com/fsharp/wiki/default.aspx/FSharpWiki/Macro.html
Thanks for the reasoned reply.
Based on what you've written, Boo definitely seems to hold some advantages for DSL creation.
In terms of your proposed book, I suggest that you flesh out the information outlined above, and investigate F#, IronPython and IronRuby, so that you can clearly make the case for your use of Boo, as it is a language with a lot less mindshare than some of the others.
Excellent argument!
Comment preview