Case Study: the first RavenDB’s production deployment story
When I sat down over some years ago to actually decide how to go about dealing with that document database idea that wouldn’t let me sleep, I did several things. One of them was draw down the architecture of the system, and how to approach the technical challenges that we were seeing. But a much more interesting aspect was deciding to make this into a commercial product. One of the things that I did there was build a business plan for RavenDB. According to that business plan, we were supposed to hit the first production deployment of RavenDB round about now.
In practice, adoption of RavenDB has far outstripped my hopes, and we actually had production deployments for RavenDB within a few months of the public release. What actually took longer than deploying RavenDB was getting the stories back about it . I am going to start posting those as soon as I get the authorization for the customers to do so.
The following is from the first production deployment of RavenDB…
1. Who are you? (Name, company, position)
Henning Christiansen, Senior Consultant at Webstep Fokus, Norway (www.webstep.no) I am working on a development team for a client in the financial sector.
2. In what kind of project / environment did you deploy RavenDB?
The client is, naturally, very focused on delivering solutions and value to the business at a high pace. The solutions we build are both for internal and external end users. On top of the result oriented environment, there is a heavy focus on building sustainable and maintainable solutions. It is crucial that modules can be changed, removed or added in the future.
In this particular project, we built a solution based on NServiceBus for communication and RavenDB for persistence. This project is part of a larger development effort, and integrates with both old and new systems. The project sounds unimpressive when described in short, but I'll give it a go:
The project's main purpose was to replace an existing system for distribution of financial analysis reports. Analysts/researchers work on reports, and submit them to a proprietary system which adds additional content such as tables and graphs of relevant financial data, and generates the final report as XML and PDF. One of the systems created during this project is notified when a report is submitted, pulls it out of the proprietary system, tags it with relevant metadata and stores it in a RavenDB instance before notifying subscribing systems that a new report is available. The reports are instantly available on the client's website for customers with research access.
One of the subscribing systems is the distribution system which distributes the reports by email or sms depending on the customer's preferences. The customers have a very fine-grained control over their subscriptions, and can filter them on things such as sector, company, and report type among other things. The user preferences are stored in RavenDB. When a user changes preferences, notifications are given to other systems so that other actions can be performed based on what the customers are interested in.
3. What made you select a NoSQL solution for your project?
We knew the data would be read a lot more than it would be written, and it needed to be read fast. A lot of the team members were heavily battle-scarred from struggling with ORMs in the past, and with a very tight deadline we weren't very interested in spending a lot of time maintaining schemas and mappings.
Most of what we would store in this project could be considered a read model (à la CQRS) or an aggregate root (DDD), so a NoSQL solution seemed like a perfect fit. Getting rid of the impedance mismatch couldn't hurt either. We had a lot of reasons that nudged us in the direction of NoSQL, so if it hadn't been RavenDB it would have been something else.
4. What made you select RavenDB as the NoSQL solution?
It was the new kid on the block when the project started, and had a few very compelling features such as a native .NET API which is maintained and shipped with the database itself. Another thing was the transaction support. A few of us had played a bit with RavenDB, and compared to other NoSQL solutions it seemed like the most hassle-free way to get up and running in a .NET environment. We were of course worried about RavenDB being in an early development stage and without reference projects, so we had a plan B in case we should hit a roadblock with RavenDB. Plan B was never put into action.
5. How did you discover RavenDB?
I subscribe to your blog (http://ayende.com/Blog/) :)
There was a lot of fuss about NoSQL at the time, and RavenDB received numerous positive comments on Twitter and in blog posts.
6. How long did it take to learn to use RavenDB?
I assume you're asking about the basics, as there's a lot to be learnt if you want to. What the team struggled with the most with was indexes. This was before dynamic indexes, so we had to define every index up front, and make sure they were in the database before querying. Breaking free from the RDBMS mindset and wrapping our heads around how indexes work, map/reduce, and how and when to apply the different analyzers took some time, and the documentation was quite sparse back then. The good thing is that you don't really need to know a lot about this stuff to be able to use RavenDB on a basic level.
The team members were differently exposed to RavenDB, so guessing at how long it took to learn is hard. But in general I think it's fair to say that indexes was the team's biggest hurdle when learning how to use RavenDB.
7. What are you doing with RavenDB?
On this particular project we weren't using a lot of features, as we were learning to use RavenDB while racing to meet a deadline.
Aside from storing documents, we use custom and dynamic indexes, projections, the client API, and transactions. We're also doing some hand-rolled Lucene queries.
On newer projects however, with more experience and confidence with RavenDB, and as features and bundles keep on coming, we're doing our best to keep up with the development and making the best use of RavenDB's features to solve our problems.
8. What was the experience, compared to other technologies?
For one thing, getting up and running with RavenDB is super-easy and only takes a couple of minutes. This is very different from the RDBMS+ORM experience, which in comparison seems like a huge hassle. Working with an immature and rapidly changing domain model was also a lot easier, as we didn't need to maintain mappings. Also, since everything is a document, which in turn easily maps to an object, you're sort of forced to always work through your aggregate roots. This requires you to think through your domain model perhaps a bit more carefully than you'd do with other technologies which might easier allow you to take shortcuts, and thus compromise your domain model.
9. What do you consider to be RavenDB strengths?
It's fast, easy to get started with, and it has a growing community of helpful and enthusiastic users. Our support experience has also been excellent, any issues we've had have usually been fixed within hours. The native .NET API is also a huge benefit if you're working in a .NET environment
10. What do you consider to be RavenDB weaknesses?
If we're comparing apples to apples, I can't think of any weaknesses compared to the other NoSQL solutions out there aside from the fact that it's new. Hence it's not as heavily tested in production environments as some of the older NoSQL alternatives might be. The relatively limited documentation, which admittedly has improved tremendously over the last few months, was also a challenge. The community is very helpful, so anything you can't find in the documentation can normally be answered by someone on the forum. There's also a lot of blog posts and example applications out there.
I find the current web admin UI a bit lacking in functionality, but hopefully the new Silverlight UI will take care of that.
11. Now that you are in production, do you think that choosing RavenDB was the right choice?
Yes, definitely. We've had a few pains and issues along the way, but that's the price you have to pay for being an early adopter. They were all quickly sorted out, and now everything's been ticking along like clockwork for months. I'm confident that choosing RavenDB over another persistence technology has allowed us to develop faster and spend more time on the problem at hand.
12. What would you tell other developers who are evaluating RavenDB?
I have little experience with other document databases, but obviously tested a bit and read blog posts when evaluating NoSQL solutions for this project. Since we decided to go with RavenDB there's been a tremendous amount of development done, and at this time none of the competitors are even close featurewise.
Comments
What's odd is that his team members are familiar with O/R mappers (so can get started right away) but they decide to go into unfamiliar territory and have a hard time learning the cornerstone of proper querying of that new technology: indexes. All this because the O/R mapping setup wasn't suitable to meet a deadline.
It might be me, but that simply doesn't add up. As in: he just wanted to use NoSQL and found the reason to use it afterwards. That's fine, but at least admit it.
@Frans,
I didn't read that into what they were saying. I read that for a changing Data Model, a traditional ORM was a pain plus he said that the developers had pains with ORMs in the past.
Also, they probably didn't know about the problems they'd have with Indexes until they started the project, whereas they knew about the ORM issues that they had experienced in the past.
I'll admit i kind of saw the same thing as Frans did reading this. "We have super ultra tight deadlines, and ORMs quirks slow us down....so instead lets start exploring unknown technology!!"
I consider myself pretty adventurous when it comes to technology, but when I have a super deadline, I tend to become pretty conservative... Seems like they like living on the edge :)
I think that is highly dependent on what Plan B was. Plus he said he and some of his team members already gave it a spin.
I have to say, as someone who also has several years experience with ORMs, getting up and running with raven (even before dynamic indexing) is pretty straight forward and easy.
They had a chance to try completely different technology and they succeeded. Not because they have made a series of well thought of decisions but because the technology worked in their case so the plan B could be ditched. That's nice, shows that Raven can be used in some real world situations, but it also suggests their requirements could be satisfied with a RDBMS as well. I'd like to hear more stories, especially where the technology meets some limits - of database size, of performance, of manageability etc and would also to see how Raven behaves with millions of records, thousands of users and hundreds transaction per second.
I have to agree with Frans, this is not a story of success - its a story of blind dumb luck!
@Frans, @Francois,
When looking over this again, I see that the index thing might seem like it was a big problem; it wasn't. It was the biggest one we had though, and that's why it made it into this interview / case study.
A couple of us had tested RavenDB and had a fair understanding of how indexes work before we decided to go for RavenDB as our persistence platform, but once you grok it, it's quite easy to explain to others. It's just different.
I guess you could say we traded a set of, known, old and tedious problems for a new and lesser known one, but at this point we had already decided that a NoSQL solution would be the best fit for us, so using an ORM was already way down on the list. Any NoSQL solution would be new territory for us and bring along its own set of problems.
Also, this project was not just about reaching a deadline, it was about laying the foundation for a number of other projects to come, so our perspective was a bit more long term than what comes across in the interview.
The decision to go for RavenDB was not an easy one, and a lot of time was spent discussing, reviewing, testing, etc. before the decision was made. To include all the reasoning behind this decision was obviously out of scope for this interview.
@Me, to suggest that this is a story of dumb luck is just ridiculous.
@Henning
No I dont think its rediculous. The story reads that way. Look at it this way, a client came to you to replace an existing product, you say you have a team with ORM experience, so you understand the problems and (presumably) know how to ease the friction with its use. However, you chose to go completly in the other direction, and, by using a product that you have no experience with put the entire project at risk.
Great that you had a plan B, and even better that never had to be activated.
You lucked out that RavenDB happens to be a great product with an active support group.
You lucked out that the learning curve was not as steep as it could have been.
You lucked out that the problems you faced were only minor and easy to overcome.
I dont beleive that saying you had tinkered with RavenDB prior to implementing it in a commercial project counts as 'experience'.
If I were the client, and when you suggested that you were going to use an untried and untested and unknown product for the storage, I would have run a mile, especially if the timeframe was tight.
So yes, blind dumb luck.
If you were trying to prove the technology, and the project was not highly critical, perhaps I'm being overly harsh. However these case studys are being used as tools to sell RavenDB to potential clients.
I personally think RavenDB is a great product and I am in the process right now of trying to prepare a case for using RavenDB with our next product for the business I work for. I have been looking forward to reading some real world experiences to assist with this. Your story, while ultimately sucessful, does not suggest that RavenDB was the perfect fit, and would probably have been able to be sucessfuly delivered in the same timeframe using an ORM and RDBMS with a more 'traditional' approach.
I want to read / hear stories where RavenDB comes to the rescue and solves some tricky and critical problems that could not be overcome using a traditional solution.
Comment preview