+972 52-548-6969
It isn’t a feature that is killing you, it is the intersection of features
Over time, projects get more features. And that is fine, as long as they are orthogonal features. It is when those features overlap that they are really putting the hurt on us.
For example, with the recent Changes API addition to RavenDB, one of the things that was really annoying is that in order to actually implement this feature, I had to implement this to:
- Embedded
- Client Server
- Sharded
- Silverlight
And that one is easy. We have bugs right now that are happening because people are using two or three bundles at the same time, and each of them works fine, but in conjunction, you get strange results.
What should happen when the Unique Constraints bundle creates an internal document when you have the Versioning bundle enabled? How about when we add replication to the mix?
I am not sure if I have any good ideas about the matter. Most of the things that we do are orthogonal to one another, but when used in combination, they actually have to know about their impact on other things.
My main worry is that as time goes by, we have more & more of those intersections. And that adds to the cost of maintaining and support the product.
Comments
Sounds like a tough one...
Maybe it helps if you'd know the different areas that can be affected by a bundle inside RavenDB and make sure that bundles state which areas they affect. This could help to identify conflicting bundle features.
The cheap way out would then be to not allow 2 bundles to run if they touch the same area.
A concept of Meta-Bundle could help to reconciliate N conflicting bundles in that it is itself a bundle, but would have detailed knowledge of 2 or more other bundles that would be otherwise conflicting, such that it can orchestrate those in a meaningful fashion...
hm
What about a testing environment that tests out every combination of bundles against a set of standard scenarios - setting up a db, doing crud, map-reduce etc.
Sounds quite simple, so I guess you've considered this.
The next remark might sound harsh and stupid, but it's not. I'll explain afterwards.
"For features to add to your product, it's best not to listen too well to your customers but more to yourself".
Users of software products only see their own project, and your product connected with it. In almost all cases, these users are interested in features which help their project. So they ask for feature X which would help their project tremendously. If you implement feature X, you have helped that user out. However, it might very well be, X is actually a feature which isn't of use by many other users. Or, and this is often the case, X is actually a symptom of the lack of another, different feature Y. Implementing X will then actually hurt the product, even though it might not seem that way.
While the user who brings up the feature request for X might have helped you finding a better feature to add, Y, and thus it might seem good to listen to that user, it is also dangerous to do so: it might be Y is a feature which will make your product change direction, make it look like a different product or make it less maintainable, as Y might branch out in a direction that wasn't explored very well.
Unless the ISV is run by retards, they very well know what they wanted to achieve when they released a product, which direction they want to go in. Features added to the product should make the product move into that direction and keep it moving in that direction. Of course they have done research what kind of directions they could choose with a product and which one made the most sense. Feedback from users can strengthen the results of the research or show that mistakes have been made. If the direction chosen for a product is the wrong one, the product will fail. Changing direction might help a bit, but chances are the product will simply be doomed to fail anyway, simply because it jumps all over the place, there's no philosophy behind it anymore. The best choice in that case is then to simply do a new product.
Why my wall of words as reply to your post? Well, your post screams "I listened to my customers and implemented what they wanted". Which is perfectly fine, doing so shows you care about your users / customers. But as I tried to explain above: it has a downside, and you learned about it the hard way. You're not alone, I've made the same mistakes as well: a new toolkit/library/software product has to grow, fast, so you're eager to please whoever wants to give feedback. But doing so might very well bring you from the path you had designed up front or make the product jump all over the place, fragment it and it will be hard to fix it. After all: a feature added is very hard to remove, if not impossible.
So take a step back, look at what you want to offer, what you had in mind for the software. Then offer that. Chances are, the features you now added, fall inside the features you want to provide.
it is not a silver bullet, and it can be difficult to identify the correct shape of things, but instead of saying explicitly "use-versioning" (that will lead to one budle knowing the others, like you said) you could try to identify some more high level, conceptual "logical characteristics" and use them: those will drive the behaviour of the other bundles o r the other part of the system.
Like an "IsTemporary" flag, or "IsMetadata" or "IsNotTrackable" o something like that. I know it could be hard to identify these concepts, and may lead to situations like "yeah, cool, but setting IsMetadata what actually does ?" but, you know, you have to stop at some point.
(I'd like to add, that by not (always) listening to the ISV's users it doesn't mean the ISV doesn't care for its customers/users, the customers/users aren't the right group of people to ask which direction the product should go in).
Also, the solution to your problem isn't a technical one.
Frans, I would have thought that the very idea of having bundles is to not burden the core product with features that are only relevant to specific scenarios and hence provide the "backdoor" (usually called extensibility) for those that have hit some corner case.
Frans, I think Ayende is not complaining about too many features being added in general, but rather about how you prevent the combinatorial explosion you get when features (or bundles in this case) start interacting in unforeseen ways.
It sounds like a good sign of the increasing maturity of RavenDB :)
I attest that the bulk of dev resources is spend in non-fonctional requirement, this includes ergonomy, code structuring, correctness/testing/contract, performance, memory consumption, product infrastructure (licensing...).
Code Structure is the key to master the classical combinatorial explosion you are facing. More precisely:
-> use abstraction carefully, typically patterns like the Visitor can help transforming [N*M] into [N+M] impl
-> another appropriate pattern is mediators objects, that avoid features to know about each others (a bit like a form makes its children controls interact, without knowing each others)
-> layer your components, I know I am redundant on that one, but this is the right way to avoid your features impl to become entangled in the future
-> developers are tempted by global states when features combinatory explosion is luring. Avoid singleton and static fields referring to mutable state at all prices, it will ruin the testability and maintainability of your code
@Ayende: This is why I have little faith in the idea that an architecture can "emerge" without guidance as long as everyone is doing the "simplest thing that can possibly work". The complexity involved in the intersection of those features is unavoidable, it can only be managed - if someone is focused on recognizing it up front and directing the evolution of the app so as to minimize the pain.
@Frans: Absolutely. Part of serving your customer is recognizing when a requested feature will impact existing ones. An ISV is in the position of not only recognizing that, but also having to make the decision to include or not.
Having the same issue in the JavaScript world.
Gene, the simplest thing that can possibly work does work if you
@Frank: you also need to add "don't mind a lot of technical debt (redundant and/or vestigial parts, etc.)". Igor Lobanov put together a good post a while back that rain's pretty hard on the emergent architecture parade.
Frank, How do you define an "area"? If a bundle decide it needs to create a document, and another bundle deals with document creation, what is supposed to happen? What about a bundle the run async, like the replication bundle? It may run minutes or hours after the first bundle run, and what should its behavior be?
Nick, a) First, we have 14 bundles at last count, that gives us a test matrix of close to 200 options. b) Second, how do you define what is the appropriate behavior for each instace?
Frans, In general, I agree with you. Except that I think that there is something fundemntal that is missing here. There are actually several separate issues.
It gets complex when you are mixing all of those together. For example, adding a feature such as the Changes() API is something that need to take to account all of those.
The reason that I said that I agree with you is that from the get go we had a good way to tell the user that this isn't a feature that we wanted to implement, and here is the extension point that they can use to make it happen on their own.
Njy, We have some of those (is virtually deleted, is system document), but the problem isn't with that. See my reply to Frans about the growing costs of adding features.
I don't think 200 options is a problem. And then you'll want to test different size combinations - 3 packages, 4 packages etc so there will be 1000s. Seems reasonable we can automate computers to do this kind of work, though.
As for appropriate behaviour, I implied that you could run through some use cases and verify as much works as possible. You probably can't hope to catch everything. But you can verify the raven db does the main things it should do. Although if you have to specify specific acceptance criteria for each combination, then maybe that is a bit too difficult.
Your the man knee deep in the code. I'm certain you know a lot more than me.
Frans, What you are talking about here is the notion of a vision for the product. Letting the users define what the vision for the product is would almost always be a mistake. I discussed this here:
http://ayende.com/blog/103425/donrsquo-t-give-a-chicken-access-to-your-schedule-feature-set
Frank, That is the idea, yep.
Gene, Oh, architecture can emerge. But only if you have good architects working on the product. If you have devs working on it, even very good ones, they will all tend to optimize to local solutions, driving project to the ground.
Andrzej, Yes, I found out about that about two years ago. Bundles have consisted (and well defined) ordering.
@Ayende, I don't think there's a true consensus on the terminology, but it seems like most use "emergent architecture" for the "it just happens" variety (Darwinian evolution) versus "evolutionary architecture" for the iteratively designed type ("intelligent design"?). I very much agree that the architect role is key to making sure someone is paying attention to the big picture.
And even using the definitions above, an architecture can emerge...whether a sustainable architecture can emerge is a different question. :-)
It looks like a standard story of a product maturing. You envision an architecture which you think will best support any potential changes/growth of a product. You implement the said architecture and at some point in time you realize that the architecture does not fully support you any more and you have to fight it to continue. My guess that the real problem (although please correct if I am wrong) is that you do not know at this point in time how to improve your architecture to make further development easier (or as you would say "frictionless"). IMHO there is no magic bullet, the only solution would be to have a brainstorming to come up with new ideas. Unfortunately this requires a group of smart and competent people that understand the problem and have enough experience and knowledge to be productive participants (and in my experience such groups are very rare). If you do manage to come up with better architecture, then its time for 2.0 :)
Comment preview