+972 52-548-6969
Actual scenario testing with Raven
Yesterday I posted about doing scenario testing with Raven, and I showed the concept of what i am doing. This time, I wanted to show what I am actually talking about, and how this is implemented. Here are the current scenarios for Raven.
Each scenario is looks something like this (showing PutAndGetDocument here):
And the second request:
The scenarios are being picked up using:
public class AllScenariosWithoutExplicitScenario { [Theory] [PropertyData("ScenariosWithoutExplicitScenario")] public void Execute(string file) new Scenario(Path.Combine(ScenariosPath, file+".saz")).Execute(); } public static string ScenariosPath { get { return Directory.Exists(@"..\..\bin") // running in VS ? @"..\..\Scenarios" : @"..\Raven.Scenarios\Scenarios"; } } public static IEnumerable<object[]> ScenariosWithoutExplicitScenario { get { foreach (var file in Directory.GetFiles(ScenariosPath,"*.saz")) { if (typeof(Scenario).Assembly.GetType("Raven.Scenarios." + Path.GetFileNameWithoutExtension(file) +"Scenario") != null) continue; yield return new object[] {Path.GetFileNameWithoutExtension(file)}; }; } } }
There are two reasons why I am ignoring explicit scenarios. Adding a class for a specific scenario allows me to run the scenario in the debugger, and also allow me to selectively skip certain scenarios if I need to.
Scenario.Execute is fairly involved, it parse the Fiddler’s saz file, build appropriate request and compare to the expect response, it is also smart enough to handle changing things like ETags and pass them along.
The end result is that I can very easily add new scenarios as I get new features to that requires tests.
Comments
I think I've also had some success with the scenario based approach (state black box)
We had complex mortgage lead distribution rules which were nicely captured in a scenario xml file.
Even built a simple Scenario UI where we could set up current and expected state after the distribution code ran.
My MBUnit test looked like this:
DataSet data; // my mock DB without NH :-)
[Factory]
public string[] ScenarioFile()
{
string dir = ConfigurationSettings.AppSettings[SCENARIO_DIR];
return Directory.GetFiles(dir, "*.xml");
}
[CombinatorialTest]
public void TestOneScenario([UsingFactories("ScenarioFile")] string scenarioFile)
{
data.Clear();
data.ReadXml(scenarioFile, XmlReadMode.IgnoreSchema);
LeadDistributorMock mock = new LeadDistributorMock(DB, data);
Assert.IsTrue(mock.RunAndVerifyDistribution());
}
And some files were:
DailyMax.xml
DistributionGroupPriority.xml
FreeLead_AggregatorPricing.xml
FreeLead_IOBudget.xml
Nice. Seems very similar to RowTests/Factories people did with MbUnit a lot of time ago. I wouldn't say this approach is specific to scenarios, with other kinds of data in files it can be used for unit-testing as well.
Btw why "running in VS" hack is required? I generally just put required files into embedded resources.
Andrey,
Because I want an easy way to be able to debug a single scenario, rather than executing all of them in one shot.
You could also use something like watiN http://watin.sourceforge.net/.
Imran,
No, I could not.
Different purposes all together
yeh just realised that you don't actually have have a web form. I saw 'get' methods and assumed this.
Ayende,
i'm interested in how you are parsing the .saz (zip) file. FiddlerCore doesn't include this ability. I'm interested if you are unzipping and then using FiddlerCore. I'd like to take advantage of some Fiddler functions to give timing information across a graph of requests (html + associated images for instance).
Jay,
I wasn't actually aware that there WAS a fiddler core :-(
I unzipped the files manually, read the stored values and acted upon them, nothing smart or anything like it, I am afraid.
Comment preview