+972 52-548-6969
Reproducible Errors
For some reason, I am talking a lot about errors recently, didja notice?
Today I had our QA inform me that if we fill some form completely (without leaving any optional stuff out), it will not execute the correct post processing. (I am cutting the conversation here, mind you).
That particular form is critical in the application. It also have about 100 fields, 95 of them are optional.
We could consistently reproduce the error on the QA machine, but I couldn't figure out how something like that could happen. What it worse, I couldn't isolate the problem. When we entered the fully loaded form, it failed. When we entered partial information, it worked. Trying with different subsets always worked.
This is a critical form, did I mention that?
At one point I got annoyed with trying to enter all the permutation, and pulled out Fiddler. Several minutes afterward, I had a Boo script that was making requests against my pages (couldn't use fiddler, needed windows authentication). Now, I could scan through the permutations in a far easier manner. I quickly spot the issue, we had a validation error in one of the fields, we were allowing bad data.
When we were entering just a subset of the data, we always put correct information there, when we had to deal with entering 100 fields, we always put random data there. That was a very quick fix, but getting to that took a long while.
Comments
Boo script? - care to elaborate for me on that please?
Gregor,
A simple boo script that threw requests at the server, basically WebRequest.Create(), with authentication and POST parameters.
It seems to me that it's a mistake to have a form that large for many reasons, validation messages, reproducing errors, maintainability, etc.
@Martin,
Did you read the 95 fields optional part?
Sorry not being from the States i had no idea what you were referring to.
Cheers
What exactly are you missing?
No I agree with Martin even if 95 of them are optional. Its better to split the form if possible, but I think they are talking about filling in a form to provide data for MS CRM, which means its very much controlled by how MS CRM wants the data.
Hi Ayende
How about using watin and the new watin recorder for these kinda problems? http://watintestrecord.sourceforge.net/
The QA guys probably can't create boo scripts while, with maybe a little help at first, they can create a reproduction script with the Watin test recorder and in that way help you out a lot.
Or are you talking about a winforms app?
Ugh... testers who can't code... On one hand: very sad. On the other: I hope my competition believes in this kind of organization.
Ayende, 100 fields.... That's just nasty stuff.
Comment preview