+972 52-548-6969
ÜberProf and Continuous Integration
One of the features that keep popping up for ÜberProf is that people want to use that in CI scenarios, usually to be able to programmatically check that they don’t have things like SELECT N+1 popping up, etc.
With build 562 of ÜberProf, this is now possible. How does this works? We now have a command line interface for ÜberProf, with the following options:
/CmdLineMode[+|-] (short form /C)
/File:<string> (short form /F)
/ReportFormat:{Xml|Html} (short form /R)
/Port:<int> (short form /P)
/Shutdown[+|-] (short form /S)
Starting up with no options will result in the usual UI showing up, but you have a new mode available for you. Let us say that you want to output the results of your integration tests into a format that you can easily work with programmatically. Here is how it can be done:
nhprof.exe /CmdLineMode /File:Output.xml /ReportFormat:Xml <-- starts listening to applications
xunit.console.exe Northwind.IntegrationTests.dll
nhprof.exe /Shutdown <-- stop listening to applications and output the report
The example is using NH Prof, but the same holds for all the other variants.
The way it works is very simple and should be pretty easy to integrate into your CI process. The XML output can give you programmatic access to the report, while the HTML version is human readable.
One thing that you might want to be aware of, writing the report file is done in an async manner, so the shutdown command may return before writing the file is done. If you need to process the file as part of your build process, you need to wait until the first profiler instance is completed. Using PowerShell, this is done like this:
nhprof.exe /CmdLineMode /File:Output.xml /ReportFormat:Xml
xunit.console.exe Northwind.IntegrationTests.dll
nhprof.exe /Shutdown
get-Process nhprof | where {$_.WaitForExit() } <-- wait until the report export is completed
Please note that from a licensing perspective, the CI mode is the same as the normal GUI mode. On one hand, it means that you don’t need to do anything if you have the profiler already. On the other, if you want to run it on a CI machine, you would need an additional license for that.
Comments
is there a way to fail a build if XProf finds some issues were introduced?
Krzysztof,
You get the XML, you can read it and decide to abort if the profiler has alerts to report.
ah ok.
I was thinking rather along the lines of non-zero return code
Krzysztof,
No, I explicitly do not want to do that. There is too much subjectivity in there.
Just curious, how did you set up your project to perform as both a WPF or WinForm app as well as a console app? I've been wanting to do the same thing, but from what I've read it's almost impossible to do correctly in .NET.
Ted,
It isn't possible.
The issue relates to how windows decides how to execute, each EXE has a flag in its header that decide how this is done.
What I did was hook into the console subsystem and attach myself to the parent console. Not ideal, but it works.
Sounds like it would be worth adding a /Wait argument to be used alongside shutdown. You might need to build a separate nhprof.console.exe application for that though, depending on how the exe is configured.
(I hope the nhprof startup isn't also asynchronous since that will create a potential for race conditions...)
The behavior that @Krzysztof wants could be provided by a separate report processing utility that has the option of returning different exit codes based on observed results. Alternately it would be useful to have some MSBuild or NAnt integration that returns statistics as optional output properties for subsequent handling in the build script.
attaching to the parent console is indeed the only way to do it, but it does have side issues (like it's just emitting output to the console, the user can still interact with the console as if nothing happens, like entering commands and pressing enter, which made me decide to go for a console variant of my .exe instead of use the hack). Isn't it easy to do btw? your main logic is in a centralized dll anyway, not in the ui, so using a console variant isn't that hard and avoids the mess with attaching to a parent console.
Frans,
The main problem is that the nature of the work is async by nature, you have to create a process that starts listening, doing this as a console app is tricky.
Does it have to be async in all scenarios?
In an integration test, specifically designed to check for unwanted query behavior, I wouldn't mind something like...
using (var context=ProfilingContext.SetupFor.....)
{
//Do stuff
Assert.That(context.NumberOfQueries,Is.EqualTo(5));
}
Please check:
ayende.com/.../...re-programmatic-integration.aspx
Comment preview