(This .NET Performance by Design review turned out to be a summary of resources, tools and tips & tricks surrounding .NET Performance and Designing for Performance. Hope you enjoy it as much as I did writing it.)
There is a lot you can do in respect to taking the correct steps to make sure you have considered performance to a big enough extent in your design of an application in .NET. What’s with the belt and suspenders mumbo-jumbo in the opening sentence here? Well. First of all; it depends. Second I don’t want to offend the YAGNI/premature optimization guys out there right off the bat. (At least not too much.) The whole thing is a trade off and not an easy concept. But that does NEVER excuse not thinking about it up front. Stick around and I will embellish as well as give a few helpful hints picked up along the way during a full day of “Performance by Design” pre-conference workshop at PDC08.
According to Rico Mariani* there are two rules you need to follow in order to take enough (or perhaps rather the right) height for performance on the .NET platform. (Guess the same rules would apply to any software development project/platform):
Rule #1) Measure
First you have to measure anything at all. If you don’t then you are only guessing and using your artistic freedom. Measuring and interpreting the results is an engineering practice. How else will you know what the capabilities of your application are?
Rule #2) Do your homework
If you don’t take the time to find out what it is you need to acquire in terms of performance you are not going to reach it!
A good attitude to have as a performance eager software developer is to realize that if you are really good at performance work you will only be wrong about 19/20 times! Meaning in fact that the real difference between top of the line performance experts compared to those who are just good at performance work lies in the ability to quickly be able to find the good things to measure in any given scenario.
Another interesting fact is that the quote “Premature optimization is the root of all evil!” Is most often attributed to Donald Knuth is in fact not Knuth's original quote. More over it is miss quoted and misused to mean that all optimizing work done early is wrong! The full quote goes something like:
We should ignore small performance gains 97% of the time: Premature optimization is the root of all evil!
This was originally stated by Charles Antony “Tony” Richard Hoare most known for his inception of the little algorithm known as Quicksort. This puts the quote in a whole new light and gives meaning to Rico’s rules above. Nothing in this quote states that you should not think about performance up front and this is a very important distinction!
What you really need to have in your organization is a “performance culture” that consist of three steps; Budget, Plan, Verify.
- Budget: You can’t make a plan before you know what budget you have to spend. You can’t make a budget before you know what you need to deliver. You cannot know what you are expected to deliver before you talk to the customer. The idea here is to break out of the destructive behavior of “I cannot measure performance on it before I build it”. That’s just not true! There are many things you can know to test (in your plan) if you just find out what it is you are supposed to build.
- Plan: Planning for performance stops when you know you can tolerate the risk of mitigating the rest of the performance tuning that you could do. Sounds easy right?
- Verify: What do you need to verify; see budget and plan above! Performance that cannot be verified does not exist.
So what does usually go wrong in performance? Here are a few ordinary and tragic things:
I) Programs take on a dependency that they cannot afford. You have to understand the costs of your dependencies in terms of your critical resources and you need to use dependencies in the context they were intended.
II) Programs use an algorithm that is fundamentally unsuitable. What you need to do is model your real algorithm with real data before you assume it is OK!
III) Programs do a lot of work that doesn’t constitute “forward progress”. I order to get to the root of this problem you need to score your algorithm compared to the minimum amount of work needed to get this work done. In any app or framework there is overhead that is not really “forward progress”. Then you need to begin to remove stuff that is not needed. We need to find the stuff we wish we didn’t need to do and try to remove some of it. This is the #1 reason code runs slower than it could be!
IV) Programs are designed to do more than they need to do. You have to focus on the customers needs only!
Next up was Vance Morrison*. He started off with how to measure the performance we want. Time to get concrete:
Low tech: Use Stopwatch (System.Diagnostics) to measure your code. (Tip: Set the computer to “high performance” to make the results more even and easier to compare between runs.
Medium tech: Use Measureit a tool featured in Vance Morrisons two part MSDN Magazine article: Measure Early and Often for Performance, Part 1 and Measure Early and Often for Performance, Part 2 to make better use of the Stopwatch. The program acts as a wrapper around stopwatch and enables you to make small tests on how performing different code samples are. Very useful for early research in your development projects.
High tech: Sample Based (CPU) Profiling. Use the performance capabilities inside of Visual Studio. Make sure you get > 1 000 samples. Increase the work if you don’t (or the sampling frequency with the downside of bigger test impact on your code. Check out the “call tree view” hotpath. Also look at the callee caller view – if you know your code does not make recursive calls (otherwise your samples will not add up correctly).
Future: Instrumenting your apps for performance.
What about multi threading? Read What Every Dev Must Know About Multithreaded Apps.
For web apps there’s a good tool for analyzing roundtrips graphically: Visual Round Trip Analyzer.
The key to measuring performance is to make it easy enough so that you DO it!
* Rico Mariani is the chief designer for Visual Studio and he has apx 30 yrs experience with performance work from his carrier.
** Vance Morrison is an Architect on the .NET Runtime Team, specializing in performance issues with the runtime or managed code in general.
Last up was Mark Friedman who talked a lot about performance for ASP.NET. This was the wrong topic to end the day on for me. Because a lot of this was stuff I already knew I had a hard time keeping focused on finding the few gems in his very fine presentation that I could have used. In fact I started to write this piece instead. Sorry Mark – you still did a really good presentation.
I hope you find some useful advice in this review. I did when I attended today and I will continue on to read the resources quoted. That’s all we can hope for – that we take this more seriously and do this better. Performance is not assured at the end. It is planned for via a budget based on client requirements and validated to bring it all home.
Cheers,
M.
Technorati Tags:
PDC08,
Microsoft,
.NET
posted @ Monday, October 27, 2008 2:49 AM