darrencauthon's posterous

My IoC container of choice is Unity.

Why?  Well, it was the first IoC container I used on a real project.  And it worked out, so I used it on the next, and then the next, and then the next.  I knew how it worked, I trusted it, and I haven’t switched because I have no reason to take on another risk.

However, I am familiar with all of the popular IoC containers, except for Spring.Net.  I am a contributor to MVC Turbine, which contains a service locator abstraction for every popular IoC container (except Spring.Net).  I have worked on the service locator for every supported container.  I’ve even gone as far as spending hours on an AutoFac and Hiro implementation.

So while I’m a Unity fan, I’m no slouch when it comes to the rest.  In fact, I don’t care what IoC container people use, so long as they use it the right way.

The Accusation

A coworker of mine is a Castle Windsor fan.  So a couple days ago when he came across a post by Philip Mateescu that showed that Unity was orders of magnitude slower than *EVERY* IoC container, he made sure to show me.  But I still took a look, and found these types of numbers:

              10k - 1st  10k - 3rd  100k - 1st  100k - 3rd
------------------------------------------------------------
Autofac                25         23         166         163
Castle.Windsor         35         39         342         328
Ninject               128        133       1,053       1,040
Spring.Net             59         36         223         229
StructureMap          114        118         289         348
Unity               6,012      5,945      62,278      59,292

Yikes, right?  Unity is getting killed.  I’ve said before that the differences in the speed of the IoC containers isn’t enough of a factor to choose one over the other, but these types of numbers suggest that Unity is doing something incredibly wrong.  If these numbers were true, I’d stop using Unity today.

The Error

Philip posted his code on Github, so I took a look.  And sure enough, it didn’t take long to find the problem:

But more importantly, why is that IsRegistered call in his tests?  Philip certainly knows it’s being registered, as that’s exactly what he’s testing.  Plus, in multiple places in his blog post, he states that he’s testing the Resolve()-ing of each container... but he’s not.  He’s actually testing the combination of the IsRegistered method (which he does for every container) in addition to the Resolve().

So, I took out all of the type checks so each test just ran the resolve call.  And not surprisingly, the numbers were drastically different:
 

The Lie

Normally, I’d consider this just a simple error and leave it at that.  However, the problem is -- Philip knows of this issue.  I checked him on Twitter, and there were three people who showed him this problem and told him how to correct it.  His responses?

@philipmatix: @roryprimrose in absence of IsRegistered you're left with handling ResolutionFailedException. pluggable apps need that check.

and

@philipmatix: @slaneyrw that's most likely true, but in a pluggable app you either check IsRegistered or handle ResolutionFailedExceptions.

Basically, because *Philip* believes that we should be calling this IsRegistered method before we resolve out of the container, then it’s perfectly acceptable to consider that part of the cost of resolving.

That is an opinion.  Not one that I agree with, but let’s say that I do.  And let’s say that everybody does.  Still, you have to make it clear how you are measuring something when you present numbers about those measurements.  He doesn’t.  Without someone actually spending the time to analyze his code and his measurements (which the majority of his readers aren’t going to do), they’re going to leave his post with the impress that Unity really is as slow as he states.  Which is not true, and he knows it’s not true.

Think about it this way.  Let’s say I wrote a blog post about how many calories are in the most popular hamburger buns.  Using Philip’s logic, I can say that a hamburger bun below is over 800 calories...

… because I have a personal belief that nobody should ever eat that hamburger bun without a big, delicious hamburger patty and fixings like this:

This is obviously a very bad way to measure things.  I believe Philip should either change his test methodology and make the proper changes to the results, or he needs to make it SO CLEAR to anybody who reads that post that he’s not testing just the Resolve method.

Throwing Stones

It’s bad enough that Philip has to report these bad numbers, but he also has to take a few swipes at the Unity team as he’s doing it.  In his post he questions their sanity, says Unity is weird, and a makes few other jabs.  And unlike most blog posts, he does not accept comments (which would be enough to help to fix the errors in his post).

In summary, he makes up these ridiculous tests, doesn’t explain what they actually are and takes unfair shots at devs who produced something of real value to a lot of people.  But worst of all:  He leaves the post out there despite knowing the issues, which will only lead to confuse more and more people that hit his blog post over the months and years.