Friday, January 22, 2010

Generics performance vs. non-generics performance

Today I was reading some book on the .net development and found there interesting thing.

Guy explains "Why to use Generics?"
He wrote that Frameworks 1.0 and 1.1 did not support Generics, so developers were using Object.
He says, that generics offers two significant advantages over using the Object class:
1) Reduced run-time errors
That is because type-safety.
2) Improved perfomance
Casting requires boxing and unboxing, which slows performance. Using of Generics doesn't require casting or boxing, which improves run-time performance.

And then funny thing...
He put a box which looks like:
Real Word
(his name)
I haven't been able to reproduce the performance benefits of generics; however, according to Microsoft, generics are faster than using casting. In practice, casting proved to be several times faster than using a generic. However, you probably won't notice performance differences in your applications. (My tests over 100,000 iterations took only a few seconds.) So you should still use generics because they are type-safe.

OMG! I could not believe in his words. As per me this should be BULLSHIT, unless I 100% missed something there.


Test

I wrote a really quick verification like:

namespace TestGenericsPerfomance
{
    class Program
    {
        internal static void Main(string[] args)
        {
            Stopwatch stopWatch = new Stopwatch();

            //Working with value objects
            //Generic wins 100%

            stopWatch.Start();
            ArrayList nonGenericArrayList = new ArrayList();
            for (int i = 0; i < 10000000; i++)
            {
                nonGenericArrayList.Add(i);//takes Object so boxing is performed here...
            }
            stopWatch.Stop();
            Console.WriteLine(string.Format("Int32: ArrayList (boxing): {0:0}", stopWatch.Elapsed.TotalMilliseconds));

            stopWatch.Restart();
            List<Int32> someCustomersIds = new List<Int32>();
            for (int i = 0; i < 10000000; i++)
            {
                someCustomersIds.Add(i);
            }
            stopWatch.Stop();
            Console.WriteLine(string.Format("Int32: List<Int32> (generic): {0:0}", stopWatch.Elapsed.TotalMilliseconds));

 
            //Working with reference objects
            //Generic still wins, but not so sure..

            Customer sharedCustomer = new Customer();

            stopWatch.Restart();
            ArrayList nonGenericArrayListCustomers = new ArrayList();
            for (int i = 0; i < 10000000; i++)
            {
                nonGenericArrayListCustomers.Add(sharedCustomer);//no boxing.. jut putting the same reference
            }
            stopWatch.Stop();
            Console.WriteLine(string.Format("Customer: ArrayList (no boxing): {0:0}", stopWatch.Elapsed.TotalMilliseconds));

            stopWatch.Restart();
            List<Customer> genericBasedOnInterface = new List<Customer>();
            for (int i = 0; i < 10000000; i++)
            {
                genericBasedOnInterface.Add(sharedCustomer);//just put the same reference in generic list
            }
            stopWatch.Stop();
            Console.WriteLine(string.Format("Customer: List<Customer> (no boxing): {0:0}", stopWatch.Elapsed.TotalMilliseconds));

        }

        private class Customer
        {
            public string Name = "Andriy Buday";
            public string Buy(string what)
            {
                return "I'm glad I bought that.";
            }
        }
    }
}

And the result is:

Int32: ArrayList (boxing): 2443
Int32: List<Int32> (generic): 294
Customer: ArrayList (no boxing): 586
Customer: List<Customer> (no boxing): 315
Press any key to continue . . .

As you see generics are much faster. Also I searched over internet and found a lot of different stuff that says that generics provide better performance.

My Opinion

You would said what kind of book do I read. Is it "C# for Complete Dummy"?
No, that is Microsoft's training Kit for exam 70-536.
Even if this was a book "C# for Dummy" it should not contain mistakes. And even if this is a book for kids it should not contain wrong thing. Yea.. it could be very simple, but not wrong!

I thought to write e-mail to that guy, but then decided that there is no need in this.

Just be careful when you read books and others thoughts, even mine :)

7 comments:

  1. By the following link: http://dotnetbutchering.blogspot.com/2008/09/net-generic-perfomance-myth-and-tony.html
    I found that guy's post like:
    -----------------------
    Tony said...

    By the way, I'm releasing a new version of that book in November of 2008. It's 50-60% brand new (I re-wrote the other author's chapters), and all known errors have been corrected.
    -----------------------

    ReplyDelete
  2. Hi Andriy,

    i am also reading the Training Kit Book by Tony and i am just as disapointed by it as you are.

    But i think your benchmark code might contain some flaw:
    It compares System.Collections.Generic.List to an System.Collections.ArrayList. I do not know the internas of both of them but i guess that the implementation (not only the fact that one is using generics while to other is not) differs.
    So you might end up measuring not only generics vs. non-gernerics, but measuring the internal implementations of both collection types.

    Secondly i think your might be to short / has to few itterations. So you end up doing a micro benchmark, which could lead to missleading results.

    I wrote another version of your test that simply compares a generic class to a non generic, where both classes have exactly the same logic.

    Here it gooes:
    public void Test()
    {
    int runs = 100000000;

    int valueType = 10;
    Customer referenceType = new Customer();

    Generical valueTypeGenerical = new Generical();
    NonGenerical valueTypeNonGenerical = new NonGenerical();
    Generical referenceTypeGenerical = new Generical();
    NonGenerical referenceTypeNonGenerical = new NonGenerical();

    Stopwatch stopWatch = new Stopwatch();

    stopWatch.Start();
    for (int i = 0; i < runs; i++)
    {
    valueTypeNonGenerical.Do(valueType);
    }
    stopWatch.Stop();
    Console.WriteLine("Non Generical - Value Type: {0}", stopWatch.ElapsedMilliseconds);

    stopWatch.Reset();
    stopWatch.Start();
    for (int i = 0; i < runs; i++)
    {
    valueTypeGenerical.Do(valueType);
    }
    stopWatch.Stop();
    Console.WriteLine("Generical - Value Type: {0}", stopWatch.ElapsedMilliseconds);

    stopWatch.Reset();
    stopWatch.Start();
    for (int i = 0; i < runs; i++)
    {
    referenceTypeNonGenerical.Do(referenceType);
    }
    stopWatch.Stop();
    Console.WriteLine("Non Generical - Reference Type: {0}", stopWatch.ElapsedMilliseconds);

    stopWatch.Reset();
    stopWatch.Start();
    for (int i = 0; i < runs; i++)
    {
    referenceTypeGenerical.Do(referenceType);
    }
    stopWatch.Stop();
    Console.WriteLine("Generical - Reference Type: {0}", stopWatch.ElapsedMilliseconds);
    }

    public class Generical
    {
    public void Do(T value)
    {
    value.GetHashCode();
    }
    }

    public class NonGenerical
    {
    public void Do(Object value)
    {
    value.GetHashCode();
    }
    }

    The results for me are:
    Non Generical - Value Type: 1553
    Generical - Value Type: 658
    Non Generical - Reference Type: 6766
    Generical - Reference Type: 6962

    I have repeated this test a couple of times and generics with reference types sometimes is faster and sometimes is slower that non generics with reference types, that said they are quite equaly in terms of performance.

    I hope i did not made any mistake myself :-)

    BTW: I realy realy dislike the 70-536 Training Kit Book. It contains so many mistakes (i counted 25 serios mistakes so far and been only up to Chapter 4) that it is a shame and i realy hope there not as many mistakes in the actual exam itself.

    ReplyDelete
  3. David, thank you so much for this great comment - the biggest I ever had. It is even quite better than blog post itself :)

    I was also getting almost equal results for the generics and non-generics when working with reference types, but mistakenly highlight it if generics are quite better in this case.

    I mean this two results:
    Customer: ArrayList (no boxing): 586
    Customer: List (no boxing): 315



    and yes, book is very dry to read. Probably you would need to get second edition.



    And good luck at exam!!!

    ReplyDelete
  4. Hi Andriy,

    thank you for your kind words.
    I am glad i could be of a least some help.

    >> and yes, book is very dry to read. Probably you would need to get second edition.

    Do you know what's realy sad?
    Actually i DO have the second edition :-)
    And it still sucks a lot.

    I wish the Author was here, i would give him such a piece of my mind!

    I always think that people who teach should not make that many mistakes.
    Some are okay, after all we all are just humans.
    But this book conatins way to much mistakes and false statments.

    And if that would not have been enough already, the Author does not even follow the Microsoft Framework Design Guidelines.
    I mean what kind of sick style is a parameter name like this for example: _value ? (I mean the totaly unnecessary underscore).
    That's like seeing a 3 year old coding :-)

    However, i hope the book will at least prepare me a bit for the exam.

    BTW: Did you take the exam yet?

    I am scared a bit, because some questions in the pratice test are realy hard.

    Seems like you have to know each and every class of the BCL deeply.

    And don't get me started about the .NET security system, which i never liked, because of it's uneccessary complexity. (Which is btw replaced by a much simpler security model in .NET 4).

    I do .NET Development for many years now, but i do not have the complete MSDN Documentation in my head, but who does ? :-)

    Sincerely yours,
    David

    ReplyDelete
  5. David, I did not take exam yet. Would like to take it to the end of this month.

    I'm now at 11-12th chapters and which are about security, and as you said it is just uneccessary complex. It's not possible to have that all in mind.

    And yes, questions are like you need to know exact names of classes and methods and all the stuff that you just could simply google. But programming with .NET is probably more about gluing it all together.

    Thanks one more time, man! I really appreciate your comments here.

    ReplyDelete
  6. Hi Andriy,

    i wish you good luck with your exam.
    I am planning to take it somwhen this sommer.
    I will take 70-536 and -70-502 (WPF Exam) together to qualify for MCTS.

    Do you plan to take any other exam?

    I sympahtize with you reading the chapter about security.
    It must be horrible. But i'll see myself when i get there :-)

    But i guess we do almost anything to gain the microsoft certification...

    I just think that the exam is missdirected.
    I think it is not so imported for one to know each and every class.
    As you said it's much more imported to understand how to bring it all together.
    And i think understanding the concepts behind .NET/C# is more imported than beeing able to remeber class names :-)

    But i guess it's the same with all exams (not only for certifications but also like the ones in schools and universities):
    They're all just about remembering instead of understanding.

    No wonder most smart people sucked in school.
    School is all about effort and nothing about intelligence :-)

    BTW:
    When i think about it, it's pretty ironic to even take the exam.
    I mean after many years of successfull developing with .NET (also in huge enterprise projects),
    i finaly will take an exam to actually prove that i know how to use .NET :-)

    I guess it's the same with you, is it? :-)


    Anyway, i hope we will suceed in taken the exam!


    BTW:
    It would been realy nicde of you if you could drop me a email when you finished your exam.
    I would realy appreciate to hear how it was.
    And of course to congratualte you on succeeding on it (which i am pretty sure you will!).

    You'll find my email on my website: http://www.rent-a-developer.de
    (Can't post it here because of spamers...)

    Sincerely yours,
    David

    ReplyDelete
  7. I have 2+ years of experience developing .NET applications. Looks that not so much as you have, but the situation is the same... I spend 8 hours in VS everyday and exam still doesn't look to be trivial, but it is just introduction exam to all others.

    I have plan to pass also 505, so this will gave me MCTS.

    And of course you will be informed once I will pass the exam.

    Thanks for so kind words.

    ReplyDelete