I've added a permanent news page for the Richmond .NET Code Camp to my site. It has news and information you'll want if you're a .NET developer in the mid-Atlantic region.

http://www.gotnet.biz/Blog/page/Richmond-NET-Code-Camp-News.aspx

Check it out!

A few days ago, I stumbled on an article by Amit Raz about the SortedList<K,T> on Dev102.com. In the article, which compares and contrasts the SortedList collection class in the .NET BCL to the SortedDictionary class, Amit concludes with, "So what is the SortedList good for? Beats me. I deem it useless." His conclusion seemed to be predicated on the fact that the Add() method would throw an exception if the programmer attempted to insert a duplicately keyed entry into a SortedList. However, this is documented behavior. And both the SortedList and the SortedDictionary exhibit that same behavior. An indexer exists on each of those collections that will allow the insertion of a duplicately keyed item. For both classes, duplicates replace the original values when using the indexer.

I had a difficult time following Amit's logic but he's a bright fellow so I wanted to find out if the SortedList really was useless as he had proclaimed. On this page, Microsoft provides a table that describes the benefits and relative drawbacks of these two seemingly similar classes. Long story short, the key advantages of the SortedList<K,T> are that it uses less memory than the SortedDictionary<K,T> and that some of its members that return keys and values are indexed. OK, so the SortedList uses less memory. But what's that latter claim about?

Well, the SortedList<K,T> has a few members that the SortedDictionary<K,T> does not. Among them are:

• int IndexOfKey( K key )
• int IndexOfValue( T value )

Kicking around in this class in Reflector, one sees that the internal storage for the SortedList<K,T> is a pair of arrays: one for the keys which is kept in sorted order and one for the values which is kept in insertion order. When using the IndexOfKey method noted above, Array.BinarySearch() is used to perform an efficient search for the desired key. However, the IndexOfValue method uses a brute force (O(n)) scan to find the requested value. Another special case advantage of the SortedList is that insertions are O(1) for data inserted in sorted order whereas for the SortedDictionary, the average insertion cost is around O(log n). In fact, for data already in sorted order, the SortedDictionary pays a bit of a penalty on insertion because of the required balancing of the tree structure used to store the information. More on that later. So, if you have small lists and the keys are already sorted before insertion, the SortedList might be a good choice. If your keys are not sorted, insertion into a SortedList could be as bad as O(n). So it may be that you have to know something about your data to use the SortedList in a way that makes it worthwhile. Not understanding your data, could make the SortedList worthless, as Amit claims.

I wrote a small test harness that exercises the SortedList<K,T> and SortedDictionary<K,T>. I've included a link to the source code below. The application runs a battery of tests using a small list of 1,000 items on each type of dictionary and the same battery using a large list of 40,000 items. The main window looks like this:

These results show an average test run on my Windows Server 2008 machine using the .NET Framework 3.5. For the small list of 1,000 items, the SortedList seems to be a bit more efficient than the SortedDictionary with respect to time, despite the fact that the keys are not in sorted order for that test. However, when it comes to the larger list of 40,000 items, the SortedDictionary is the clear winner for both insertions and removals. But what about memory? Remember that Microsoft's MSDN topic said that the SortedList can be more memory efficient? I put 10 seconds of time in between the 4 test groups shown on the screen shot above and ran the .NET memory profiler to see what was happening. It's not all that conclusive, in my opinion. Here's a graphic showing the Gen 0, 1 and 2 heap sizes over the lifetime of the test. Perhaps you can help me analyze what you see:

There is a very large spike during the third test in the Gen 1 heap size when the large data set of 40,000 values in placed into the SortedList. Most of that newly allocated memory seems to move to the Gen 2 heap when the last test kicks off, returning the Gen 1 heap almost back to it's former value. The internal implementation of the SortedDictionary is based on a private, internal class in System.Collections.Generic called TreeSet<T>. In an upcoming blog post, I will be examining the TreeSet<T> class in detail. It uses a special kind of binary tree implementation known as a red-black tree. Why Microsoft didn't expose this incredibly cool class, I don't understand. So I suppose I should do that, right? Until next time...

Source Code for the DictionaryTestHarness Application (6KB)

I presented tonight (10 July 2008) to the Richmond .NET User Group. We had a pretty good turnout, I'm guessing 40 to 45 developers. I gave this same presentation at my office today as a dry run and as a training opportunity within the company. It's so good to see the developer community eager to learn. I've attached my slides and the three demonstrations projects I used in this post. I'll be giving this same presentation to the Charlottesville .NET User Group next Thursday (17 July 2008). The abstract we put on both user group websites follows:

Silverlight is a client-side technology. So it’s not really a part of your SOA strategy, right? You may want to think twice about that. SOAP and WSDL support are coming to the web desktop via Silverlight. And Silverlight has good client support for REST+ JSON/POX and RSS/ATOM-based web services, too. During this discussion, we’ll dive into data serialization, security and cross-domain access policy capabilities inside Silverlight 2 Beta 2. We also talk about the nuances and pitfalls of provisioning your web services for an Internet audience. This presentation will be heavy on coding, demonstration and interactive discussion.

Powerpoint Presentation (289KB)

Twitter solution showing how to invoke a cross-domain RESTful service by way of an in-domain SOAP service bypassing the cross-domain access policy problem. (842KB)

REST solution showing how to create RESTful services in WCF and how to consume RESTful services in Silverlight (307KB)

Silverlight syndication solution showing how to consume cross-domain RSS and Atom feeds using the SyndicationFeed class. (11KB)

I wrote a handy little class called Impersonator a while ago to assist me in doing Windows impersonation. There are dozens of examples of this type of thing on the net. However, none of them were as "handy" as I prefer. By handy, I mean idiot-proof, of course. I like the IDisposable pattern a lot (I use the word pattern here loosely). IDisposable is usually meant to help idiots like me avoid mistakes. And in the case of changing the user identity on the running thread, forgetting to revert to the initial identity could be catastrophic. When combined with C#'s using statement, I get an extra measure of safety from forgetfulness or from rogue exceptions that might be thrown inside the impersonating code.

For the uninitiated, C#'s using statement makes IDisposable really hum by encapsulating the statement's code in a try/finally block where the IDisposable.Dispose method is called automatically and reliably. With my Impersonator class, you can now write code like this:

Pretty simple, huh? The code compiled by the C# compiler would actually look something like this:

Now, if IDisposable were a real design pattern, there might be better compiler support. Wouldn't it be great that for classes implementing IDisposable, they could NOT be instantiated outside of a using statement context? That might be a bit restrictive in some cases but think about how fool-proof that would be. If you were required to instantiate IDisposable types within a using statement, you could guarantee that a type, which could be doing radical things like changing the identity on the running thread, would always clean up after itself. Alas, the C# team at Microsoft has been ignoring my plea for help.

Here's the code for the Impersonator class for your enjoyment. Let me know if you like it. I have many other gems like this one lying around.

UPDATE: I've included a videocast with this blog post. Let me know what you think. 

A few days ago, my buddy Justin Etheredge wrote a blog post about Efficient Paging in SQL Server. I was thinking about how transparent Language Integrated Query (LINQ) makes paging and I thought I'd blog about it. Two of the more interesting extension methods offered by LINQ are Skip() and Take(). You can use these extension methods to skip rows at the beginning of the query result and take only those you want to return. Sounds like paging to me. I wonder if Skip() and Take() used in combination with LINQ to SQL behave as efficiently as Justin's example? Let's take a look. Consider the following LINQ query:

This small example uses the AdventureWorks SalesOrderHeaders as the input sequence and shapes the output sequence to include the associated Contact's name parts and the total value of each order. The total value is computed as the OrderQty times the UnitPrice for each associated item in the SalesOrderDetails table. There is a filter placed on the query to restrict the results to orders placed in the 'Northeast' territory. This simple query shows how easy it is to filter, perform arithmetic and use the relationships in a LINQ to SQL data context to traverse table relationships. What does this query look like when it's compiled for execution on SQL Server?

You can see the territory filter applied as a WHERE clause. Note that even when a string literal is used in the C# code, LINQ to SQL still passes filtering variables as parameters. In this case, the territory name 'Northeast' is passed as a variable named @p0. This is always a good practice because it helps to thwart the injection of potentially malicious T-SQL into your query. We can see another interesting feature of LINQ to SQL in the T-SQL that is created called projection. Because the C# code shown above shapes the output sequence to only a few required columns, the LINQ to SQL engine is smart enough to T-SQL shape the query to return only what's needed. Projection often improves query performance and always improves transportation speed on the wire.

Finally, notice that the third column projected into the output sequence, i.e. the sum of each order's value, is instatiated as a two-part, nested sub-SELECT operation in the T-SQL statement. The inner SELECT does the math on the order quantity and price. The containing SELECT aggregates the line item totals and them filters them to the rows selected by the outer query. Nicely done, LINQ! Now, we see that this is a long list, returning thousands of rows. If this query is meant for human consumption, we should break it into smaller chunks to make it easier to handle. How do we do that in LINQ? Add this to the C# code shown before.

This modification uses the Skip() and Take() extension methods to skip 40 rows and take the next 20 rows. In other words, at 20 results per page, this query now returns the 3^rd page. Though the magic of deferred execution, we can add the Skip() and Take() extentions at any time before we begin iterating over the result set. This comes in handy when you want to enable paging for human consumption but to disable it for B2B or ETL scenarios. Is the paged T-SQL query shown here efficient though? You tell me. Here the T-SQL that is produced:

If you read the query from the inside out, you'll see that on the inside it's essentially the same query that we saw before we added the paging feature. It has all the original SELECTs named t1 through t4 and is wrapped as a new result called t5. The SQL Server ROW_NUMBER() function is used to inject a row number into t5 ordered by all 3 projected columns. That looks a lot like the query Justin showed us in his blog post. Very efficient! The new result containing the row numbers is named t6.

Finally, the t6 result is filtered by a starting row number and ending row number using two new variables @p1 and @p2. For page 3 paged in 20 row chunks as shown above, these variables would have the values 40 and 59, respectively. LINQ to SQL injects these starting and ending row number parameters whenever you use Skip() and Take() together. Well, it almost always does that. If you happen to specify Skip(0), it reverts to the behavior that Take() uses without Skip() which is to use SQL Server's TOP() function instead. LINQ to SQL sure knows how to sweet talk SQL Server, don't you think?

You know your kids are geeks when...

You tell them to use the high-temperature configuration on the dishwasher and they collectively refer to that as setting number 41.

I received Microsoft's MVP Award yesterday. I tried blogging about the experience at the end of the day but I didn't feel that I was ready to do it. The MVP Award is an unusual and very different kind of designation in the professional world. Many professional awards are about recognizing accomplishment. But the Microsoft MVP Award is about recognizing service. When you are singled out in this way, it's the community's method of saying, "Keep up the good work."

I've achieved many certifications in my career. I've held certifications from Sun, Oracle, IBM, Microsoft, HP, Siemens, Cisco and a few others. But I've never advertised them publicly because they were all my doing. I studied. I worked hard. I passed the certification exams. And I was a better developer and software architect for doing it. But, as a professional, I think those things are expected of me. So there was really no sense in wearing those certifications on my sleeve, in my opinion. To paraphrase the old adage, the proof is IN the pudding, not ON it, if you know what I mean.

The MVP Award, based on community and industry service, is not something you apply for. You get nominated by other MVPs and you go through a process of demonstrating what you've been up to for the past year. A review board checks you out (you can't hide from the search engines) and determines which MVP area of expertise you would fit into. Then, they determine among the other candidates in your group if your contributions to the community are exemplary. My MVP area of expertise is Visual C#, by the way. I think they picked the correct area for me because I know, love and promote the dickens out of C#. It's my favorite programming language in my career of 25 years so far.

It's a bit overwhelming to get the kind of support you need to win the MVP Award. When you receive "the magical e-mail" as some have called it, there's a stark moment of realization that you're standing on the shoulders of so many others who contributed to it. No MVP has ever achieved the title on his own. And I'm certainly no different. That's the way communities work. We lift each other up. I want to recognize a few folks who have lifted me up and encouraged me along the way to serve better within the Microsoft developer community:

• Kevin Israel
• Robin Edwards
• Darrell Norton
• Frank La Vigne
• Andrew Duthie
• Justin Etheredge

These folks are my role models. In particular, I want to recognize Andy Leonard. If we weren't about the same age, I'd call Andy my techno-Dad. He's a father figure to all of us in the mid-Atlantic user group community, I believe. Andy is humble and brilliant, a rare combination. He's also the hardest working person I know. If I could follow Andy's lead to 50% efficiency, I know I would be successful, too. We are sorry to be losing him to Atlanta soon but he's leaving us in great shape. Thank you, Andy.

I send out a heartfelt thanks to these folks and the cast of hundreds who pour their time and energy into making the user groups, Code Camps and DevCamps throughout the mid-Atlantic region great successes. My profession would be just a job if it weren't for all of you. I look forward to serving you all in the year to come.

Frank La Vigne called me out on the Software Development Meme. Michael Eaton started this one. The path to me is:

Michael Eaton >> Dan Rigsby >> Chad Campbell >> Pete Brown >> Frank La Vigne >> Kevin Hazzard 

Here's my response:

How old were you when you first started programming?

I got a hand-me-down TRS-80 Model II in 1981 from a friend who had recently upgraded to a new machine. I was 16 years old then. It was like crack for me. I was instantly addicted to software development or at least the idea of it.

How did you get started in programming?

In the early days, commercial opportunities outside of the mainframe computing space were rare. At age 16, I didn't have access to a mainframe yet. Small businesses likely to use things I could invent used paper to track their accounting and they weren't likely to switch to using a computing system that didn't fully replace their accounts payable, accounts receivable, general ledger and payroll systems.

Although spreadsheets like VisiCalc and Lotus 1-2-3 arrived on the scene early, they were seen initially only as helper applications for doing what-if analysis and such. And they weren't highly programmable anyway. So real programming jobs related to the first "killer application" were hard to create. As a result, those of us working on smaller, home-based computers had no real market for our ideas. So we built games and so-called bulletin boards until our age arrived.

It wasn't until I bought an AT&T PC 6300 Plus with an 80286 CPU in 1986 that I found my first real commercial opportunity writing a turn-key accounting and inventory tracking system for a chain of video stores.

What was your first language?

Zilog-80 Assembler. I was a math geek so assembly language spoke to my soul.

What was the first real program you wrote?

I had just turned 17. My Mom was working for a local hospital organizing paper-based records. The hospital had a new numbering system for patient records for which they had to apply brightly-colored sticky labels to manila folders. I wrote a program for her that determined how many of each numeric digit was required to label a range of numbers. I started with a brute force approach which was very slow and then created an algorithm with a fixed number of computations to yield the same result. The hospital bought the program from me for $50 to help them manage the disbursement of labels to all of the contractors working on the project. It was written in Color BASIC for the Tandy Radio Shack Color Computer. I ported it to GW-BASIC to run on the hospital's IBM PCs.

What languages have you used since you started programming?

Various assembly language dialects: Z80A, MC6809, Intel 80x86, DEC Alpha and Motorola PowerPC. I wrote assembler on a Perkin-Elmer mainframe but I don't remember what the CPU architecture was, to be honest. I remember it was pretty weird, more like a macro compiler than an assembler. I got my first C language compiler for the Tandy Radio Shack Color Computer although I preferred using the assembler on that machine.

I was introduced to C With Classes (which later became C++) in 1984. There was no C++ compiler for the Perkin-Elmer mainframe I was working on. So we used something called Cfront to translate the C++ text into C language which was then further compiled into a set of linkable objects. I worked in C and C++ almost exclusively from 1988 until 2002 with brief stints into Java-land.

In 2002, the bright light of C# shone upon me and I've been pretty happy since then. C# is more expressive and makes me more productive than any other language I've ever worked with. I love Python and I'm itching to use it commercially. However, finding real applications that I can't implement with C# is pretty tough. I'm not sure PowerShell is a real programming language but, if it is, it's definitely one of my favorites.

Every time I've been forced to work with JavaScript (directly or indirectly), I've felt like I needed to take a shower. If JavaScript or HTML ever becomes the right way to implement anything, I'm switching professions. Silverlight has me very excited, as you can imagine.

What was your first professional programming gig?

In my senior year as an undergraduate student, I was contracted to write an application for tracking the inventory and sales for a small chain of video stores. I built the whole thing using Symantec's Q&A product. It was a way-cool, weird product that was part word processor, part database, part reporting engine. You could do amazing things in a short time with Q&A that would take days or weeks to implement using other tools. I think it was the first RAD tool there ever was in the PC space.

If you knew then what you know now, would you have started programming?

Yes. I enjoy software development now more than ever.

If there is one thing you learned along the way that you would tell new developers, what would it be?

Teach part time. Whether you do it at work, in the user group community or as an adjunct faculty member at a local college (I do all 3), you'll find that you learn more, faster by preparing to teach than you could ever learn as a student. Teaching is a fantastic form of mental catharsis. Teaching also helps build your public speaking skills, which translates into greater responsibility and a higher salary as you move forward in your career. Developers who can build cogent arguments and present them to executives often go further and faster in their careers.

What’s the most fun you’ve ever had … programming?

When I worked at Intel Corporation in the Architecture Labs, the supercomputing group parallelized one of my fractal algorithms and made it available as a 3D "game". I had written the Pascal language implementation of the Julia Set solver in college while I was working on a paper concerning the Riemann-Stieltjes Integral. The Julia Set is a particularly beautiful fractal based on the concept of a Riemann map. Working with the supercomputing group to parallelize my algorithm and see it work in real-time 3D was a heart-poundingly cool experience. You could visually "drive through" the set to infinite levels of detail, zooming, panning and scanning in real time. What took my PC days to compute, the supercomputer was performing in milliseconds, allowing you to move through the Julia Set fluidly as if flying in a flight simulator. It was a breathtaking experience for me.

Who are you calling out?

• Justin Etheredge
• Alan Stevens
• Chris Love
• James Avery

I just read Andy Leonard's Five Bugs blog post concerning the Entity Famework Vote of No Confidence. You should read it, too.

That's the right approach Andy. As I said in my earlier blog post, this is not the Continental Congress facing off against King George. There are already plenty of alternatives to any technology that Microsoft makes available today. And yet, Microsoft's strategy of continual improvement works and they almost always gain market share.
 
Most good developers have the patience and foresight to use this strategy to their advantage. The weaknesses in a Microsoft technology early in the game create all sorts of opportunities for developers and architects to show off their prowess in closing those gaps. We build tools and techniques. We write about our experiences. We speak about issues in public, constructively. We build trust and support in our spheres of influence.
 
These actions are like a rising tide. Broad support for the right ways of serving our clients has the effect of lifting everyone's boat, including Microsoft's. Sharp attacks like the ones we've seen in recent days are more akin to using torpedoes instead. It just doesn't work. Look at Scott Bellware's tweets over the last day or two. There's a sample of them below. There's no professionalism in that vitriol. I agree with Andy. There's something else going on here. This can't be about making Microsoft better.
 
Here are some of Scott Bellware's tweets from yesterday and today with a few <snip>s to protect the innocent:

• @<snip> petty sabotage in this community is done because it's implicitly permissible by the towering gods of tech ed
• Lean Microsoft... for real... i would work myself to the bone for a software company that behaves like Toyota rather than Boris and Natasha
• imagine a microsoft that believed getting it right the first time was an imperative
• so much potential, so little will to transform it into betterment
• people, it's ok to say in public those things you say in email and in person. the more courages voices, the better things get for all
• we somehow got convinced that it's not ok to dissent to microsoft. when it's impolitic to hold a corporation to account, it has impunity.
• the tech ed crowd and rd crowd has the power to change .net mainstream culture. it appears too busy protecting its own interests to care
• dreaming of a .net community where integrity, courage, and diversity were unassailable values that everyone defends at all cost
• if only a high-profile blogger or podcaster in mainstream .net community had the integrity to talk about this problem. yeah. dreaming.
• THE hallmark of microsoft community is the drive to limit free speach
• mainstream .net's answer to the ef letter is to continue to spam the list. this is the clearest indicator of how shallow this segment is
• after years of dumbing down customer community, and taking input from that community, microsoft is itself a victim of its own craven ways
• the movie idiocracy is an allegory for the results of microsoft holding its customer community back from the advances it can't afford
• the long tail of fear, uncertainty, and doubt entrenched into ms culture by microsoft has limited its own community's intellectual potential
• alt.net: i think you could benefit from some OO design fundamentals / mainstream: nyah, nyah, nyah, i know you are but what am i
• even arguments that call people out by name are valuable when accurate and substantial, but name calling is what we get from ms community
• the mainstream .net community is always ready willing, willing, and able to be petty in its opposition to alt.net
• @<snip> other dev communities think the .net community is so primitive that they barely think it's worth paying attention to
• <snip> signed the ef letter. i think it's a hoax. can someone at ms confirm? if it's a fake, i'll delete it
• is there a magazine in the microsoft space that isn't on microsoft's payrol?
• the continued quoting of roger jennings' stilted characterization of the ef letter in microsoft media shows how corrupt microsoft media is

Read anything in there that makes you feel like a professional? Not me.

I use NHibernate and LINQ to SQL on a site that supports millions of end users. Murray Gordon, an Architect Evangelist at Microsoft, has written a nice synopsis of the Entity Framework Vote of No Confidence debate so far. More than anything else I've read on the subject, this article brings together facts and opinions that seem to ring true from my deep experience.

I'd like to go on record and say that the way this is being handle by the MVPs who signed the petition is unprofessional, in my opinion. Microsoft MVPs are not required to tow the line and agree with everything that Microsoft says and does. But there are ways to communicate with Microsoft that are constructive and there are ways that would make any corporation, including Microsoft, digs its heels into the dirt. This petition used the latter strategy unfortunately. In particular, Microsoft MVPs have channels that the rest of us don't have. They should use them and not the blogosphere to make plain there grievances.

In some sense, I feel as though the signatories of the petition feel like they are playing Continental Congress against King George. But there's no Boston Tea Party here. Microsoft didn't raise any undue taxes from any of us. They simply put out a framework that's clearly a v1.0 product. Microsoft doesn't win with v1.0 products. It wins with version 4 products because as a corporation, it knows how to get the first down (an American football term meaning the team gets to stay on the offensive).

I say let Microsoft run the ball for a few plays and let's see how they do.