got net?

Kevin Hazzard's Brain Spigot

About the author

Welcome to Kevin Hazzard's blog.
E-mail me Send mail

Recent posts

Recent comments

Authors

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2010

Dynamic Language Runtime Performance Demos

I spoke at the Charlottesville .NET User Group this week and at the Raleigh Code Camp. I cheated and did the same presentation to both groups. Call me lazy but, in the middle of planning our own Code Camp in Richmond, I really didn't want to prepare two separate talks. I did a talk back at CodeStock 2009 on a similar topic back in June 2009 but it's evolved a lot since then based on my own growth and understanding. You can find the code and slides below.

This talk takes about an hour and covers the basic concepts of metaprogramming and how the DynamicMetaObject class, the IDynamicMetaObjectProvider interface and the DynamicObject helper class work in the .NET Framework 4.0. The talk is very demo-centric, focusing on the code that support the concepts outlined in a few slides. The first demo focuses on helping you to understand the dispatch mechanism in the DLR and how you can implement your own classes that participate in the call binding process. It's a fairly simple fluent XML parser implemented using the DynamicObject helper class. While it's simplistic, it helps to introduce developers who are new to the DLR to the concepts of dispatch and binding in C# before diving into things Pythonic.

The second demo focuses on performance measurement. Having learned about the delegate and rule caching that goes on in the DLR's CallSite<T> and ActionBinder classes in the presentation, these tests show the effect of all that awesome work that Microsoft has done for us to enhance the performance of dynamic code. The result is that we see the caching yields about 5,000 times the performance of traditional static to dynamic language interoperability. Rather than the hundreds of transactions per second that we are accustomed to when doing dynamic language invocation from a context like C#, we now get millions of transactions per second. To put it all in context, the same transactions performed in pure C# are still about 4 times as fast as the DLR with delegate and rule caching. However, you have to admit that what the DLR provides today is pretty amazing. For many, this level of performance makes the use of languages like Python for scripting a large application feasible for the first time.

The question I most often get about this is, "When will the dynamic language performance be just as good as 'real' .NET code?" OK, so I take exception to the term 'real' there but I know what you all mean. I don't know the answer and it doesn't really matter that Ruby and Python performance be just as good as C#. Different languages have their own strengths and sometimes that includes performance. But when pure C# generates 6 million transactions per second, traditional thunking across to dynamic languages performs at a rate of several hundred per second, the fact that adaptive Inline Caching in the DLR helps me to yield close to 2 million transactions per second is pretty darned good. We're well on the way to my other favorite language, Python, becoming a first-class citizen in the .NET family. It takes education, of course, for people to know just how wonderful the DLR is which is the real purpose of my evangelism. I just love the DLR, if you can't tell already.

I am doing a Channel 9 geekSpeak next week on this topic and I'll refer to some of this material. And my talk at the October 2009 NoVA Code Camp will use an even further evolved version of this talk. Drop me a note and let me know what you think about this topic. Do you have success stories? Horror stories regarding dynamic language integration?

Slides for DLR Performance Talk September 2009 [PPTX] (184 kB)

Slides for DLR Performance Talk September 2009 [PDF] (680 kB)

Code for DLR Performance talk September 2009 [Updated 21 Sept 2009] (16 kB)


Posted by kevin on Saturday, September 19, 2009 4:20 PM
Permalink | Comments (2) | Post RSSRSS comment feed

How I Learned to Love Metaprogramming

UPDATED on 30 June 2009

I spoke at the CodeStock 2009 conference and I thought it would be helpful for the attendees and others to be able to download my code and slides. The title of my presentation was "How I Learned to Love Metaprogramming" and it concerns Dynamic Language Runtime architecture, performance of dynamic typing and Python to C# integration. The slides and source code are linked below. I will be giving this talk again in September at the Charlottesville .NET User Group meeting. Both of the demos require C# 4.0 which is available in Visual Studio 2010.

  • Demo One - shows how to do XML parsing using a fluent interface based on a DynamicObject derivation in C# 4.0
  • Demo Two - shows how the Level 0, 1 and 2 CallSite and ActionBinder caches perform. UPDATED: I added a demo on 30 June 2009 that shows how the DLR 0.9 compares by invoking dynamic code through the DLR hosting APIs, thereby bypassing the CallSite caching mechanisms. The results are very instructive, showing that the DLR's polymorphic inline caching can yield a 250000% increase in performance. You read that correctly: a two hundrend fifty thousand percent increase in performance.

Slides in PDF (Acrobat) format (688.49 kb) 
Slides in PPTX (PowerPoint 2007) format (639.23 kb) 
Demo One Source Code - MetaObjectPlay200905.zip (5.17 kb) 
Demo Two Source Code - PythonIntegration200906.zip (5.33 kb)


Posted by kevin on Saturday, June 27, 2009 8:40 AM
Permalink | Comments (2) | Post RSSRSS comment feed

The DLR is the Language of Languages

The more I study the Dynamic Language Runtime (DLR), the more I am convinced that it deserves to be the centerpiece of Microsoft's .NET 4.0 strategy. I am a statically typed languages guy. Or at least I have been. I love the robustness that the compiler applies to certain preconditions for using objects. Type safety is great but I don't think that statically typed languages go far enough to be honest. I am excited about Spec# because of its supports for invariants, pre and post conditions, static program verification, etc. Will Spec# ever go mainstream? It doesn't appear to have the momentum that F# or IronPython have, to be frank about it. But maybe we'll see some ideas creep into C# from the Spec# research that Microsoft is doing. I have my fingers crossed.

On the other end of the spectrum is IronPython 2.0, Microsoft's first DLR-based implementation of the Python programming language. I just love Python. It's so clean and simple that almost anyone can learn and use it effectively after studying it for an hour or two. If I could send a note to myself in the past to change one thing about my career (besides the winning lottery numbers for the past decade) it would be to encourage my younger self to learn Python back in the mid-1990s. I knew about the language back then but I was a C++ bigot. I wish I had been more open minded. So much of my approach to software development would have been better if I had learned Python back then. I was working in the Intel Architecture Labs and had the perfect opportunity to do so, too. We were helping a very young company named Yahoo.com to evaluate e-mail providers to purchase. We ended up recommending RocketMail technology which was based almost entirely on server-side Python scripting. I had the perfect opportunity to learn Python from real experts right there in front of me and I let it slip right by me.

Well, no sense living in the past, right? I know Python really well now and it has changed the way I think about software development. Learning Python made me fall in love with Ruby and LISP and F#, too. I'm a real polyglot now and being able to understand how these other languages solve problems helps me design better in the language I use every day: C#. By the way, the term polyglot from its Greek roots literally means many tongues. I'm excited about the DLR because it's enabling Python and Ruby on my favorite platform. But the DLR is so much more important than that. It's the Language of Languages. The DLR defines the primitives that must exist on the boundaries of our languages to make them interoperate. In the same way that COM unified the dispatch model in the 1990s through IUnknown and IDispatch, the DLR will define the call semantics and dispatch models for the .NET Framework for the 2010s through IDynamicObject and DynamicMetaObject.

Pretty soon, we'll see a flurry of languages appear on top of the DLR. A bunch of them are already in development. But we'll also see a wave of DLR object binders appear that have no significant language support on the platform, too. The effects will be profound. Imagine a Java RMI bridge written as a DLR object binder. Using the dynamic type in C# 4.0, you could make calls to remote Java objects from .NET as if they were just part of the framework. Or imagine a type binder that exposes RESTful services as a dynamic object model. There's really no end to the possibilities here. The key to all of these scenarios is the enablement of dynamic services and that sounds a lot like what Web 2.0 was supposed to be.


Tags:
Categories: Architecture | C# | DLR | F# | Python | Ruby | Software Dev
Posted by kevin on Wednesday, January 07, 2009 9:28 AM
Permalink | Comments (12) | Post RSSRSS comment feed

Methods Should Never Return Void

Returning void (or Nothing in VB.NET) from a method is best the way to say, "I don't care about you, caller." Consider this silly C# code:

var engine = Python.CreateEngine();
var scope = engine.CreateScope();
scope.SetVariable( "PIRoot", Math.Sqrt( Math.PI ) );
var result = engine.Execute<double>( "PIRoot ** 2", scope );
Console.WriteLine( result );

This code could be a lot more fluid but the ScriptScope.SetVariable method returns void. If the method returned a reference to its Engine property, the code could have been written more succinctly as:

Console.WriteLine( Python
    .CreateEngine()
    .CreateScope()
    .SetVariable( "PIRoot", Math.Sqrt( Math.PI ) )
    .Execute<double>( "PIRoot ** 2", scope ) );

Wouldn't that be nice? So don't return void (or Nothing in VB.NET). When you have nothing else you can return from an instance method, return the this pointer (or Me in VB.NET).

 


Categories: C# | DLR | Rant | Software Dev
Posted by kevin on Tuesday, January 06, 2009 9:53 PM
Permalink | Comments (4) | Post RSSRSS comment feed

Mixing Static & Dynamic .NET Languages - November 2008 Update

This is the November 2008 update of my now famous "Mixing Static & Dynamic .NET Languages" presentation. OK, so it's not a famous talk. But it still rocks and the folks at the code camps love it. This is the material I will be using at my talk at the upcoming Raleigh Code Camp on 11/15/2008. The source code and slides are attached below. When you run the demo code, you'll see a dialog that looks like this: 

The application, written primarily in C#, implements a shopping cart to which various products can be added. The shopping cart isn't the cool part, of course. What's interesting is the use of Python code to implement the discounting/marketing rules within the cart. If you press the Show Rules Editor button at the top of the form, you'll see the rule editor dialog: 



Python code is pretty easy to read isn't it? You can probably follow the logic to see that if certain counts of Entertainment products are added, they get discounted at various levels. And a trigger is set on the total count of items in the cart, too. There's a bug in the code though. But that's part of the demo. Can you find the bug?

Hopefully, if you run this code and understand it, you mind will be expanded a bit. One of the most often asked questions I get when talking about IronPython is, "How do I convince my boss to let me write that application in Python?" My answer is, "You don't have to write the whole thing in Python. With the fantastically rich hosting APIs in the .NET Dynamic Language Runtime (DLR), you can write the portions that need dynamism in Python and write the rest in C# or VB.NET." In fact, for most applications, the portion that can benefit from dynamic behavior is usually fairly small as a percentage of the application's total surface area. So this model of mixing static and dynamic languges is a nice approach to those kinds of problems.

Slides for MixingStaticAndDynamicDotNETLanguages 20081115.pptx (251KB)

Source Code for MixingStaticAndDynamicLanguagesInDotNet 20081109 (43KB)


Posted by kevin on Friday, November 07, 2008 5:17 PM
Permalink | Comments (3) | Post RSSRSS comment feed

Mixing Static and Dynamic .NET Languages for Philly Code Camp 2008.3

I presented at the Philly Code Camp on 11 October 2008 for a group of about 20 developers. Thanks to all who came out to listen to my presentation. And thanks especially to Don Demsak (donxml) for attending and really helping me to shape the talk. Don added a lot of anecdotal information that I would not have included on my own. It was a very fluid discussion with lots of give and take. When I give this talk again at the Raleigh Code Camp in November 2008, the folks who attend will benefit from what happened in Philly.

The gist of this presentation is that it's possible to mix the Dynamic Language Runtime (DLR) into statically-typed, early-bound languages like C# to make them much more flexible. In this talk, I demonstrated how a ShoppingCart being filled with Products can adjust discount rates based on marketing rules written in an external Domain Specific Language (DSL). In this case, my DSL was really just Python. I chose to use Python because the syntax is so simple and clean. It's so light, it doesn't get in the way. It's not a real DSL, of course, but by injecting .NET objects into a ScriptScope on a ScriptRuntime (all DLR hosting terms), the Python syntax acting on those injected types looks an awful lot like a language for managing product discounts.

The few slides I had and the source code are linked below. For this code, I used IronPython 2.0 Beta 5. You will need to download and install IronPython to compile the code.

MixingStaticAndDynamicDotNETLanguages20081011.pptx (91.66 kb)

MixingStaticAndDynamicDotNetLanguages20081011.zip (16.65 kb)


Posted by kevin on Saturday, October 11, 2008 3:58 PM
Permalink | Comments (0) | Post RSSRSS comment feed

July 2008 Richmond Meet and Code Notes

The Richmond Meet and Code Dinner in Richmond tonight was awesome. We had 30+ people turn out. Our key presenter had to cancel at the very last moment but Justin Etheredge stepped up to the plate and pitted his self-proclaimed meager Ruby skills against the barrage of questions from the crowd. Justin did a great job, nascent Ruby skills or not. Harper Trow also presented on the history and current state of Ruby and man, I've so been looking forward to that! Harper was just great. I sure hope he steps out and presents more often. Harper's whole life experience oozes with "I love .NET and I love everything else, too." We need more of that kind of healthy alternative-yet-embracing thinking in the .NET community I believe.

The Meet and Code Dinner format is excellent, in my opinion. I think what Justin is doing is commendable. The goal of his group is to build up the community, not potential sponsors. He's going to be setting up a website and taking donations via PayPal. I am definitely going to support him financially in his effort.

In between two of Justin's sessions, I presented my ProxyGen tool again. This is the same tool that I presented at the last Richmond .NET Code Camp and at the last NOVA .NET Code Camp. Except this time, I focused not on the task of dynamic proxy generation against WSDL contracts but on the use of the ScriptRuntime and ScriptScope classes in Microsoft.Scripting to host a Python or Ruby scripting engine within a C# application. I think I got the brains of the attendees pumping with ideas which is all I was after. I described an application that could be statically typed and early bound, written in C# with a dynamic lower edge that could be scripted from a remote source. People in the crowd started coming up with all sorts of great ideas to implement business logic in dynamic code and inject it. Awesome thinking!

I've attached the latest build of the ProxyGen code below. Here are a couple of screenshots that show how it works. This first screen shot shows running IPY.EXE to execute a Python script to call a SOAP-based web service with no precompiled proxy. The only magic here is in some dynamic code generation that my ProxyForWsdl class does by downloading the WSDL contract and building classes for services, operations and data contracts. As you can see, I am calling an integer factoring service dynamically. No new Python knowledge yet. But read on.

The next screen shot is of the test harness in the sample code showing how a Python script similar to the one shown in IPY.EXE above can be injected into a C# application.

The C# code to embed the Python engine is simpler than you would think. I wrote a little wrapper class to make it easier to digest:

using System.Collections.Generic;
using Microsoft.Scripting;
using Microsoft.Scripting.Hosting;

namespace TestHarness
{
    public class DynEngine
    {
        private readonly ScriptEngine _engine;
        private readonly ScriptScope _scope;

        public DynEngine( string engine )
        {
            // get an engine from the script runtime of
            // the desired type
            _engine = ScriptRuntime.Create().GetEngine(engine);

            // creating a scope gives us a dynamic space
            // to run code in
            _scope = _engine.CreateScope();
        }

        public T ExecuteStatements<T>( string codeText,
            string resultVarName,
            IEnumerable<KeyValuePair<string, object>> scriptVars)
        {
            // inject some variables into the scope
            foreach (var kvp in scriptVars)
                _scope.SetVariable(kvp.Key, kvp.Value);

            // "compile" the code and execute it
            var source = _engine.CreateScriptSourceFromString(
                codeText, SourceCodeKind.Statements);
            source.Execute(_scope);

            // pull a typed variable from the scope as the result
            return _scope.GetVariable<T>(resultVarName);
        }

    }
}

Invoking the Python code is as easy as loading up variables into a dictionary, instantiating my wrapper class with the type "py" for Python and calling ExecuteStatements with the Python source code text:

var vars = new Dictionary<string, object>
           {
              { "url", tbUrl.Text },
              { "ep", ep }
           };

var engine = new DynEngine( "py" );
result = (List<int>)engine.ExecuteStatements<object>(
   tbPythonCode.Text, tbEvaluationExpression.Text, vars );

The HTTP path to the web service WSDL contract is passed as a parameter by injecting it as a script variable named url in the Python script. And the ServiceEndpoint is also injected as a variable named ep so that it's Name property can be selected in the Python script. This is a good example of marshalling a .NET object into the script scope where it can be fully discovered and used by a dynamic language. In the DynEngine class shown above, you can see the SetVariable and GetVariable methods on the ScriptScope being used to inject and extract variables as a type of Inter-Process Communication (IPC) mechanism. This isn't the only way to communicate between a host and a dynamic script but it's simple for illustration purposes.

That's pretty much it. Hosting a dynamic Python in C# is not hard at all. In fact, if I referenced the IronRuby assemblies on the TestHarness project, I could switch the "py" constructor parameter to the DynEngine shown above to "rb" and inject Ruby script just as easily. That's literally all we would have to do to move from Python to Ruby in this case. As we used to say in America in the 1980s, that's tasty!

The ProxyGen code's attached below. Be sure to download the IronPython distribution from CodePlex and reference four (4) assemblies in the TestHarness project. I am using IronPython 2.0 Beta 3, by the way. These are the four (4) IronPython assemblies you'll need to reference:

  • IronPython.dll
  • IronPython.Modules.dll
  • Microsoft.Scripting.dll
  • Microsoft.Scripting.Core.dll

All of them can be found in the root of the IronPython BIN distribution ZIP. Here's the ProxyGen code I demoed at the meeting.

ProxyGen20080731.zip (21.40 kb)


Posted by kevin on Thursday, July 31, 2008 10:18 PM
Permalink | Comments (3) | Post RSSRSS comment feed

SuperSOAP Slides and Code from NOVA Code Camp 2008.1

Accessing web services with SOAP can be just as easy as using REST with all the enterprise-class features you've come to expect from WSDL and SOAP. Who says that the cycle of metadata and proxy generation should be so hard? I gave a talk at the NOVA Code Camp 2008.1 that shows how by using the CodeDOM, and the ServiceModel.MetadataImporter, you can generate proxy code dynamically.

In this talk, I also showed how IronPython can be used to add a dynamic "lower edge" to a C# application to make it much more dynamic feeling. Finally, we finished with a discussion about features that may be added to Visual Studio 10 and the C# 4.0 language sprecification to make SOA achievable for many more developers. It was a lively discussion with lots of great questions. Here are the slides and the demo code:

ProxyGen20080517.zip (20.86 kb) - Sample code that demonstrates the use of IronPython and some custom CodeDOM code to avoid generating proxies for WCF integration via SOAP/WSDL. IronPython 2.0 Beta 1 or newer is required to compile this code.

Simple SOA with SuperSOAP by Kevin Hazzard.pptx (208.46 kb) - my PowerPoint slides from this discussion.


Posted by kevin on Saturday, May 17, 2008 4:28 PM
Permalink | Comments (0) | Post RSSRSS comment feed

Silverlight Richmond Code Camp Presentation

Here are my slides from the Richmond Code Camp 2008.1 talk concerning Silverlight.

Silverlight Version 2 - Hazzard - RCC2008.1 - April 2008.pptx (237.23 kb)

And here are some demo links that I used during the presentation: 

Deep Zoom
http://joestegman.members.winisp.net/DeepZoom/
http://www.neogeo.com/samples/SilverZoom/

Financial Management
http://www.cookingwithxaml.com/meals/financials/default.html

WeatherBug
http://sl.weatherbug.com/?zip=23060

Learning and Discovery
http://www.loc.gov/exhibits/earlyamericas/readingartifacts/

Image Snipper
http://silverlight.net/Samples/2b1/ImageSnipper/testpage.html

Form Controls
http://silverlight.net/Samples/2b1/SilverlightControls/run/default.html
http://labs.componentone.com/Sapphire/


Posted by kevin on Saturday, April 26, 2008 1:02 PM
Permalink | Comments (0) | Post RSSRSS comment feed

Demo links used in my Innsbrook .NET User Group Meeting on 2008/03/26

Deep Zoom
http://joestegman.members.winisp.net/DeepZoom/
http://www.neogeo.com/samples/SilverZoom/

Financial Management
http://www.cookingwithxaml.com/meals/financials/default.html

WeatherBug
http://sl.weatherbug.com/?zip=23060
 
Learning and Discovery
http://www.loc.gov/exhibits/earlyamericas/readingartifacts/

Image Snipper
http://silverlight.net/Samples/2b1/ImageSnipper/testpage.html

Form Controls
http://silverlight.net/Samples/2b1/SilverlightControls/run/default.html
http://labs.componentone.com/Sapphire/


Posted by kevin on Wednesday, March 26, 2008 5:15 PM
Permalink | Comments (0) | Post RSSRSS comment feed