CSS Header for code highlighting

Friday, January 18, 2013

Calling HttpClient async functions from constructors in ASP.NET 4.5

Recently, we have been using async/await a lot in our ASP.NET MVC project. It is very powerful. However, as with most power, comes great responsibility. In multithreading, this adage can be translated to "with great parallelism, don't deadlock".
As it has been documented and explained, in the current ASP.NET stack, when you await during a request, you will get a callback on the originating thread (that's the point of await/async after all). When you make an awaitable call to let's say HttpClient.GetStringAsync(), it will want to come back on that thread also. As well explained by Stephen Cleary in his StackOverflow answer (here), this goes south pretty much immediately if you start blocking somewhere in the call stack. This is true if you call a Task<T>.Result.

No async in constructors

Unfortunately, for good reasons, constructors cannot be async. Therefore, they cannot await on anything. In our case, we needed to retrieve some data from another service within a constructor to initialize some configuration. We are making extensive use of MEF, so we pretty much let the container resolve what it needs for us. This was an issue as there is no way to make a synchronous call to HttpClient to get data.


Fortunately, instead of forcing the issue and make the all to HttpClient.GetStringAsync() blocking by using Task<T>.Result, we went the opposite direction and made the call twice asynchronous:
var task = Task.Factory.StartNew(()=>client.GetStringAsync("url").Result);
var data = task.Result;
If I get this right, the reason this works is because the first line will start a new task on a separate thread. It is then on that second thread that the GetStringAsync will callback, leaving the main thread available for the outgoing call. If you think I am way off and have a good explanation, please share. I am not an expert on the subject at this time.
In the mean time, I hope this will get a few people out of a tough spot.

Sunday, January 6, 2013

HttpClient, Basic authentication and Bing

On Friday, I was trying to use the Bing API to test an issue we were having while using .NET 4.5, LINQ and async/await patterns. While Bing recommends you use their .NET library, I needed to keep my example as a pure core libraries example so people could try to reproduce my issue without having to download any third party library.
The Bing API offers Basic authentication or OAuth (to keep track of how much your are using of your monthly quota). So, I had to make a remote call to a web service using Basic authentication (to keep it simple). In .NET 4.5, the recommended way tot access remote resources is to use HttpClient. I spent a fair bit of time researching this online and either it is so simple or nobody ever tried, but bottom line was that nobody has a clear and simple example
There are several key elements to a successful Bing query using HttpClient.

Header info

According to the RFC 2617, basic authentication information is passed as an HTTP header. Fortunately, the HttpClient instance does expose its header information and does provide access to common headers, including Authorization. The proper syntax is:
using (var client = new HttpClient(new HttpClientHandler(), true))
   client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", "<your authenciation value>");

Authentication value

Following the instructions from the Bing API FAQ, I grabbed my key and slapped it on as my authentication parameter. No luck, all I got was this message:
The authorization type you provided is not supported.  Only Basic and OAuth are supported
A bit more digging revealed that I had to encode the data properly. Not a problem, .NET provides an easy way to do this:
using (var client = new HttpClient(new HttpClientHandler(), true))
    var encodedKey = Convert.ToBase64String(Encoding.ASCII.GetBytes("my-Bing-API-Key"));
    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", encodedKey);
Same error message. Using Fiddler, I confirmed that my data was being sent properly.

The Solution

Finally, after noticing a post on a related issue, I realized that the format of the encoded string is username:password. Since the Bing API migration document states to only specify the password when using a browser, all I had to do was this:
using (var client = new HttpClient(new HttpClientHandler(), true))
    var bingKey = "my-bing-api-key";
    var authentication = string.Format("{0}:{1}", string.Empty, bingKey);
    var encodedKey = Convert.ToBase64String(Encoding.ASCII.GetBytes(authentication));
    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", encodedKey);
    var data = await client.GetStringAsync("https://api.datamarket.azure.com/Bing/Search/v1/Web?Query=%27otixo%27");
I hope this will help the next person trying to get something apparently so trivial.

Monday, December 5, 2011

T4 templates: the lazy programmer’s friend

Text Template Transformation Toolkit (T4 templates) are a very powerful asset when used appropriately. Their main advantage is their ability to generate large amount of repetitive code over and over, at the click of a button and with a Swiss watchmaker’s precision.

My current project, Responder, an Outage Management System, contains dozens of xml files to define its data model and for configuration. Until a few months ago, adding a new column to a data table was a several hours process. It involved a lot of manual editing of XML files, updating SQL scripts, updating of code classes used to manipulate data, etc.. Errors were frequent and the simplest typo could prove dramatic if not caught on time by manual testing.

This all started changing with a T4 template. By generating all the necessary code straight out of the XML file, we removed most of the hand-coding, making the code (our RxDb.* and ArchivesDb.* structures for those who know) a perfect translation of the content of the original XML file. In the process, we have found typos and missing code that slipped through the QA cracks.

This change to code generation was started a couple of years ago. Now, other team members have embraced the T4 power to generate business objects to manipulate data in our complex DataSets. This has allowed us to update our data schema with minimal friction and add a lot of convenience methods to all our business objects with just a couple of line of actual code change

Of course, T4 templates have their own complexity and can be hard to navigate as they are a mingling of template and code. This becomes even more blurry (literally) when you are generating a C# file using C# as the template language. But, with the right tools, you can make more sense out of this funky looking piece of code/template.

Here is the MSDN main documentation for T4 templates, fully supported in VS2010. You can find the supporting source code for this post here. I have to apologize for dummying down the attached code, but I wanted to ensure that it did not contain any proprietary information. I took the safe route and took out any business-related terms. I just kept the basic structure.
Don’t forget to download the free tangible T4 Editor for VS2010. There are plenty very valuable resources online which will guide you through your first steps.

But here are some pointers that can help you keep things clean.

The first pointer I will give you is regarding accessing local but relative resources (such as XML files). Feel free to visit StackOverflow and read through my question and the answer here. Basically, if you want your template generating code to be aware of its surroundings, you must set the hostspecific flag to true AND use the Host class to access host-specific information. This makes your template portable.
Here is the tag declaration:

<#@ template language="C#" hostspecific="true" #>

And here is the use of the Host class:

XmlFile = Host.ResolvePath(@"..\Data\Schema.xml")

The following pointer was not in the original post but came after several months of using more and more T4 templates. We started realizing that, as we add more code to the generated code, our templates were getting pretty hard to read. We were able to simplify some of our templates by using some of the following techniques:

  • Abstract base class: whether you are generating a single class or many classes, putting some of the common code into an abstract base class will keep your template focused on the code that needs to be generated
  • Extension methods: introduced with C# 3.0, those allow you to add public methods to existing classes without getting access to the extended class. There are some restrictions with this approach, but it will still allow you to separate code that must be generated from code that can stay static from one code-gen to another
  • Partial classes: the best approach in my opinion. This allows you to put all your static code in one or more code file and leave only the generated code in your T4 template. And the static code can include private members, helping you keep your objects nicely encapsulated

Of course, the best approach is a good mix of all 3 of them. Try to keep each code file, be it template, extensions, partial or base class, as easy to read as possible. The attached code sample contains both an abstract base class and partial classes.

Thursday, December 1, 2011

Let’s get blogging!

Hello dear visitor. This will officially be my first blog post. I know, not that exciting, but I have been posting a bit on our internal blogs for the past few months, sort of getting my feet wet. But I decided it was time to take my posting to a larger audience. Not that the world has been missing what I have been saying, but because I want to get more feedback from the software engineering community at large.

For years, I have been following and admiring Jeff Atwood from his Coding Horror days to Stack Overflow. On multiple occasions, he reminded us that sometimes, you just need to get out there, and get started. You won’t be an internet superstar overnight, but overnight success does not happen overnight. That goes for writing software, to writing blogs and many other aspects of our lives. And recently, my coworker Shawn Kendrot decided to take that step also and start blogging to the world on Visually Located. I actually borrowed his disclaimer.

To get started, I am going take a very, very small page out of Martin Fowler’s book and retread my internal post to the public. They will be edited to match the audience and to avoid releasing any proprietary information. I look forward to your feedback and hope you will come back and visit often.