Running Asp.Net MVC controller actions on STA threads

Recently while we were converting a legacy asp application (Yes they still exist) to asp.net mvc, we had to work with a set of 3rd party business critical components.

These components were legacy COM components.  In load testing we found that controller actions that contain calls to these components were crashing w3wp process almost every minute.   A little bit of research around this problem yielded the following article.

Running ASMX Web Services on STA Threads

The summary of the problem is, MVC action methods run in COM multithreaded apartment (MTA) threads.  These legacy components were being created from MTA threads are being serialized and are being processed by single STA thread.  On top of that these components are loading tons of data in to memory and the load is causing the memory corruption.

So the solution is to make the MVC action method to run on STA thread, thus allowing COM to place the object instances on the creator’s thread.

Here is an MSDN thread that ties up the STA with asp.net mvc

AspCompat=true does not work with MVC

The above solution doesn’t support asp.net sessions and also was written for asp.net mvc 1.0

Here is the asp.net mvc 3.0 adjusted solution.

First the RouteHandler, it derives from MVCRouteHandler and instantiate a IHttpHandler derived class.

public class STARouteHandler : MvcRouteHandler
{
   protected override IHttpHandler GetHttpHandler(RequestContext requestContext)
   {
       return new STARequestHandler(requestContext);
   }
}

Next the STARequestHandler,  the key to this entire magic to work is in the BeginProcessRequest and EndProcessRequest methods. These two methods create an aspnet compat wrapper around the actual execution of action method.  This handler also implements IRequiresSessionState marker interface to support the sessions.

public class STARequestHandler : Page, IHttpAsyncHandler, IRequiresSessionState
{
    public STARequestHandler(RequestContext requestContext)
    {
        if (requestContext == null)
            throw new ArgumentNullException("requestContext");
        this.RequestContext = requestContext;
    }

    private ControllerBuilder _controllerBuilder;

    internal ControllerBuilder ControllerBuilder
    {
        get { return this._controllerBuilder ?? (this._controllerBuilder = ControllerBuilder.Current);}
    }

    public RequestContext RequestContext { get; set; }

    protected override void OnInit(EventArgs e)
    {
        string requiredString = this.RequestContext.RouteData.GetRequiredString("controller");
        var controllerFactory = this.ControllerBuilder.GetControllerFactory();
        var controller = controllerFactory.CreateController(this.RequestContext, requiredString);
        if (controller == null)
            throw new InvalidOperationException("Could not find controller: " + requiredString);
        try
        {
            controller.Execute(this.RequestContext);
        }
        finally
        {
            controllerFactory.ReleaseController(controller);
        }
        this.Context.ApplicationInstance.CompleteRequest();
    }

    public override void ProcessRequest(HttpContext httpContext)
    {
        throw new NotSupportedException("This should not get called for an STA");
    }

    public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
    {
        return this.AspCompatBeginProcessRequest(context, cb, extraData);
    }

    public void EndProcessRequest(IAsyncResult result)
    {
        this.AspCompatEndProcessRequest(result);
    }

    void IHttpHandler.ProcessRequest(HttpContext httpContext)
    {
        this.ProcessRequest(httpContext);
    }
}

 And lastly the usage of this handler.

While creating a route for the action method, simply attach this handler to the route definition.

context.MapRoute("STARoute", "{controller/{action}",
                    new { controller = "Home", action = "Index")
                    .RouteHandler = new STARouteHandler();

You can test the apartment state by calling the following line of code in the action method.

Thread.CurrentThread.ApartmentState.ToString();

MongoDB mapreduce for stackoverflow.com recent tags

I was playing with MongoDB's mapreduce, and wanted to write a query that simulates the list of 'Recent Tags' feature on stackoveflow home page.

I am using mongo-csharp-driver for this experiment.

Here I am taking a guess at stackoverflow's domain model. Here is the Question model with enough properties to demonstrate the mapreduce query.
Every question is associated with a list of Tags and has a CreatedOn property.

public class Question{
public ObjectId Id { get; set; }
public DateTime CreatedOn { get; set; }
public ICollection Tags { get; set; }
}

Create a database with some name and add 'Questions' collection to the mongodb.

We are going to make the following call to find the recent tags.

var recentTags = questions.MapReduce(map, reduce).GetResultsAs<ResentTagResult>();

RecentTagResult holds the results of the mapreduce query and defined as

public class RecentTagResult {

public string Id;
[BsonElement("value")]
public RecentTag Value;

}

MongoDb's mapreduce call outputs a result set with two properties _id and value. So here I am defining a mapper class with a property 'Id' and property 'Value' of type RecentTag. BsonElement attribute in the above code simply maps lower case property 'value' from the result set to title case property 'Value'. _id property from result is automatically mapped to Id by 'GetResultsAs' call.

RecentTag is defined as follows. Here I am expecting that the query results are going to contain Tag, count of tags, last time when a question was created with this tag. As you can guess our map/reduce functions must emit the values that match the following class definition.

public class RecentTag {

public string Tag { get; set; }
public int Count { get; set; }
public DateTime LastSeenOn { get; set; }

}

Now coming to the meat of the problem. A question can have more than one tag. We are looking for a list of tags used by questions asked in the last month. Here is the definition of the map function (which is completely written in javascript) that goes over entire collection of questions and emits a result each time a tag is found if that question is asked in the last month.

private string map =
@"function() {
if(this.CreatedOn >= new Date('Oct 1, 2011') && this.CreatedOn {
var lastseen = this.CreatedOn;
this.Tags.forEach(function(tag) {emit(tag, {Tag: tag, LastSeenOn:lastseen, Count: 1});});
}}";

It is very important that the emit function's value should match our RecentTag type definition.

emit(tag, {Tag: tag, LastSeenOn:lastseen, Count: 1})

Now coming to reduce, we have a bunch of emitted results from map function and we simply count them to find the total count of each tag found in last month.
A tag might appear more than once in the last month. If the tag appears only once, reduce function will never be called.

private string reduce =
@"function (key, arr_values) {
var dates = [];
arr_values.forEach(function(val) {dates.push(val.lastseenon)});
var result = {Tag: key, LastSeenOn: new Date(Math.max.apply(Math, dates )), Count:0};
for(var i in arr_values)
{
temp = arr_values[i];
result.Count += temp.Count;
}
return result;
}}";

Here att_values contain all emits for a single tag. Again it is important that our return type must match with ResultTag definition similar to the map function.
We start with that definition first

var result = {Tag: '', LastSeenOn: new Date(Math.max.apply(Math, dates )), Count:0};

And then iterate through arr_values and simply increment the count to get the final result.

Filling LastSeenOn property is little tricky. Here we are trying to find out the Max of the CreatedOn property of all emitted values of a single tag.

var dates = [];
arr_values.forEach(function(val) {dates.push(val.lastseenon)});

Here we are gathering all the dates from lastseenon property in to an array. And then while defining the result we are applying the javascrpt's Math.Max function to find the last seen date.

LastSeenOn: new Date(Math.max.apply(Math, dates )

That is all to it.

Open Source Dependencies

Total Cost of Ownership is a critical metric which I would like to pay attention to in software development. A project with lots of open source dependencies can become very difficult to maintain.
With advent of modern package systems like gems, Nuget and CDNs it has never been this easy to use open source in software projects. As of this writing there are thousands of JQuery plugins, hundreds of ruby gems, and hundreds of Nuget packages. I have seen developers arguing about their favorite ORM tool. I have not seen enough arguments about which JQuery light box plug-in should be used for the user interface. If you try to take a stock of all dependencies (both commercial/open source) on your code, you might be surprised to see the list.

I surveyed few .Net web applications. A single web application can have following list of components, without counting the major dependencies like Object Relational Mapper

1. PDF
2. Charting
3. Ajax
4. Half a dozen JQuery plugins/other JavaScript alternatives
5. Spread sheet
6. Dashboard
7. Social networking
8. Payment gateway
9. Reporting
10. Other value added services like support, feedback, live help
11. External web services
12. Scheduling
13. Email
14. JSON
15. Mocking
16. Unit Testing

.. and the list goes on.

Now imagine using a open source component for each one of these dependencies. That is a lot of code to maintain!

All non-trivial abstractions, to some degree, are leaky

At one point or the other your team needs to know internals of every open source library used in your project. Some of these libraries have hard dependencies, with a potential of preventing the future upgrades.

Never under estimate the testing burden. Take the example of a JQuery plugins. Usually these plugins depend on JQuery. When you upgrade JQuery, you are forced to upgrade the dependent plugins. Depending on quality of the plugin, many times you end of spending hours in debugging why a web page is failing to find the correct version of plugin that works. Cross browser testing is also time killer. All this can become very complex when all you trying to do is upgrading JQuery to its latest version, which itself is a trivial task.

You might not see this kind of risk with popular open source libraries. As they operate similar to commercial offerings. But not all of the open source projects are popular.

Some tips to control the proliferation of these dependencies in to your projects

1. Always maintain a list of open source dependencies. Make it available to QA team and developers.
2. Create a test suite to test all these dependencies. And run the tests with every deployment.
3. Always check-in the source code for the open source project in to source control.
4. Try to keep these dependencies to minimum. Remember that your team needs to re-learn these dependencies on every upgrade or major functionality rewrite.
5. If possible try to stick to a known set of controls from a single vendor.
6. While choosing the libraries weigh in your team’s skill set, team’s composition and future direction. If you don’t have dedicated resources to focus on Ajax work of the website, it is better to stick to a commercial solution than using a laundry list of multiple open source offerings.

The Upgrade Dilemma

If you are working on a project that uses 3rd party products you should be facing this dilemma regarding upgrades, regularly. It could be a core system upgrade such as .Net framework version or a SQL Server database version or a significant 3rd party component upgrade. Usually these upgrades proposals come from tech teams.

Assuming that cost of the software that is being upgraded itself is not prohibitive, most of the arguments are centered on the work that needs to be done by the team to stabilize the product with the upgraded version

Arguments

1. Arguments in favor of the upgrade

a. If we don’t upgrade now, when would we upgrade? After we acquired another 1000 customers?
b. It would be an incentive for the developers. It will help with both hiring and retaining good developers, since they tend to want to work on the latest and greatest.
c. We could get rid of some of the nagging bugs in the product, as the new upgrade solves them
d. New upgrade contains significant and important changes and we should take advantage of them
e. If we are up to date with the latest version, our product won’t become obsolete

2. Arguments against the upgrade

a. There aren’t many useful features in the new version
b. Everything works now so why should we unnecessarily disturb the code base?
c. What is in it for customers, since they didn’t ask for the upgrade?
d. If we upgrade component A, component B will not work
e. We will need to test the whole product, which would be very expensive. We might miss some parts which can put the product at risk
f. The whole process could be exhaustive and we may burn out the team
Most of the arguments against the upgrade are centered on changing the code base, and the fact that changing code could be a risky affair. However, I usually favor staying as current as possible, because the speed with which new upgrades are being released to the market is extremely fast, and if we fail to stay current, the product will end up becoming stale and will almost certainly require a big rewrite in the future.

Process – What Not To Do

Upgrade by committee

It is very tempting to gather all parties involved and conduct formal meetings, get all of their buy ins, create and manage a project plan. This means lot of meetings, follow ups, emails and creates inevitable creation of heroes who will end up taking on the bulk of the work to meet some arbitrary deadline. It may sound logical, and works most of the time. But this is a very expensive process, and usually involves lots of meetings. These meetings are the breeding grounds for “Fear, Uncertainty, Doubt (“FUD”). Any one person in these meetings can derail the attempt or significantly delay the process. After one such upgrade experience you will lose the appetite for any future attempts.

Process – What To Do

If you are really looking ways for successfully convince the organization or your boss for an upgrade here is a summary of what worked for me.

1. Build confidence

Your organization must have faith in your team. They should believe that your team can deliver. They should know that you can fix issues fast and that you care about the success of both the business and the product. This won’t come in a single day. You need to build that confidence every day. One trick that works is to constantly solve simple problems. These kinds of problems won’t take much time. They don’t need big project plans. They come from all parts of the organization. Solving them proves that you are for continuous improvement.
Try to tackle troubled areas of the products. These are the parts of the software that attract frequent bugs, frequent code churns. Try to take initiative and make that area trouble free. Once you successfully solve such problems, you are guaranteed to gain the support in any of your future upgrade proposals.

2. Build safety net

If you don’t have them already, you should start writing tests for important areas of your application. These tests when written properly help you in two ways. They help you identify the problem quickly and also help serve as a repository of business knowledge. You must run these tests with every code check-in.

3. QA is your friend

The most important help you can get from you QA team is in the form of exploratory tests and also from testing tedious, non-automated parts of the application. A tester with an intention to solve the problem is the best resource than a tester with an intention to just report a problem. Have them involved in every non trivial decision your team makes.

4. Research

If the upgrade is going to have big impact on your product, wait for about 6 months after the release date of the upgrade. Go through the release notes, breaking changes, write small prototypes to test the areas of the concern.
Most importantly go through the product support forums to understand the quality of the version you are upgrading. Even if you decide not to upgrade, this research helps you decide the future direction of your product.

5. Build excitement within the team

Try to lay out the broad plan for the upgrade. Take your team’s consensus. Understand possible real problems. Tell them why the upgrade is important. A self-organizing, motivated team can make the whole upgrade process a trivial affair. As a team, you should try to think through and anticipate every major problem, and come up with possible options. Consider other major projects your team is dealing with to make sure that they are not being unnecessarily burdened. Come up with a possible time line.

6. Sell the proposal

Get the major stake holders in a room and sell your proposal. If you did your homework properly this should go well.

7. Appoint a facilitator

Find someone in the team (a PM, BA, team lead etc.) who can act as glue with all the team members. Their job is to continuously update all stake holders (management, team, end users etc.) about the team’s decisions, choices and find answers.

8. Short but to the point standup meetings

Meet twice a day to talk about the most important issues that you team is facing that day. This helps the team to share the work, gather questions, seek answers and inform stake holders about problems.
I too often here developers complaining that their organization is not good at keeping at the top of the latest technology trends. I believe that this is mostly due to the lack of effort from the software teams themselves.

Routes with non optional parameters

You can define restful routes in asp.net mvc applications. I recently ran into an interesting problem.

I defined the following route.

routes.MapRoute("MemberDetail", "Member/{memberId}", new { controller = "Employee", action = "Details" });

Here I am defining a route named "MemberDetails", that allows user to use urls similar to Member/1234 to get a member's detail.

This route works fine. However somewhere else in the code, when I try to output the route url in a page, by doing the following

UrlHelper.RouteUrl("MemberDetail")

To my surprise I saw an empty string in the page output. I wrote a test to see the problem.

[Test]
public void MemberDetailRoute()
{
Assert.IsNotNull(_mockedUrlHelper.RouteUrl("MemberDetail"))
}

Yes that test fails. It turns out that in case of non optional parameters, I must pass their values for MVC framework to identify the route. So the following test passes.

[Test]
public void MemberDetailRoute()
{
Assert.AreEqual("/Member/123", _mockedUrlHelper.RouteUrl("MemberDetail", new {memberId = 123}))
}

And in a view, if you ever want to dynamically construct the above url using javascript, you could do the following
Here we are using jquery to bind a click event to a button, that grabs the user entered value and gets the member detail url.

<script type="text/javascript">
var memberUrl = '<%: HttpUtility.UrlDecode(Url.RouteUrl("MemberDetail", new {memberId = {0}) )%>';
$('#Go').click(function() {
var memberId = $('#memberId').val(); // get the member id from a textbox
window.location.href = HttpUtility.UrlEncode(url.replace("{0}", memberId));
});
</script>

Here I am passing a dummy value {0} to get the correct url and then replacing that value with the memberId entered by user.
Note that Url.RouteUrl method always encodes the url string.

Reference

Microsoft Patterns and Practices Content

Connecting to a windows 7 printer from Mac

It seems that I am ending up searching for these instructions many times, just so I don't forget these steps

On the Windows 7 PC

1. In Windows 7 Control Panel

2. Select "Programs and Features" pane

3. Click "Turn Windows Features on or off"

4. Click on the "LPD Print Service". Under "Print and Document Services"

5. Ensure printer(s) are shared

On the Mac

1. Open "System Preferences" - "Print & Fax"

2. Click "+" to add a printer

3. Right click -> tool bar to customize and add "Advanced" button

4. Select "Advanced"

5. In the "Type:" field select "LPD/LPR Host or Printer"

6. In the "URL:" field enter lpd://ipaddressofwindows7machine/PrinterShareName

Using lpd://ServeName/PrinterShareName does not work, the IP address must replace ServerName

The IP address is the address of your PC, the one assigned by your router

7. Click the "Add" button

If nothing works, just reset the print system on your mac.  This would delete the existing printers allowing you to start fresh.

Asp.net WebSite Administration Tool

Asp.net allows to quickly setup a security for the site using WebSite Administration tool, which can be instantiated from visual studio.  If that tool throws an error, there is no place find what went wrong.

I ran into one such case with the following cryptic error.

"An error was encountered. Please return to the previous page and try again."

I found a work around for this problem.  I needed to hit F5 (run the site in the debug mode) and then click the "WebSite Administration tool" icon from solution explorer to get rid of the above error.