Tukukkk

Our ASP.NET web application has a form editor where users can create their own pages by dragging and dropping user controls onto a page and setting various properties for each user control. When end-users then open that page during runtime the usercontrols are placed on the page according to that configuration.

The editor also contains a business rule engine where the users can create simple if-then-else statements to direct page flow according to business logic. For example, a user can create the rule 'if Data.Customer.DateOfBirth equals to Today then HappyBirthdayScript becomes visible', where the HappyBirthdayScript is an instance of a user control. These business rules are then compiled by lambda expression trees, which in turn are cached for a day for performance.

Now the problem one of our customers has, is that generally during off-peak hours (after midnight) but sometimes even during peak hours during the day, ASP.NET seems to recompile the dynamically generated dlls for one or more of the ascx files. This leads to errors similar to this one:

System.InvalidCastException: [A]ASP.ui_controls_scripting_ascx cannot
be cast to [B]ASP.ui_controls_scripting_ascx. Type A originates from
'App_Web_scripting.ascx.cb748d43.1ce9kuug, Version=0.0.0.0,
Culture=neutral, PublicKeyToken=null' in the context 'Default' at
location 'C:WindowsMicrosoft.NETFramework64v4.0.30319Temporary
ASP.NET
Filesroota3b6cf39a9985cbdApp_Web_scripting.ascx.cb748d43.1ce9kuug.dll'.
Type B originates from 'App_Web_scripting.ascx.cb748d43.e6qrsk0w,
Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context
'Default' at location
'C:WindowsMicrosoft.NETFramework64v4.0.30319Temporary ASP.NET
Filesroota3b6cf39a9985cbdApp_Web_scripting.ascx.cb748d43.e6qrsk0w.dll'

This exception is caused because the Scripting usercontrol Type in the cached compiled business rule is different from the one in the new recompiled dll in the temporary ASP.NET files. The compiled expression tree casts the user control to its specified Type to access its properties without having to use the slower Reflection.

When you check this temporary folder both the old and the new dlls still exist. An application pool recycle obviously fixes the issue as the business rule will be recompiled when the cache is discarded.

It happens once every few days on either or both of their load-balanced servers. The application pools of both servers are set to recycle every day at 3:33 AM. None of our other customers have this problem. It also doesn't occur at the same time every time and it doesn't happen with the same control. So I'm kind of stumped to figure out the root cause of this.

As a workaround we're going to give this customer a special build where we precompile all UI components through the web publish functionality with a non-updateable UI in an attempt to stem this issue, but this isn't feasible in the long term as we have functionality to put new user controls in plugins, which still requires dynamic compilation. The customer thus wouldn't be able to use this feature if we turn off dynamic compilation altogether.

I'd prefer not recompiling the business rule to make use of the new Type whenever a cast exception occurs, as besides being another workaround, this might cause performance issues due to the cost of compilation.

Does anyone have any idea how I can investigate this issue further? I'm especially interested in more information how IIS determines when to dynamically recompile ascx files, and to find more logging on this, but so far I have yet to find anything useful. The Understanding ASP.NET Dynamic Compilation article on MSDN states the ascx/aspx/ashx/etc files are compiled on their first request and on changes to the file. However after a deployment these files should be static, especially at night when no one with RDP access to the server is even working at that time.