How to debug silent crashes in .Net

Here’s a quick note to my future self, and any other interested parties in the present on how to diagnose a particularly tricky kind of unhandled exception. I’m talking about those ninja exceptions which manage to evade any kind of unhandled exception logging you’ve rigged up around your .Net applications.

I was debugging just such a problem today. A customer would double click the application, and nothing would happen. Our application is configured to log any unhandled exceptions to a log file. But no log file was being created. There was, however, an entry written to the application event log: it told me that a TypeInitializationException had been thrown, and even gave me a stack trace. But it told me nothing about why the type had failed to initialize.

To complicate matters, this was one of those heisenbugs where, though I could replicate it under normal circumstances, it would go away if I attached a debugger to the process. What I needed was a crash dump – a snapshot of the entire state of the application at the moment the exception happened.

DebugDiag was my first port of call. That’s a nifty tool from Microsoft which can be configured to create dumps from your application under particular circumstances,  including when it crashes. Handily, it will also analyse the dump file, and help you work out why the application crashed. Inexplicably, DebugDiag failed to capture any dumps for my application.

So I turned to Windows Error Reporting. From Windows Vista SP1 onwards you can tweak some flags in the registry, and have windows automatically capture dumps for you – assuming you’re using .Net 4.0.

Capturing dump files with Windows Error Reporting

All you need to do is set create a key at HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting\LocalDumps\[Your Application Exe Name]. In that key, create a string value called DumpFolder, and set it to the folder where you want dumps to be written. Then create a DWORD value called DumpType with a value of 2.

Now get those ninjas to crash your app.

You should see a *.dmp file appearing in the folder you chose. Double click it – and you’ll see some Visual Studio magic, introduced in VS 2010. You can debug a dump file almost as if it were a live application. When you see the Minidump File Summary screen, you just need to click Debug with Mixed.

Unmasking the ninja

So what was this ninja exception that was evading my logging routines? It was an exception being thrown by my exception handling code!

The original exception was a ConfigurationErrorsException, thrown by the configuration API when loading my application’s user settings from a corrupt user.config file. My exception handling code looked like this:

private void ReportUnhandledException(Exception exception) { try { Tracing.TraceUnhandledError(exception); ReportExceptionToUser(exception); } catch (Exception fatalException) { Tracing.TraceUnhandledError(fatalException); } }

The problem was caused by a static constructor in the Tracing class. We’re using the System.Diagnostics TraceSource api to log our exceptions, and the constructor was setting all of that up. The TraceSource API, it turns out, also tries to load the application configuration, so that was throwing an exception.

The solution? Use Environment.FailFast instead of trying to log the secondary exception. That simply writes a message to the event log then bails out, with no chance of causing mayhem by raising further exceptions.

private void ReportUnhandledException(Exception exception) { try { Tracing.TraceUnhandledError(exception); ReportExceptionToUser(exception); } catch (Exception fatalException) { Environment.FailFast("An error occured whilst reporting an error.", exception); } }

A quick guide to Registration-Free COM in .Net–and how to Unit Test it

A couple of times recently I’ve needed to set up a .Net application to use Registration-Free COM, and each time I’ve had to hunt around to recall the details. Further, just this week I needed to write some unit tests that involve instantiating these un-registered COM objects, and that wasn’t straight forward. So, as much for the benefit of my future self as for you, my loyal reader, I’m going to summarise my know-how in quick blog post before it becomes used-to-know-how.

What is Registration-Free COM?

If you’re still reading, I’ll assume you know all about COM, Microsoft’s ancient technology for enabling components written in different languages to talk to each other (I wrote a little about it here, with some links to introductory articles). You are probably also aware of DLL Hell. That isn’t a place where bad executables are sent when they are terminated. Rather, it was a pain inflicted on developers by the necessity of registering COM components (and other DLLs) in a central place in the OS. Since all components were dumped into the same pool, one application could cause all kinds of hell for others by registering different versions of shared DLLs. The OS doesn’t police this pool, and it certainly doesn’t enforce compatibility, so much unexpected weird and wonderful behaviour was the result.

Starting with Windows XP, it has been possible to more-or-less escape this hell by not registering components in a central location, and instead using Registration-Free COM. This makes it much easier to deploy applications, because you can just copy a bunch of files – RegSvr32 is not involved, and there are no Registry keys to be written. You can be confident that your application will have no impact on others once installed.

It is all done using manifests.

Individual Manifest Files

For each dll, or ocx file (or ax files in my case – I’m working with DirectShow filters) containing COM components you need to create a manifest.

Suppose your dll is called MyCOMComponent.dll. Your manifest file should be called MyCOMComponent.sxs.manifest, and it should contain the following:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> <assemblyIdentity type="win32" name="MyCOMComponent.sxs" version="1.0.0.0" /> <file name="MyCOMComponent.dll"> <comClass description="MyCOMComponent" clsid="{AB12C3D4-567D-4156-802B-40A1387ADE61}" threadingModel="Both" /> </file> </assembly>

Obviously you need to make sure that the clsid inside comClass is correct for your component. If you have more than one COM object in your dll you can add multiple comClass elements. For those not wanting to generate these manifests by hand, a StackOverflow answer lists some tools that might help.

About Deployment

When you deploy your application you should deploy both the dll/ocx/ax file and its manifest into the same directory as your .Net exe/dlls. When developing in Visual Studio, I customise the build process to make sure all these dlls get copied into the correct place for running and debugging the application. I stole the technique for doing this from the way ASP.Net MVC applications manage their dlls.

Put all the dlls and manifests into a folder called _bin_deployableAssemblies alongside the rest of your source code. Then modify your csproj file and add the following Target at the end of it:

<!-- ============================================================ CopyBinDeployableAssemblies This target copies the contents of ProjectDir\_bin_deployableAssemblies to the bin folder, preserving the relative paths ============================================================ --> <Target Name="CopyBinDeployableAssemblies" Condition="Exists('$(MSBuildProjectDirectory)\_bin_deployableAssemblies')"> <CreateItem Include="$(MSBuildProjectDirectory)\_bin_deployableAssemblies\**\*.*" Condition="Exists('$(MSBuildProjectDirectory)\_bin_deployableAssemblies')"> <Output ItemName="_binDeployableAssemblies" TaskParameter="Include" /> </CreateItem> <Copy SourceFiles="@(_binDeployableAssemblies)" DestinationFolder="$(OutDir)\%(RecursiveDir)" SkipUnchangedFiles="true" Retries="$(CopyRetryCount)" RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)" /> </Target>

To make sure that target is called when you build, update the AfterBuild target (uncomment it first if you’re not currently using it):

 <Target Name="AfterBuild" DependsOnTargets="MyOtherTarget;CopyBinDeployableAssemblies" />

The Application Manifest

Now you need to make sure your application declares its dependencies.

First add an app.manifest file to your project, if you haven’t already got one. To do this in Visual Studio, right click the project, select Add –> New Item … and then choose Application Manifest File. Having added the manifest, you need to ensure it is compiled into your executable. You do this by right-clicking the project, choosing Properties, then going to the Application tab. In the resources section you’ll see a Manifest textbox: make sure your app.manifest file is selected.

Now you need to add a section to the app.manifest file for each dependency.

By default your app.manifest file will probably already have a dependency for the Windows Common Controls. After that (so, nested directly inside the root element) you should add the following for each of the manifest files you created earlier:

<dependency> <dependentAssembly> <assemblyIdentity type="win32" name="MyCOMComponent.sxs" version="1.0.0.0" /> </dependentAssembly> </dependency>

Notice that we drop the “.manifest” off the end of the manifest file name when we refer to it here. The other important thing is that the version number here and the one in the manifest file should exactly match, though I don’t think there’s any reason to change it from 1.0.0.0.

Disabling the Visual Studio Hosting Process

There’s just one more thing to do before you try running your application, and that is to turn off the Visual Studio hosting process. The hosting process apparently helps improve debugging performance, amongst other things (though I’ve not noticed greatly decreased performance with it disabled). The problem is that, when enabled, application executables are not loaded directly- rather, they are loaded by an intermediary executable with a name ending .vshost.exe. The upshot is that the manifest embedded in your exe is ignored, and COM components are not loaded.

Disabling the hosting process is simple:  go to the Debug tab of your project’s Properties and uncheck “Enable the Visual Studio hosting process”

With everything set up, you’ll want to try running your application. If you got everything right first time, everything will go smoothly. If not you might see an error like this:

If you do, check Windows’ Application event log for errors coming from SideBySide. These are usually pretty helpful in telling you which part of your configuration has a problem.

Summary

To re-cap briefly, here are the steps to enabling Registration-Free COM for you application:

1. Create a manifest file for each COM dll
2. Make sure both COM dlls and manifest files are deployed alongside your main executable
3. Add a manifest file to your executable which references each individual manifest file
4. Make sure you turn off the Visual Studio hosting process before debugging

Unit Testing and Registration-Free COM

And now, as promised, a word about running Unit Tests when Registration-Free COM is involved.

If you have a Unit Test which tries to create a Registration-Free COM object you’ll probably get an exception like

Retrieving the COM class factory for component with CLSID {1C123B56-3774-4EE4-A482-512B3AB7CABB} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)).

If you don’t get this error, it’s probably because the component is still registered centrally on your machine. Running regsvr32 /u [Path_to_your_dll] will unregister it.

Why do Unit Tests fail, when the application works? It is for the same reason that the Visual Studio hosting process breaks Registration-Free COM: your unit tests are actually being run in a different process (for example, the Resharper.TaskRunner), and the manifest file which you so carefully crafted for your exe is being ignored. Only the manifest on the entry executable is taken into account, and since that’s a generic unit test runner it says nothing about your COM dependencies.

But there’s a workaround. Win32 has some APIs –the Activation Context APIs- which allow you to manually load up a manifest for each thread which needs to create COM components. Spike McLarty has written some code to make these easy to use from .Net, and I’ll show you a technique to incorporate this into your code so that it works correctly whether called from unit tests or not.

Here’s Spike’s code, with a few minor modifications of my own:

/// <remarks> /// Code from http://www.atalasoft.com/blogs/spikemclarty/february-2012/dynamically-testing-an-activex-control-from-c-and /// </remarks> class ActivationContext { static public void UsingManifestDo(string manifest, Action action) { UnsafeNativeMethods.ACTCTX context = new UnsafeNativeMethods.ACTCTX(); context.cbSize = Marshal.SizeOf(typeof(UnsafeNativeMethods.ACTCTX)); if (context.cbSize != 0x20) { throw new Exception("ACTCTX.cbSize is wrong"); } context.lpSource = manifest; IntPtr hActCtx = UnsafeNativeMethods.CreateActCtx(ref context); if (hActCtx == (IntPtr)(-1)) { throw new Win32Exception(Marshal.GetLastWin32Error()); } try // with valid hActCtx { IntPtr cookie = IntPtr.Zero; if (!UnsafeNativeMethods.ActivateActCtx(hActCtx, out cookie)) { throw new Win32Exception(Marshal.GetLastWin32Error()); } try // with activated context { action(); } finally { UnsafeNativeMethods.DeactivateActCtx(0, cookie); } } finally { UnsafeNativeMethods.ReleaseActCtx(hActCtx); } } [SuppressUnmanagedCodeSecurity] internal static class UnsafeNativeMethods { // Activation Context API Functions [DllImport("Kernel32.dll", SetLastError = true, EntryPoint = "CreateActCtxW")] internal extern static IntPtr CreateActCtx(ref ACTCTX actctx); [DllImport("Kernel32.dll", SetLastError = true)] [return: MarshalAs(UnmanagedType.Bool)] internal static extern bool ActivateActCtx(IntPtr hActCtx, out IntPtr lpCookie); [DllImport("kernel32.dll", SetLastError = true)] [return: MarshalAs(UnmanagedType.Bool)] internal static extern bool DeactivateActCtx(int dwFlags, IntPtr lpCookie); [DllImport("Kernel32.dll", SetLastError = true)] internal static extern void ReleaseActCtx(IntPtr hActCtx); // Activation context structure [StructLayout(LayoutKind.Sequential, Pack = 4, CharSet = CharSet.Unicode)] internal struct ACTCTX { public Int32 cbSize; public UInt32 dwFlags; public string lpSource; public UInt16 wProcessorArchitecture; public UInt16 wLangId; public string lpAssemblyDirectory; public string lpResourceName; public string lpApplicationName; public IntPtr hModule; } } }

The method UsingManifestDo allows you to run any code of your choosing with an Activation Context loaded from a manifest file. Clearly we only need to invoke this when our code is being called from a Unit Test. But how do we structure code elegantly so that it uses the activation context when necessary, but not otherwise? Here’s my solution:

public static class COMFactory { private static Func<Func<object>, object> _creationWrapper = function => function(); public static T CreateComObject<T>() where T:new() { var instance = (T)_creationWrapper(() => new T()); return instance; } public static object CreateComObject(Guid guid) { Type type = Type.GetTypeFromCLSID(guid); var instance = _creationWrapper(() => Activator.CreateInstance(type)); return instance; } public static void UseManifestForCreation(string manifest) { _creationWrapper = function => { object result = null; ActivationContext.UsingManifestDo(manifest, () => result = function()); return result; }; } }

Whenever I need to create a COM Object in my production code, I do it by calling COMFactory.CreateCOMObject. By default this will create the COM objects directly, relying on the manifest which is embedded in the executable.

But in my Test project, before running any tests I call COMFactory.UseManifestForCreation and pass in the path to the manifest file. This ensures that the manifest gets loaded up before we try to create any COM objects in the tests.

To avoid duplicating the manifest file, I share the same file between my Test project and main executable project. You can do this right clicking your test project, choosing Add->Existing Item… then app.manifest in your main project. Finally, click the down arrow on the Add split button, and choose Add as Link.

If you’ve got any tips to share on using Registration-Free COM, whether in Unit Tests or just in applications, please do leave a comment.

Build Windows Day 1–and a peek a what’s to come

If you haven’t caught the news already, yesterday at Microsoft’s Build Windows conference Steven Sinofsky announced a preview of Windows 8 and a bunch of developer tools, including Visual Studio 11. You can download it here.

As predicted, Windows 8 comes with an all new development framework, the Windows Runtime (WinRT). I’ve not read enough myself to begin explaining. But here’s a picture, snapped during Steven Sinofsky’s keynote:

WinRT is an object-oriented API, consisting of 1800 objects, accessible to managed, unmanaged and JavaScript code alike. Sinofsky emphasised that this WinRT is a peer to Win32, not another abstraction layer built on top of it. As this diagram clearly shows, and Sinofsky stressed in his talk, Win32, .Net and Silverlight developers have nothing to fear, all current frameworks continue to work in Windows 8.

I’m sure to have more to say about WinRT in the coming days. In the meantime, InfoQ has a couple of articles summarising what we learned in the keynote. If you’re wanting a sneak peak at the API’s themselves, check out the preview documentation on MSDN.

The videos are the first few sessions of the conference are already up on the web:

• The Day 1 Keynote
• Jensen Harris: 8 traits of great Metro style apps
• Aleš Holeček and John Sheehan: Platform for Metro style apps

What’s next?

In other news, the session list is now up for the rest of the week. Here are a few that grab my attention.

• Anders Hejlsberg on Future directions for C# and Visual Basic. Doesn’t look like “Roslyn” (compiler-as-a-service) will make it into Visual Studio 11. But he might announce some new language features for C# 5.0.
• Mark Miller and Pracheeti Nagarkar give a Deep dive into the kernel of the .Net Framework. Sounds like they’ve done yet more work to improve code-generation and garbage collection.
• Stephen Toub, one of my favourite speakers from previous years, is doing two sessions: The zen of async: Best practices for performance (vital know-how, given that many of the new WinRT API’s will be asynchronous) and Building parallelized apps with .Net and Visual Studio (look for some announcements about new parallelization libraries).
• Harry Pierson and Jesse Kaplan are doing a talk on Using the Windows Runtime from C# and Visual Basic
• Out of curiosity, I make take a look at Daniel Moth’s Taming GPU compute with C++ AMP where he’s going to introduce new tools and language features for using the GPU within C++ solutions.
• Boris Jabes talk A lap around DirectX game development tools also piques my interest, as it promises the “most significant set of improvements for developing graphics-intensive apps in over a decade”.

And now, on with the show!

Microsoft’s Build Windows Conference is Next Week – What’s in Store?

It’s less than a week to go until Microsoft’s Build conference. According to tradition, about now I should be speculating on what the future holds for us faithful Microsoft developers. But there’s still no official session list. Fortunately, we do have a few official hints, and a couple of very unofficial leaks to whet our appetites for what’s in store.

Windows 8

Read the Build homepage, and you’ll see that the conference is focused very clearly on Windows 8. We’ve already had a preview of the new touch-centric, Windows Phone 7-like UI. And over on the Building 8 blog, Steven Sinofsky and others on the Windows team have started announcing some of the to-be-expected features like USB 3.0 support.

Here are some of the other things that we know already:

• Windows 8 will support ARM processors, and System on a Chip architectures. Look out for a slew of Windows 8 based tablets to compete with the iPad and Galaxy Tab. Indeed, there is a rumour doing the rounds that Microsoft will be giving away a Windows 8 tablet to all Build attendees.
• The Windows Desktop is being demoted to just another application, that some users (think Tablet users) may never see. Instead they’ll live in the new immersive Metro environment.
• Windows Explorer is being Ribbonified, and the File Copy/Move experience given an overhaul.
• It’s goodbye to Virtual CloneDrive and friends, as Windows 8 gains the ability to mount ISO files, and also simplifies mounting of VHD files.
• Windows 8 will be getting its own App Store.

And now comes the controversial part. Those of you who have not entirely delegated their memories to Google will recall that when Microsoft first showed the new Metro desktop for Windows 8, they announced that developers would be able to use HTML5 and JavaScript for building these new-fangled immersive apps. But they said nothing about the presence of .Net, WPF or Silverlight in this brave new world, stirring up an instant furore in the blogosphere.

But there’s no need to panic. It’s all under control. I think.

Direct UI

This is where we turn to the leaks. Leaked Windows 8 builds, that is.

According to Peter Bright over on Ars Technica, Microsoft are creating a new Windows Runtime (or WinRT), which is intended to be the successor to the Win32 API. It will be a native-code API, but shaped in a way that is pleasing to the eye of a managed-code developer, and what is more, said .Net developers will be able to access WinRT through a new (hopefully painless) version of COM for which support is being built into .Net 4.5.

Included in WinRT is a new UI framework called Direct UI which appears to be a lean-and-mean version of WPF/Silverlight (and also uses XAML – remember how part of the XAML team got moved to the Windows division!). With the official details coming out in less than a week, there’s little point on me elaborating further here, but you can get a taster of the new APIs by reading these two forum threads which dissect some of the leaked Windows 8 builds.

What I look forward to hearing is where WPF fits into the picture. One thing we know with a high degree of confidence, given Microsoft’s backwards-compatibility track record: current WPF applications will continue to work. The question is: will there be further development to WPF? As I reported last year, we know there are some new features coming, including fixing the airspace issues when hosting Hwnds inside WPF controls, and enabling hosting of Silverlight controls. But will there be anything beyond that? And will there be interop between WPF and Direct UI? Watch this space.

Silverlight

Remember that Microsoft stirred up another firestorm at PDC 2010 by staying mum about Silverlight? They put that right a few weeks later by announcing Silverlight 5, which was quite distinctly not a maintenance release, since it included a whole raft of new features like a 3d API, vector printing support, and P/Invoke for Trusted Applications. They’ve now made good on that, with the Silverlight 5 Release Candidate coming out just last week. It will be interesting to learn Microsoft’s vision for how Silverlight, WPF and Direct UI align.

C# 5.0

We already know the headline feature for C# 5: asynchronous methods. We’ve had a CTP. Here’s hoping for a beta release during the conference. One thing Anders did say at his talk last year is that async won’t be the only new feature in C# 5.0. So I wonder what else he has up his sleeves? It would be nice if it was the Roslyn project (Compiler as a Service).

Visual Studio

It sounds like Microsoft are preparing to release a new preview build of Visual Studio at Build. Many of the features that were previously released as PowerToys for VS 2010 are going to part of vNext. But the more interesting news to me is that Microsoft have been taking note of data coming back from PerfWatson to make some big performance improvements. Visual Studio vNext is going to make more use of multi-core machines, and will reduce memory usage in some cases by doing builds out-of-process for C# projects.

Stay Tuned

My new boss has kindly given me some time next week to follow the Build conference from the comfort of my office. And as in previous years, I’ll be reporting back the choicest titbits as I find them. Follow me on twitter to hear it as it happens.

Now, over to you. What are you looking forward to? Have you heard any rumours that I’ve not picked up on?

Debugging Windows batch files when used as scheduled tasks

Here’s a little gem that I pieced together thanks to Google this morning: how to trouble-shoot a Windows bat file when using it as a scheduled task.

We’re using SQL Express, and we want to make sure all our databases are safely backed up on a regular schedule. One thing that Microsoft cut out of SQL Server when pruning it to create the free version is SQL Agent, the tool that enables you to run scheduled tasks against the database.

No big deal: following Greg Robidoux’s advice I created a stored procedure to backup a database, and then wrote a batch file that used SQLCMD to execute it for each database on the server. Add to the batch file a call to RoboCopy to transfer the backups to our NAS drive, then set up a scheduled task against the batch file, and I’m done, I thought.

If only!

The first problem was how to get the task to run under the Local System account – I didn’t want to use a standard account, because then I have the hassle of password management (I’m using Windows Server 2003 here – if I was on Windows Server 2008 R2 I could use Managed Service Accounts and have the server take care of the passwords). Going through the Add Scheduled Task UI doesn’t give you the option of using the Local System account.

For that, I discovered, you need to use the at command:

at 20:00 /every:m,t,w,th,f,sa,su "D:\Backups\DoBackups.bat"

does the job, scheduling my batch file for 8:00pm every day of the week.

OK. Scheduled task appears in the list. Run it to check that it works. Uh oh!

The task clearly failed to complete, but how was I to find out why? Clicking Advanced > View Log in the scheduled tasks window brings up the log file – the completely useless log file that tells you that your task started and then stopped straight away “with an error code of (2)”. Right – could you be more … specific?

So I pushed further into the murky world of bat file programming.

Joshua Curtis saved the day. His post on Redirecting Output to a File in Windows Batch Scripts gave me exactly what I needed.

First, I refactored my batch script into DoBackupsCore.bat. Then, in DoBackups.bat I wrote this:

echo [%date% - %time%] Log start > D:\Backups\log.txt CALL "D:\Backups\DoBackupsCore.bat" >> D:\Backups\log.txt 2>&1

On the first line the > command redirects output to the log file, erasing anything already in it. In the 2nd line, the >> command redirects the output of my actual backup script to the log file, but appends to what’s already there. The really clever part of this magic spell is the last 4 characters: "2>&1”. I refer you to Joshua for the details, but this basically makes sure that errors are logged to the file, as well as successful outcomes

So I got what I needed: lovely, wordy error messages enabling me to fix my script and go home with that lovely backed-up feeling.