I recently did a talk at DevEvening about how I managed to using Amazon’s Simple Email Service with my (fairly) new website The Gig Market and I thought I would just share a bit more information about it if anyone was interested.

The Problem

As the website is hosted on Windows Azure there is no hosting mail server that I can use to send out emails. The answer is to use a 3rd Party service to do this for you. I had been using a company for 3 years for my other websites to do this and had been very happy with them…. until I breached the send limit one month and they  just stopped sending emails. This really annoyed me so I set about to find a better service. What I found was AWS’s SES which only costs $0.10 per 1000 emails, the problem was it is a webservice, not an smtp server, so I couldn’t just change the smtp server in my app’s settings.

The Research

What I wanted to do was to tweak my code to use SES api. To limit the impact I only wanted to change the:

SmtpClient smtp = new SmtpClient();
smtp.Send(mailMessage);

code to something that could just send the mailMessage object to Amazon SES. The problem is that out of the box the .Net MailMessage object can only be used with the .Net SmtpClient. doh.

I found a couple of other people with the same idea:

http://www.codeproject.com/KB/IP/smtpclientext.aspx

This post by Allan Eagle was a great start for me as it showed me how to get at the guts of the mail message object by using reflection. Now I only had to  send it to SES right?

http://neildeadman.wordpress.com/2011/02/01/amazon-simple-email-service-example-in-c-sendrawemail/

This post by Neil Deadman was a great article and I thought I had found my solution BUT there was a big big problem. I needed to send BCC and set the priority of the email. It turned out that I could not achieve this using the solution. So what could I do?

The Solution

I ended up hitting the Amazon docs and discovered that I would have to send a RAW message format and I would also have to write the email out by hand in MIME format. I thought this would be hard but it turned out to be pretty easy, I knocked up an extension method on the MailMessage class to do this for me.

public static string ToAmazonSesRawFormat(this MailMessage message)
        {
            var result = new StringBuilder();
            result.AppendLine("MIME-Version: 1.0");
            result.AppendLine(string.Format("From: {0}", message.From));

            if (message.To.Count > 0)
            {
                result.Append("To: ");
                int toMessageCount = 0;
                foreach (var address in message.To)
                {
                    result.Append(string.Format(
                        "{0}{1}",
                        toMessageCount == 0 ? string.Empty : ",",
                        address));

                    toMessageCount++;
                }    
            }
            
            if (message.CC.Count > 0)
            {
                result.AppendLine(string.Empty);
                result.Append("Cc: ");
                int ccMessageCount = 0;
                foreach (var address in message.CC)
                {
                    result.Append(string.Format(
                        "{0}{1}",
                        ccMessageCount == 0 ? string.Empty : ",",
                        address));

                    ccMessageCount++;
                }    
            }
            
            if (message.Bcc.Count > 0)
            {
                result.AppendLine(string.Empty);
                result.Append("Bcc: ");
                int bccMessageCount = 0;
                foreach (var address in message.Bcc)
                {
                    result.Append(string.Format(
                        "{0}{1}",
                        bccMessageCount == 0 ? string.Empty : ",",
                        address));

                    bccMessageCount++;
                }    
            }
            result.AppendLine(string.Empty);
            result.AppendLine("Subject: " + message.Subject);
            result.AppendLine(string.Format("Content-Type: {0}", message.IsBodyHtml ? "text/html;" : "text/plain;"));
            result.AppendLine(string.Format("Content-Transfer-Encoding: quoted-printable"));
             
            result.AppendLine(string.Format("X-Priority: {0}", ((int) message.Priority).ToString()));

            result.AppendLine(string.Empty);

            if (message.IsBodyHtml)
            {
                var encoder = new QuotedPrintableEncoder();
                result.AppendLine(encoder.EncodeFromString(message.Body, Encoding.ASCII));    
            } 
            else
            {
                result.AppendLine(message.Body);
            }
            
            return result.ToString();
        }

Now all I had to do was send the mesage via the SES RawMessage api:

AmazonSes.SendEmail.Instance.SendRawMessage(mailMessage);

The SendRawMessage function is a wrapper I put in the wrapper class I wrote for the API:

public void SendRawMessage(MailMessage mailMessage)
        {
            var memoryStream = new MemoryStream();
            using (memoryStream)
            {
                var encoding = new UTF8Encoding();
                var byteArray = encoding.GetBytes(mailMessage.ToAmazonSesRawFormat());

                memoryStream.Write(byteArray, 0, byteArray.Length);
                memoryStream.Position = 0;
                var message = new RawMessage(memoryStream);
                var sendRawMessageRequest = new SendRawEmailRequest(message);
                var response = Client.SendRawEmail(sendRawMessageRequest);
            }
            
            memoryStream.Close();
        }

On my current Microsoft Dynamics Crm project we have done a lot of customisations, both creating custom pages and manipulating the existing crm pages via the OnLoad method. This post describes a method of ensuring the load order plus the benefit in increasing performance compared with loading multiple external JavaScript files the more conventional way.

The existing approach

One of the big problems we faced was making sure that the JavaScript files load in a specific order. This is because we have got some common functions in a file which can be re-used in multiple pages. These need to be loaded before the main page JavaScript file which might call one of the common functions.

There are a number of good blog post about this topic so I won’t go over this ground again. Here are a few articles I’ve found:

http://danielcai.blogspot.com/2010/02/another-talk-about-referencing-external.html
http://www.henrycordes.nl/post/2008/05/27/External-js-file-and-CRM.aspx

What is different with this new approach

The solution that I propose here was the brainchild of @njwatkins who came up with the idea of implementing at generic .net http handler as part of our custom webpages proejct that would read in the various external JavaScript files, in the correct order, compress them and then stream them back to the browsers as a single JavaScript file.

Unfortunately I cannot post the source code of the handler we use on this project here but a google round on topics like Gzip and StreadReaders and Handlers should enough to get most developers to a solution.

So the is script that goes on the onload of the Crm Enitty:

function loadJavaScript(file, onComplete) {
        var script = document.createElement('script');
        script.type = 'text/javascript';
        script.src = file;
        if (onComplete) {
            script.onreadystatechange = function() {
            if (this.readyState == 'complete' || this.readyState == 'loaded') {
                onComplete();
            }
        }
    }
    document.getElementsByTagName('head')[0].appendChild(script); 
};

loadJavaScript('/ISV/Northwind/ExternalJavaScript/contact.ashx', function() { if (EntityFormOnLoad) { EntityFormOnLoad(); } });

The contact.ashx file is the handler that does all the work and where you would define which JavaScript files to include and does the merging and compressing. It means you only need to load one external JavaScript reference and you can guarantee which order they external files will be loaded.

Other benefits of this approach is that you can control things like caching length which can be a problem when changing and deploying new external files to clients. We achieved a significant saving on load time of the javascript from 900ms down to 200ms using this approach and we have ideas on how to improve it further but this is as far as we have got today!

Now this might seem like something that would be easy to do but I’ve just spent 2 days struggling to do just this because of what I consider a bug in one of the SDK wrappers. I have now found a work around to enable unit testing which I will share with you now.

The Error message

Test method XrmEntityWrappers.Tests.CaseEntity.GetCaseByTicketNumber threw exception:  System.TypeInitializationException: The type initializer for 'Microsoft.Xrm.Client.Caching.Cache' threw an exception. --->  System.IO.DirectoryNotFoundException: Could not find a part of the path 'appDomain=UnitTestAdapterDomain_ForC:\Projects\Thg.Ohov.Crm\SourceCode\Thg.Ohov.Crm\TestResults\dh27_WIN-51UPWVCUQ6V 2010-09-16 18_21_40\Out\XrmEntityWrappers.Tests.dll:key=Microsoft.Xrm.Client.Caching.InMemoryCacheProvider'..

System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
b__0(Object userData)
System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
System.Threading.Mutex..ctor(Boolean initiallyOwned, String name, Boolean& createdNew, MutexSecurity mutexSecurity)
System.Threading.Mutex..ctor(Boolean initiallyOwned, String name)
Microsoft.Xrm.Client.Threading.MutexExtensions.Lock(String key, Int32 millisecondsTimeout, Action`1 action)
Microsoft.Xrm.Client.Threading.MutexExtensions.Get[T](String key, Int32 millisecondsTimeout, Func`2 loadFromCache, Func`2 loadFromService)
Microsoft.Xrm.Client.Threading.MutexExtensions.Get[T](String key, Int32 millisecondsTimeout, Func`2 loadFromCache, Func`2 loadFromService, Action`2 addToCache)
Microsoft.Xrm.Client.Threading.MutexExtensions.Get[T](String key, Func`2 loadFromCache, Func`2 loadFromService, Action`2 addToCache)
Microsoft.Xrm.Client.Caching.InMemoryCacheProvider.GetExtendedCache()
Microsoft.Xrm.Client.Caching.CacheManager.GetExtendedCache()
Microsoft.Xrm.Client.Caching.Cache..cctor()
Microsoft.Xrm.Client.Caching.Cache.Get[T](String label, Func`2 load)
Microsoft.Xrm.Client.CrmConnection..ctor(String connectionStringName, String connectionString)
Microsoft.Xrm.Client.CrmConnection.Parse(String connectionString)
Thg.Ohov.Crm.Core.XrmEntityWrappers.XrmAdapter..ctor() in C:\Projects\Thg.Ohov.Crm\SourceCode\Thg.Ohov.Crm\Core\XrmEntityWrappers\XrmAdapter.cs: line 28
Thg.Ohov.Crm.Core.XrmEntityWrappers.incident.get_XrmAdapter() in C:\Projects\Thg.Ohov.Crm\SourceCode\Thg.Ohov.Crm\Core\XrmEntityWrappers\incident.cs: line 25
Thg.Ohov.Crm.Core.XrmEntityWrappers.incident.GetIncident(String caseId) in C:\Projects\Thg.Ohov.Crm\SourceCode\Thg.Ohov.Crm\Core\XrmEntityWrappers\incident.cs: line 44
XrmEntityWrappers.Tests.CaseEntity.GetCaseByTicketNumber() in C:\Projects\Thg.Ohov.Crm\SourceCode\Thg.Ohov.Crm\XrmEntityWrappers.Tests\CaseEntity.cs: line 23

The reason for the error

The Microsoft.Xrm.Client.dll tries to create a Mutex object with the

Thread.GetDomain().FriendlyName;

When running ordinary in a console app or web app this is not a problem as the FriendlyName does not contain any ‘\’ characters. However UnitTest frameworks do put ‘\’ characters in the GetDomain().FriendlyName which then causes the Mutex object to throw a ‘System.IO.DirectoryNotFoundException’.

The fix

The real fix is for Microsoft to update the Microsoft.Xrm.Client.dll so that it doesn’t put any ‘\’ characters into the Mutex constructor. However my work around for this is thanks to Nick Watkins who found this article on how to change the GetDomain().FriendlyName

http://www.timvasil.com/blog14/post/2008/11/Fixing-Instance-names-used-for-writing-to-custom-counters-must-be-127-characters-or-less.aspx

The key bit of code being this if you want to set the FriendlyName to ‘Test’ (which doesn’t have any ‘\’ characters!):

typeof(AppDomain).GetMethod("nSetupFriendlyName", BindingFlags.NonPublic | BindingFlags.Instance).Invoke(AppDomain.CurrentDomain, new object[] { "Test" });

To rename the GetDomain().FriendlyName before calling any of the wrapper code in the unit tests. So the test might look a bit like this:

[TestMethod()]
        public void GetIncidentTest()
        {

typeof(AppDomain).GetMethod("nSetupFriendlyName", BindingFlags.NonPublic | BindingFlags.Instance).Invoke(AppDomain.CurrentDomain, new object[] { "Test" });

            string caseId = "1234"; // TODO: Initialize to an appropriate value
            incident expected = null; // TODO: Initialize to an appropriate value
            incident actual;
            actual = incident.GetIncident(caseId);
            Assert.AreEqual(expected, actual);

}

Summary

I’m happy now I can unit test my custom code that uses the xRM Wrappers and I hope that my support call with Microsoft will result in the SDK dll being updated.

I signed up for this Paypal event 5 weeks ago when I first saw it on Twitter - CharityHack. I thought it was going to be a couple of days of workshops where Paypal show developers how to use their new Adaptive Payments API…. how wrong was I!

The penny only dropped [more] about what the event was really going to be like last Thursday morning, 2 days before the event. I was reading their website and reading the “What to bring” section and it included a sleeping bag, I had never been to a workshop like this before.

See PayPals lets talk page for a more detailed run down (plus a video which I get a couple of seconds in!)

The event was in fact an opportunity for Paypal, JustGiving and MissionFish to talk to developers about their new APIs then us developers were given 24hours to create an application that could raise money for charity (preferably using the APIs we had been told about).

The idea I came up with in the hour after the penny dropped was to create an application that local volunteers can log their expenses when doing charity work on a website. The charity that they work for would then approve the expenses and they become visible to the public to pay on the charities behalf. If at the end of the month the expense had not been paid then the charity would have to settle up as they are obliged to do.

A member of the public would be able to donate with just a few easy steps:

1. Search for volunteers in their area with a postcode / radius search
2. Select the volunteers they want to pay
3. Click “Donate” and make a single payment to the charity for the amount.

Using the new Paypal adaptive payment API it is possible to split the donation behind the scenes and distribute it to the individual volunteers automatically, which is really cool. This is called a “Chained Payment”.

Our team name –

Redbull and Coffee

I teamed up with one of the only other .Net developers in the room of over 100. His name was Lee Mallon and immediately struck up a great working relationship. Lee has a great idea as well around allowing skilled professionals to donate their time to charity, however after discussing it further we felt that idea was just too big for a 24hour effort.

The Paypal venue was great and the Paypal team kept everyone very well fed and watered through the 24hours. There were lots of ideas being developed and a really nice feel to the whole event. Lee and myself were on a roll and before we knew it we had spent 16hours straight working on the application and it sort of worked! Other attendees were also hard at work, the LovePie team had even resorted to getting their sweatbands out.

Lee and I then spent another 7 hours making it look sort of presentable and testing that it all worked. After 23hours and 30mins we had finished out “Local Volunteers” charity application! We were very very chuffed.

I must state again how great the Paypal team where. The had flown their key personnel over from the American who were on hand through the night to help when we had questions and even helped me debug a rather tricky error which turned out to be a copy and paste error by me :s (sorry Rob – it was 2.30am). There were also techies from JustGiving and MissionFish on hand as well.

After 24hours all the teams were invited to present what they had achieved in the previous 24hours. There were many ideas which can be found on the Charity Hack wiki. I got a particular buzz when Musaab At-Taras, the Director of PayPal Platform commented that we had done “a really great job”.

There was one team that had worked on a particularly good idea, called CharityFrag, which I feel will have great appeal to a largely untapped community by charities. It was a mod for a First Person Shooter game. Players pledged money to charity before entering the game and picked their own charity. When they killed someone in game they took some of that persons money for their charity, equally when they were killed some of their money went to their opponents charity. This was all fed to a website keeping a real-time tracking on who had donated what charities – it is a truly great idea I hope their prize of a trip for San Fran to attend the Paypal conference there is the springboard required to get this idea into a production quality solution.

Lee and I were told we came a close second and got a special mention for our effort, which means a lot. Talking to Lee afterwards we both agreed that the best team had won.

So my hacking weekend cherry is broken and I would strongly recommend developers should seek out and got to these sort of events – it is a great social event and you can learn lots of new skills. My goal is to do one every 6 months.

I also plan to get our Local Volunteers application live in the next 3 weeks and hope that it will be a useful tool for raising money.

I have managed to arrange Ian Crowther, an ex-colleague from Avanade, to come and do a brown bag session for me and my employees this Saturday 14th March at my office near Haslemere in the UK.

Ian has been working a lot with Microsoft’s MVC.net and Yahoo UI recently. He is going to give a presentation and then run a practical coding workshop on Microsoft’s MVC.net and Yahoo UI showing how to combine them to [more]produce some sexy looking websites. In the workshop we are going to attempt to re-implement a recent asp.net web forms project See The Link have been working on as an asp.net MVC.net project. I’m really excited to find out more about how this will work and what the result will be! If they are good I’m hoping to put the MVC.net version live instead of the original web forms version.

I know that it is short notice but I am opening up this event to any techies who might be interested. I have space for another 3 people so if you want to come along then ping me an email before this Friday. We will be starting at 10am on Saturday going onto whenever (hopefully be about 4-6 hours or so)!

I have been using Blogengine.net for over a year now and have thought it is a great application. In this post I will tell you how I have successfully tweaked it to deliver a lightweight CMS system that I required. (I have published the source code at the bottom of this article)

At the time of writing I have successfully implemented a few websites powered completely with Blogengine.net:

http://www.seethelink.co.uk

http://www.petersfieldparish.org.uk

http://www.tonytinman.co.uk

http://www.marvelav.com

[more]Some of the things that I really like about Blogengine.net is how it can run just using Xml file out of the box and if you want to use a database you can. I like all the Search Engine Optimizations you get for free, I like how it implements semantic web technologies, I like how you can use 3rd Party tools to edit content (Live Writer for example)... the list goes on and on.

I have recently quit my job at Avanade and have setup my own website which I'm hoping to make a living out of. To pay the bills in the mean time I have been picking up small projects along the way. One of these was for a local parish church Petersfield Parish website. The main thing they wanted was a calendar of events & services which I completed using a standard asp.net website. They also wanted to allow members of the parish to contribute news items to the site which I thought was a perfect candidate for a blog solution.

I initially setup the blog with BlogEngine.net in a separate site and created an RSS reader to pull through the latest articles to the main site. However it wasn't the best solution as it would be nicer to have it all under one roof. I was also having problems with the site's webmaster has he used a Mac and Dreamweaver which was having all sorts of problems understanding the asp.net pages and breaking the site when new content was uploaded.

With these two problems to solve I decided to look at using BlogEngine.net to host all the content as well as the blog so the the webmaster could create and maintain content online with no risk of breaking the site and the contributors and blog articles would be under one roof.

I found this article about using Blogengine.net as a CMS solution but it seemed kind of fiddly as all the pages had to be standard.net pages with a usercontrol embedded in them for the content which would still leave me with my webmaster problem and I also thought this is essentially what Blogengine.net pages do.

So I decided to see if I could use the standard BlogEngine.net functionality to create the whole site. To my joy it was pretty straight forward to get 80% of the way there. I created a new theme based on the original site I had created and it looked the business. However there were a number of problems. The pages were in a random order and I wasn't sure how to get my funky calendar controls to work.

Fixing the page sort order was pretty straight forward as I just added a new property to the page entity in the XmlProvider to take an integer. I then amended the PageSiteMap class to sort the pages based on the new value and then Robert is your fathers brother.

I was still struggling with the calendar controls when I spied something in the BlogEngine.net source code

ERROR - UNABLE TO LOAD CONTROL : ~/path/usercontrol.ascx

Here was some standard functionality that I had not known about allowing me to drop usercontrols in to the generated pages. As I had already put my funky calendar controls in usercontrols so it was pretty straight forward to pop them in my new Blogengine.net pages. Then I had exactly what I wanted and a very happy customer (I had over delivered and undercharge.... again!)

Here is a link to the services usercontrol embedded in a page: http://www.petersfieldparish.org.uk/page/Services.aspx

The homepage has a few usercontrols, a couple pulling the last 3 articles contributed through as a summary and one showing today's services.

I then got my graphic designer to knock up a new theme for my company's, See The Link, website and very quickly I had a great looking website which is easy to maintain 100% created with Blogengine.net for us as well.

It has just re-enforced my view that is this a great product!

DaveHawes.BlogEngineTweaks.zip (12.59 kb)

I was told about a new feature in C# 3.0 called extension methods back in 2007 sometime by the technical architect at my old company. He explained that Microsoft had created it to enable LINQ and that it allows you extend base classes with your own functions, which is pretty cool. But what's a useful example?

Peter (my TA) said [more] that creating a StartOfDay and EndOfDay function for the .Net DateTime class might be useful and today I needed those functions so I have decided to write them blog how I did it.

   1:  public static class ExtensionMethods   {

   2:   

   3:          public static DateTime EndOfDay(this DateTime input)

   4:          {

   5:              return new DateTime(input.Year, input.Month, input.Day, 23, 59, 59, 59);

   6:          }

   7:   

   8:          public static DateTime StartOfDay(this DateTime input)

   9:          {

  10:              return new DateTime(input.Year, input.Month, input.Day, 00, 00, 00, 00);

  11:          }

  12:    }

   1:  private static FieldBetweenPredicate AvailabilityFilter(DateTime selectedDate)   {

   2:        return new FieldBetweenPredicate(PersonAppointmentsFields.StartDate, selectedDate.StartOfDay(), selectedDate.EndOfDay());

   3:  }

Thanks to David Hayden's blog for reminding me how to write them!

http://www.davidhayden.com/blog/dave/archive/2006/11/30/ExtensionMethodsCSharp.aspx

New note 02/Jan/2010: Just found another useful extension method for randomising a list: http://www.vbforums.com/showthread.php?s=51020c99595a4e123140a61ff2b00007&t=585855

Have you ever been given a spreadsheet to import into some system or other?

Have you then tried creating a simple dataloading program to read the data, run some dataloading rules against it before loading it into the correct fields on the target system.

Have you found this has caused problems in code because:

A) Everything is a string
B) It is hard to access the correct spreadsheet columns in code be because of a lot of column index numbers being used.
C) The Excel runtime must be installed on the computer running the data loading tool.

If you have shared this pain and have not found a better solution then please read on.
[more]

The process I now use when faced with this problem is to:

1) Define a simple xml schema per excel worksheet I need to use.
2) Map the schema elements onto the excel worksheet columns
3) Save the worksheet as ‘Xml Data’ (NOT XmlSpreadsheet)
4) Generate a .Net class using Microsoft’s Xsd tool and the same schema used on the worksheet.
5) Deserialize the ‘Xml Data’ version of the worksheet into the strongly type object created in .Net
6) Now you are in the world of .Net code and visual studio to manipulate the data in a strongly typed and intuitive way.

The attached zip file has an example of this process using data from the Northwind database.

There are two folders.

‘Step 1’ contains the source spreadsheet, schema file, example of the xml,  xsd tool, batch file to run the xsd tool, and a sample of the generated class

‘Step 2’ contains a Visual Studio 2005 solution and source code that includes the Xsd generated class, a Loader class that Deserializethe Xml file into an instance of the generated class, the Winform binds the loaded data to a DataGridView. I also included an additional partial class file to demonstrate how to extend the functionality of the generated code.

StrongTypeSpreadsheet.zip (194.80 kb)