Wednesday, November 24, 2010

Visualizing data as circles in Google Maps

Datasets that include geographic or spatial data (“geodata”) are often much easier to digest on a map.  When each data point also includes a measure of size or catchment, it could be helpful to represent the point as a circle on the map, with the radius indicating its relative size.

Here’s an example showing the UK car industry, which was created using the Google Maps API (screenshot):

google_map_circles

Source (zip) – all data is fictitious; runs locally in IE

Notice that the circle radius (number of employees) and colour (status) are used to represent different aspects of each car plant.

This makes use of the latest Google Maps API (v3), which includes the ability to draw overlays.  The big winner for me was that all of this, including loading XML and looking up postcodes, could be done with pure javascript – no hosting or server-side code required.

The body of the HTML is just a single DIV element:

  1. <body>
  2.   <div id="map"></div>
  3. </body>

The rest is all javascript.  Firstly, we need to import the Google Maps and jQuery libraries – jQuery isn’t compulsory, but it makes AJAX and XML handling a lot easier.

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js"></script>
<script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script>

Loading the XML file

We can use jQuery’s .ajax() method to load the XML file, followed by calls to .find() and .each() to iterate over the elements.

$.ajax({
    type: "GET",
    url: "data.xml",
    dataType: ($.browser.msie) ? "text" : "xml",
    success: function(data) {
        var xml = toXmlDom(data);
            
        // Iterate over postcodes
        $(xml).find('project').each(function(){
            var postcode = $(projectXml).find('postcode').text();
            // ...
        });
    }
});

I’m loath to explain why we need the toXmlDom() function and conditional logic on $.browser.ie, but bascially IE returns plain text from the .ajax() call, so this needs to be converted to an XML DOM object before using .find().

Close your eyes for a second :-)

function toXmlDom(data) {
    if (typeof data != "string") return data;

    // Convert text to xml dom (IE hack)
    var xml = new ActiveXObject("Microsoft.XMLDOM");
    xml.async = false;
    xml.loadXML(data);
    return xml;
}

Displaying the map

This just involves replacing the DIV in our HTML body with a “map” object.

var mapCenter = new google.maps.LatLng(52, -2);
var map = new google.maps.Map(document.getElementById('map'), {
    'zoom': 7,
    'center': mapCenter,
    'mapTypeId': google.maps.MapTypeId.ROADMAP
});

The tricky bit here is getting the initial latitude/longitude and zoom right – sites like www.getlatlon.com will give you a start.

Looking up a postcode

For simplicity, the XML file refers to UK postcodes.  Since the Google Maps API doesn’t include suitable geocoding support (licensing issues), we need to call another service first to get the respective Lat/Long co-ordinates.

$.ajax({
    url: 'http://www.geopostcode.org.uk/api/exact/' + postcode.replace(' ','+') + '.json',
    dataType: 'json',
    success: function(response) {
        var lat = response.wgs84.lat;
        var lon = response.wgs84.lon;
        // ...
    }
});

Notice that .ajax() method again, which simplifies the asynchronous HTTP request to www.geopostcode.corg.uk.  By default, the results of the AJAX call are cached by the browser.

Drawing a circle

This is done in two stages: create a Marker, then draw a Circle which is bound to its position (read the full API).

// Create marker
var marker = new google.maps.Marker({
  map: map,
  position: new google.maps.LatLng(53.1, -2.44),
  title: 'The armpit of Cheshire'
});

// Add circle overlay and bind to marker
var circle = new google.maps.Circle({
  map: map,
  radius: 10000,    // metres
  fillColor: '#AA0000'
});
circle.bindTo('center', marker, 'position');

The Marker and Circle are both google.maps.MVCObject instances, so their values can be tied together using bindTo().  So, for example, setting draggable:true on the Marker would mean that the Circle would follow it around the map.

I suggest you download the source code and see how it all hangs together.

Tuesday, November 9, 2010

Gmail Notes syncing missing in iOS 4.1

Rant time…

So the iOS 4.1 update seems to have fixed the speed issues that plagued the iPhone 3G since iOS 4 was introduced.  However, what I’ve only just noticed is that Apple have removed functionality from the 3G in order to do this:

The second one is an absolute PITA for me.  Apple are renowned for adding only minimal functionality to their products over long periods of time (copy-and-paste anyone?) but to remove features after already adding them – that’s much more annoying.

Add to this the fact that iOS 4.1 made me get up late due to the alarm bug it introduced, and I’m definitely not being an early adopter for the next update.  Grr.

Tuesday, August 31, 2010

VMWare black screen of death

Ever had a black screen when loading up a VMWare image?  Try booting into safe mode and deleting your display adapter from Device Manager, then restarting.  (Don’t worry, the adapter should reinstall automatically.)

Device Manager

This issue seems to appear after changing monitors – in my case, when I unplug an external monitor from my laptop.  I’m running a Windows Server 2008 guest VM on a Windows 7 host, under VMWare Player 3.1.0.

Another workaround here.

Monday, August 23, 2010

iPhone 3G running slow on iOS 4? Time to jailbreak

Update (26/10): Apple have released iOS 4.1, which pretty much fixes the speed issues on the 3G.


I’ve been slow to the iPhone jailbreaking party, but I just found the killer app for my 3G – overclocking.

Since upgrading to OS 4.0, my iPhone has been dog slow.  Apple have confirmed this is an issue and are working on a fix, but in the meantime you can do something about it have released a fix, woo.

Disclaimer: at the very least this could make your iPhone run hotter and increase battery consumption – worst case, it could brick it.  But if you think it’s worth the risk then keep reading…

To get your iPhone 3G to run at 600 MHz (normally 421 MHz), follow these instructions:

  1. Backup your iPhone through iTunes
  2. Jailbreak: open Safari on your phone, browse to www.jailbreakme.com and follow instructions [Note – iOS 4.0.2+ is not supported, so if you’ve already upgraded then restore a previous backup first]
  3. Follow instructions to overclock (I used iFile rather than SSH)

You should notice that apps like Google Maps run much faster now.  This hack also works on the 3GS.

Some more benefits of jailbreaking:

On the last point - I’m not suggesting you go and download apps like TomTom without paying for them, but there are plenty of legitimate apps that for some reason Apple haven’t approved.

And in case you were still worried…

  • Jailbreaking is completely legal (in the US at least)
  • All existing iPhone functionality will still work, including syncing with iTunes and the App Store
  • You can completely reverse the jailbreak by restoring a previous backup of your iPhone

On the downside, your phone might crash a bit more (not that I’ve noticed) and you won’t be able to get the latest official iPhone updates until they’ve been cracked by the jailbreak community.

Showing Delicious bookmarks in your Blogger sidebar

I thought this would be easy, but not so much.  Here are a few options:

1. Delicious linkrolls

Great idea, but this doesn’t seem to work with Blogger; paste the script into http://www.blogger.com/add-widget, and you just get an empty element on your blog… neat.

2. Widgetbox

There are a few widgets that claim to be Delicious feeds, but they all look pants.  And the free version requires you to have a  big fat Widgetbox advert.

3. Blogger’s “Feed” gadget

This is what I went for – it works, and it doesn’t suck too much.

Just add a gadget to your blog, select “Feed” from the list, and put in your URL (e.g. http://www.delicious.com/duncsmith).

One small thing – there seems to be a bug where the feed URL has “count=15” at the end, which doesn’t give you the latest bookmarks.  You need to change this number to match the number of items to show (max = 5).

Monday, June 7, 2010

Using T4 to generate ‘Specified’ properties for a WCF proxy

Visual Studio 2008 makes it very easy to generate a set of WCF proxy classes: just add a service reference.  Sometimes though, you’ll want to alter the behaviour of the generated classes – especially since the WCF Proxy Generator has some shortcomings.

I’m going to outline a simple way to do this using Visual Studio’s built-in code generator (T4) and a bit of XPath.

Here’s a typical problem that you might encounter with the proxy generator; let’s say our WSDL contains a PayrollNumber element:

  1. <xs:simpleType name="String1">
  2.   <xs:restriction base="xs:string">
  3.     <xs:minLength value="1" />
  4.   </xs:restriction>
  5. </xs:simpleType>
  6. ...
  7. <xs:element minOccurs="0" name="PayrollNumber" type="cc:String1"/>

Some things to note about this element:

  1. It’s a string
  2. It’s optional (minOccurs=“0”)
  3. If specified, it has a minimum length of 1

The WCF Proxy Generator will give you something like this:

  1. [Serializable, XmlType]
  2. public partial class MyClass
  3. {
  4.     [XmlElement]
  5.     public string PayrollNumber { get; set; }

If you leave this property null and call the WCF service, it will send an empty element, leading to an invalid XML schema.  Running a schema validator will give you a big fat  XMLSchemaValidationException:

The value ‘’ is invalid according to its datatype ... The actual length is less than the MinLength value.

What we actually want is for the element to be left out of the request (it's optional after all).  This is a known issue with the proxy generator, and one workaround is to manually add a ‘Specified’ property to the class, which will be picked up by the XML serializer:

  1. [XmlIgnore]
  2. public bool PayrollNumberSpecified
  3. {
  4.     get { return PayrollNumber != null; }
  5. }

Since the proxy classes are partial, we can add this property to our own partial class of the same name.  Even better, we can auto-generate these partial classes using a T4 script that parses the WSDL.

To do this, create a new file in Visual Studio called [ServiceName].tt, and paste in the following (you’ll need to change the namespace etc):

Update (30/7): download the latest version.

  1. <#@ hostspecific="true" language="C#" #>
  2. <#@ name="System.Core" #>
  3. <#@ name="System.Xml" #>
  4. <#@ namespace="System.Xml" #>
  5. <#@ namespace="System.Collections.Generic" #>
  6. <#@ namespace="System.IO" #>
  7. #pragma warning disable 1591
  8. // <autogenerated>
  9. // This code was generated by a tool. Any changes made manually will be lost
  10. // the next time this code is regenerated.
  11. // </autogenerated>
  12.    using System.Xml.Serialization;
  13.  
  14. namespace MyNamespace.MyService
  15. {<#
  16.     XmlDocument doc = new XmlDocument();
  17.     string absolutePath = Host.ResolvePath(@"..\Service References\\cf8MyService\\cf8service.wsdl");
  18.     doc.Load(absolutePath);
  19.    
  20.     XmlNamespaceManager ns = new XmlNamespaceManager(doc.NameTable);
  21.     ns.AddNamespace("xs", "http://www.w3.org/2001/XMLSchema");
  22.    
  23.     foreach(XmlNode ndClass in doc.SelectNodes(@"//xs:complexType[@name]", ns))
  24.     {
  25.         bool createdClass = false;
  26.         foreach(XmlNode ndProp in ndClass.SelectNodes(@"./xs:sequence/xs:element[starts-with(@type, 'cc:String') and @minOccurs='0' and not(@maxOccurs)]", ns))
  27.         {
  28.             if (!createdClass) {
  29. #>
  30.    
  31.     public partial class <#= ndClass.Attributes["name"].InnerText #>
  32.     {
  33. <#
  34.                 createdClass = true;
  35.             }
  36.        
  37.             string fieldName = ndProp.Attributes["name"].InnerText;
  38. #>
  39.         [XmlIgnore]
  40.         public bool <#= fieldName #>Specified { get { return !string.IsNullOrEmpty(<#= fieldName #>); } }
  41.        
  42. <#        }
  43.         if (createdClass) {
  44. #>    }
  45. <#
  46.         }   
  47.     } #>
  48. }
  49. #pragma warning restore 1591

A brief explanation:

  • Uses Host.ResolvePath to get the path of the WSDL file
  • Uses XPath to select all string elements with minOccurs=‘0’
  • Generates partial classes with ‘Specified’ properties for each corresponding element

When you hit Save, a new file called [ServiceName].cs will be generated, containing a list of partial classes.

Thursday, March 11, 2010

Multiple vsmdi files with VS 2008: the path to freedom

If you use a combination of Visual Studio 2008, MSTest and TFS, you've probably come across this problem at some point. Your solution folder is suddenly full of vsmdi files:







You may have worked out that Visual Studio generates a vsmdi file the first time you run unit tests with MSTest. This file basically stores the list of tests that will be ignored.

The problem comes when multiple developers are running tests at the same time – rather than checking out the same vsmdi file, your Visual Studio will "helpfully" create a new one, adding it to source control.

The best solution seems to be to exclude these files from source control altogether:

1) Delete all .vsmdi files from both source control and your local solution folder

2) Open the solution in Visual Studio

3) Open Test View (Test > Windows) and toggle the "enabled" property of a random test - this will force it to create a new .vsmdi file









4) In Solution Explorer, select the new vsmdi file (should be under Solution Items), right-click it and Undo Pending Changes

5) With the file still selected, click File > Source Control > Exclude [file] from Source Control

6) Check in

As long as everyone now "gets latest", you should each end up with a single writeable vsmdi.

Tuesday, February 16, 2010

Restoring notes from an iPhone backup

My iPhone got really slow the other week, and the local Apple Genius suggested a full restore to get it back to factory settings. Fair enough, it seemed to fix the problem, and after syncing with iTunes I had all my apps back, only....

No more notes.

It turns out that iTunes doesn't sync the data from the Notes app, so the only way to get them back is from an old backup. A quick Google turned up various tricks involving Grep, Python and Mac-related tools for extracting data from the iPhone backup files, but there's a much simpler way. And it works in Windows too.

1) Run Backup Extractor for iPhone (Reincubate, trial).

The trial lets you extract one SQLite database file (.db) at a time from a backup file.








Choose an iPhone backup, then select which database to restore (in this case, Library > Notes).





2) Run SQLite Data Wizard (SQLMaestro, trial).

Select the file from above, choose your output format (e.g. CSV) and the note_bodies table.












You should now have your beloved iPhone notes back, in the format of your choice.