Fil's Blog

Reactive Extensions release v1.0.10425

2011-05-03T09:20:00.002+10:00

I just ported my projects to the latest Rx. Here's the highlights:

- Namespaces are more split-out, System.Reactive now contains lots of sub-namespaces like System.Reactive.Linq, System.Reactive.Subjects

- Less assemblies - gone is System.CoreEx; everything is pretty much in System.Reactive now

- TPL 3.5 stopped; System.Interactive, System.Linq.Async, RxJS on pause

- Lots of name simplification: eg. Window*/Buffer*/Generate* are now just overloads of Window/Buffer/Generate

- Performance improvements - Subject now replaced with Subject.Synchronize, FastSubject with Subject.

- Now compatible with Async CTP SP1

- General cleanup of schedulers, new VirtualTimeScheduler and HistoricalScheduler

- Observable.CreateWithDisposable is now an overload of Observable.Create

I was still using ParallelExtensionsExtras for some extra functionality over Rx/TPL, but have now managed to remove it from my dependencies.

Lock-free multi-threading

2010-04-14T13:49:00.006+10:00

I was recently looking for a way to combine two Interlocked.CompareExchange() calls, do do something like this (atomically):

- If 0, then 1
- If 2, then 3

I looked some ways to combine two Interlocked.CompareExchange calls in sequences, but it seemed pretty impossible to ensure correctness. Then a mate referred me to Concurrent Programming on Windows. Very good - had read that one in the past.

I put together a nice generic technique in, to come up with this (some typos in the original code corrected):

public static class Concurrency
{
    public static int Atomic(ref int result, Func func)
    {
        int oldValue, newValue;
        do
        {
            oldValue = result;
            newValue = func(oldValue);
        } while (Interlocked.CompareExchange(ref result, newValue, oldValue) != oldValue);

        return oldValue;
    }
}

Which can then be used:

Concurrent.Atomic(ref val, x =>
{
    if (x == 0)
        return 1;
    if (x == 2)
        return 3;
    return x;
});

This provides a great way to combine multiple Interlocked.CompareExchange operations together in a consistent way, where there isn't a *massive* amount of contention over the result. The only down side I can see is the overhead of the delegate, which would need to be optimised out to work in a very performance intensive scenario.

out parameters and covariance

2009-12-27T22:42:00.003+11:00

I was disappointed to discover that out parameters are actually ref parameters, plus some compiler rules. That means the following does not work:

interface ITest

{

void Method2(out T t); // T must be input safe? Huh?

//void Method2(ref T t); // OK this should fail

}

(see this discussion)

I'd already reported to Microsoft as a bug. Oh well, looks like it will come back as 'by design'.

My view is that 'out' should be recognised by the CLI, and therefore not be as restricted as 'ref' is. This would no doubt require that 'out' parameters not actually pass in a value (I had assumed this was not the case, but it actually does in the debugger..).

In short, this seriously restricts the usefulness of covariance with generic interfaces in C# 4.0.

Covariance and contravariance

2009-11-03T14:08:00.005+11:00

I recently upgraded an "asynchronous observable framework" (working title: Olib) to .NET 4.0 beta 2. The reason for this was the new "reactive" pieces (IObservable<> et al.) but I ended up not using them since I needed a different interaction pattern (more of that in a future post). There wasn't anything more I wanted to use, but found a few interesting pieces which dramatically helped me:

The new ISet<> generic interface, now implemented by HashSet<>
Generics covariance and contravariance (keep reading)

Covariance and contravariance have been implemented for some time in the .NET world, in various forms (parameters, delegates etc.). This new feature makes some seemingly minor additions to generic interfaces and generic delegates (Func<> and Action<>). eg:

interface IEnumerable<out T>

This "out" keyword denotes the type as only usable as output within the interface, and therefore considered contravariant. This means that you can do:

IEnumerable<string> i = new[] { "0", "1", "2", "3" }; IEnumerable<object> o = i;

Note you cannot do anything that requires anything other than the most basic cast, eg. boxing:

IEnumerable<int> i = new[] { 0, 1, 2, 3 };
IEnumerable<object> o = i; // cannot implicitly convert

One of the best things about this (and the reason I used it for Olib) was that you can write much more powerful extension methods now, since code you write to operate on "IEnumerable" will now work with any enumerable whose items support IComparable.

For example, in Olib I'm creating Linq aggregates:

public static IAsyncObservableValue<TInner> Sum<TInner>(this IAsyncObservableSet<IAsyncObservableValue<TInner>> innerSet)

Using the right contravariant constraints (and I have to admit a major refactor to make it confirm to the rules), I can now use this extension method on types such as AsyncObservableSet<asyncobservablevalue<t>> - as the types inherit from their respective interfaces.

Tip: I found in some cases where I could not easily comply with the contravariant constraints, I split my interface into two - one "read only" (contravariant) interface and one "read/write" (invariant).

Can't go back to .NET 3.5 now.

Generic arithmetic in C#

2009-11-03T09:52:00.007+11:00

One problem that has bugged me is the lack of an IArithmetic<> or similar interface to allow generic values to be added together. I had written a MSIL-reflecting routine to statically create arithmetic methods at runtime, but have recently converted it to Linq Expressions.

The downside? No compile-time type checking. You'll blow-up at runtime if the types cannot be "arithmetized".

Here's the code:


  public static class Arithmetic<TLeft, TRight>
  {
      public static Func<TLeft, TRight, TLeft> Add = CreateBinaryOperator(Expression.Add);
      public static Func<TLeft, TRight, TLeft> Subtract = CreateBinaryOperator(Expression.Subtract);
      public static Func<TLeft, TRight, TLeft> Multiply = CreateBinaryOperator(Expression.Multiply);
      public static Func<TLeft, TRight, TLeft> Divide = CreateBinaryOperator(Expression.Divide);
      public static Func<TLeft, TRight, TLeft> Modulo = CreateBinaryOperator(Expression.Modulo);

      public static Func<TLeft, TRight, TLeft> CreateBinaryOperator(Func<Expression,Expression,BinaryExpression> op)
      {
          // declare the parameters
          var paramA = Expression.Parameter(typeof(TLeft), "a");
          var paramB = Expression.Parameter(typeof(TRight), "b");
          // add the parameters together
          BinaryExpression body = op(paramA, paramB);
          // compile it
          return Expression.Lambda<Func<TLeft,TRight,TLeft>>(body, paramA, paramB).Compile();
      }
  }

Continuous / Observable / Linq / to Events

2009-09-24T20:06:00.004+10:00

I have spent the last couple of weeks researching the above area, especially in relation to an observable C# library I built a couple of years ago. A lot has happened since, but the event that sticks out the most (and caused me to re-investigate) was the Reactive (Rx) Framework that leaked out with the Silverlight Toolkit. This post has a handy guide to moving Rx to the desktop .NET platform, from the Silverlight core. Most research involved watching hours of (mostly repetitive) video, but it all got through in the end.

Whilst the theoretical framework behind the monad/monoid/"dual of IEnumerable" is pretty interesting, I cant help but be left with the feeling that something's been missed here. The whole thing really did not make any sense to me (in relation to creating an observable/reactive framework) until the following clicked: this is not about collections.

I generally use the IEnumerable construct day-to-day to navigate collections, of one form or another. I had mistakenly thought that the mathematical dual of IEnumerable would "reverse" it's behaviour in some kind of real-world scenario. Here I was thinking some collection/enumerable object that you could subscribe to. It turns it it is, of a kind, except the dimension is totally different. Whilst there is a spatial data dimension in collections I would use IEnumerable; the dimension of IObservable is time. IObservable allows you to observe an individual value over time in an event-driven form, whilst IEnumerable allows you to navigate your own way along a collection of items.

If I had thought of IEnumerable being used to extract the evolving changes of a discrete value over time, then maybe IObservable would have made perfect sense; but the fact is I saw IEnumerable being useful to navigate over a collection of completely separate items/values/objects.

Now this has all clicked into place, I need to apply my observable library which is more focused on collections. I observable collections, where the collection members themselves are changing and can be queried. From what I've seen so far, Rx is more focused on events/discrete values which is a different thing from handling inserts/changes/deletes to a set.

My view is the Reactive Framework is totally going to change the way apps are developed (for me at least anyway) - just like Linq in general has already.

PLINQO - strange name, nice concept

2009-09-18T07:30:00.003+10:00

I've been writing a lot of code around Linq to SQL, and are getting concerned about the future support etc. from Microsoft. Competition with Linq to Entities (I only need 1:1 mapping so Linq to SQL hits the sweet spot for me), means less focus on this tool. For starters there should be synchronisation of database changes, support for enums, etc.

PLINQO is built on the CodeSmith Tools, which is a template-driven code generator. The nice thing about that is you can get in and change/create templates, rather than take the hard-coded SQLMETAL view of the world. Nice!

Regards, Fil.

Using Sierra Wireless USB 306 modem on Windows Server 2008

2009-04-01T21:58:00.003+11:00

I recently started using Windows Server 2008 as a workstation platform on my Macbook Pro - yes the whole OS X thing was good, except for developing Windows software. Winding back to XP wasn't very appealing given the server features, terminal services & RAM restrictions, something that 2008 does not suffer from.

To my suprise the new Telstra/Next-G 21 meg USB modem, did not support Winodws Server 2008. How rude. I contacted Telstra and Sierra Wireless: Telstra had no idea what Windows Server 2008 even was, and Sierra Wireless simply stated they only supported Windows 2000, XP and Vista. So much for Vista drivers being compatible with 2008.

I investigated the various .INF hacks around for different devices - I used one to use Bluetooth on Windows Server 2008. There are often filters in the .INF's to tell it to only work on certain operating systems. Widening these filters often works, but there is the risk it will not.

Anyway, it turned out the solution in this case was a lot simpler, and much less risky and error-prone than editing .INF drivers. Here's the process:

Insert the device and install it as per normal (it takes a few minutes); you'll find that it will say that it cannot find a driver for a bunch of devices near the end (it's actually all the one physical device).
Choose "Locate and install driver software (recommended)"
Choose "I don't have the disc. Show me other options."
Choose "Browse my computer for driver software (advanced)"
Point it to the path: C:\Program Files\Telstra\Telstra Turbo Connection Manager\DriverInstaller\Drivers\WinVista\32bit, and click "Next". Presumably the 64 bit Vista driver will work fine with 64-bit 2008.
The device will install, click "Close"
Repeat the above process of locating drivers as many times as it appears (8 or so); this will take some time to complete and you'll get the feeling of Ground Hog Day - but it does end.
It's ready to use!

Migrating from Outlook to GMail

2009-03-12T18:38:00.003+11:00

I made the move recently from Exchange/Outlook to Google Apps for Domains (GMail on your own domain name). The reason I did this was lugging my 4GB of email history around was making Outlook a real hog on my laptop/PC and was becoming more hungry/unreliable as time went on. It does not seem to scale well with increasing database size, and takes a lot of resources.

Enter GMail - no load on my PC at all, fully portable, and to seal the deal - an "offline" function (using Google Gears) to enable me to browse/write emails while disconnected (it does happen).

In all the experience was pretty positive, with the exception of migrating the email history. The main task required is to change the DNS MX entries to point to Google - plenty is written on that and Google have pretty good instructions on this.

Migrating email history is another story - there is a plethora of options here: multiple upload channels (POP3, IMAP pull, IMAP push, migration API). The latter is the most serious, and scalable option, but only works with the Premier (paid) verison of Google Apps.

The best option I found was to attach the new GMail account as a IMAP account to my Outlook, and copy the emails from a .PST archive. I found that this was far more reliable than copying directly from an Exchange account (.OST cache archive) - not sure why but copying from Exchange was very unreliable. Even then I was unable to copy it all over in one hit - but one year at a time did work. Doing this via IMAP gives you good control over what-goes-where in your new GMail account.

To migrate a significant number of email accounts I'd certainly suggest getting the Premier Edition of Google Apps, and go for a migration product (there are heaps around) that will no doubt do this much better and faster.

Regards, Fil.

Using strings within unions in P/Invoke

2008-04-23T16:13:00.005+10:00

I googled for hours finding this solution. If you are trying to use strings within unions with P/Invoke, here is the answer. This is not possible normally since string is a reference type, and these can't be mixed in unions (confuses the garbage collector)

Use the unsafe "fixed" array modifier:

union blah
{
int i;
char[2] s;
}

[StructLayoutAttribute(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
unsafe public struct Blah
{
[FieldOffsetAttribute(0)]
public int i;

[FieldOffsetAttribute(0)]
public fixed char s[2];
}

Hauppauge 1100 with Vista - solution cannot find tuner card problem

2008-02-10T11:16:00.000+11:00

I encountered the common "I cant get Vista Media Center to recognise my 1100 tuner card" that noone seems to be able to solve when reinstalling Vista. Found the solution (probably what I must have done last time):

Use HCWCLEAR.EXE to remove any existing drivers
Download the raw drivers from the UK site: http://www.hauppauge.co.uk/pages/support/support_driversonly.html
Manually install the driver: skip the automatic process of finding a driver (the standard MS ones don't work). Then go find the "unknown" devices in Device Manager: right click and choose "Update driver"

Great backup solution

2008-01-13T21:48:00.001+11:00

I have signed up a whole lot of family and friends to a great service that makes the backup headache a thing of the past: Mozy (owned by EMC).

Mozy uploads your files on a daily basis to Mozy's servers. If you need a file back, you can download it (or get them to FedEx a DVD for larger emergencies), plus you can choose the date on which you want the snapshot to be taken. You can even access your Mozy store via a virtual drive on your computer, which is really handy. Mozy only uploads changes to your data, and as such is really efficient, nightly while you're not using your PC or broadband connection. You'll hardly notice it after the initial upload.

The best thing is that Mozy give away for FREE personal accounts that hold up to 2GB! That's plenty for the average bear. It's a bit hard to find (they must prefer you to pay) but register here. Make sure you use the referal code "6S8MP3" when signing up in order to get an extra 256MB of space (at time of writing).

I personally subscribe to the unlimited plan (USD5/month), and have about 14GB of data - 95% of which is original digital camera images. No need for physical off-site backups any more! (except for the home videos) For the record it took about a week to get the initial snapshot of data uploaded to Mozy's servers. Since then it has been lightning fast!

Go ahead and solve your personal backup problems once and for all! And if you're like me, and your family/friends technology problems inevitably come back to me your problem, buy yourself an easy way out...

(Disclosure: Mozy didn't give me anything except peace-of-mind!)

Using Apple remote in Boot Camp

2008-01-13T21:42:00.000+11:00

When I unpacked my Mac, I thought "what do I do with this?" - I have to do something!

I found a great use for it, using a utility called EventGhost. It's an open source automation tool for Windows - nothing to do with Boot Camp directly. Essentially it lets you set up triggers, and create actions that run in response to those events.

One of the cool triggers it supports is Apple Remote buttons! I personally use the left/right button on the remote to fire left/right key-presses. Good for PowerPoint presentations. By default I tihnk left/right do prev/next track (used by Windows Media).

I now think EventGhost is a must-have utility for all those general automation tasks, like switching things on/off when sleeping or hybernating, undocking etc. Very easy to use, it does not require any script or programming skills - very user friendly after you get over the dawnting list of things it can do..

Video driver for Boot Camp

2008-01-13T21:31:00.000+11:00

I had issues with setting up an external monitor with any decent resolution (1024 x 768). Did some digging and found a whole world of drivers out there.

My MacBook Pro uses a nVIDIA GeForce 8600M GT. M is for mobile.

It turns out that the mobile edition of nVIDIA display technology is customised by the OEM (Apple in this case), and as such drivers are only released via them. That means when nVIDIA release a new driver, it has to go to Apple for QA, package, release. What are the chances of getting the latest driver? None. Also, what if Apple have not done something properly - eg. setup for external monitors?

So, the solution is to grab the standard nVIDIA driver, and overlay a "modded INF" to open up the functionality of the driver. This allows it to work on mobile video hardware, and opens up greater functionality such as external monitor resolutions :)

For me, this is a must for anyone running Boot Camp...

Experience with Boot Camp (Windows on Mac)

2008-01-13T21:23:00.000+11:00

I've been using Boot Camp for six months or so now, after buying the wife a 17" MacBook Pro. I've now ventured into a 15" myself. Why?

Apple hardware is very high spec (fast)
Industrial design is of a very high standard
It looks great

I found Bootcamp works pretty well with Vista on it, in all. In fact probably the best machine I've run Vista on! The only annoyance for a Windoze boy such as me was:

Getting used to the keyboard, but not a big deal. Keyboard is a lot more compact than the PC ones generally - just like a PC based laptop, just, different and a little quirky..
Runs VERY hot. The top surface is uncomfortable when it is doing a lot of work. Similarly when sitting on your lap.
Not massively mobile: MacBook Pro's are not known to be a featherweight, but you know that in advance. My inclination is towards power anyway...
Video drivers are pretty limited / old? More on this later..

For the record I've booted OS X twice - once for each, in order to then run Boot Camp. I have no time for OS X - so shoot me.

Great speaker sound dampning enclosures

2008-01-02T12:14:00.000+11:00

I found these at CEDIA07 in QLD: DYNAMAT wrap-arounds for in-wall speakers, plus boxes to sit over the top of in-ceiling speakers. A whole bunch of other materials for pre-gyprock preparation too. The speaker boxes are quite incredible in preventing noise escaping above into the ceiling space (and therefore throughout the house)..

Supplied in Australia by AudioXtra.

Heap Corruption

2007-09-13T09:40:00.001+10:00

I did not find much mention of the following error that was plagueing me:

Critical error detected c0000374

This is a generic heap corruption error, but the trick is that it is COM's heap that it stores BSTR's in that was the issue - not the traditional CRT heap that all the references will lead you towards.

There seemed to always be corrupt BSTR's being processed at the time. The answer was that I was SysFreeString'ing a BSTR and then keeping it around as if it were still a valid string.

Vantage InFusion: Under the software hood

2007-07-08T22:27:00.000+10:00

I’ve been doing a lot of work in extending the InFusion product, since even though it is the most advanced home automation control system around – as a programmer I find it extremely limiting. My extensions are intended to make it a little less so.

I have written a software package called ‘VertigoTools’ which consists of a software installer that takes a bunch of InFusion Design Center procedures and third party objects; and installs them into an existing Design Center installation.

It performs this by modifying a bunch of system files used, and adding some new ones as well. Don’t worry – any additions to the system are clearly marked in the XML data as being from the VertigoTools so there is no confusion.

The following functionality is added:

New Data type: Decimal; allows variables to store fixed-point values
New Objects:
o AnalogueInput: represents an analogue input channel
o ADAM-6017: driver for an 8-channel analogue input station, that connects via TCP/IP.
New Procedures:
o Three Speed Fan Control: controls a multi-speed fan that have a three-leg fan capacitor (eg. Hunter Pacific Concept2) . LED output is provided to indicate Off/Low/Med/High with light intensity.
o Light and Fan Motion Sensor: a combination of light-and-fan, and motion sensor functionality into a single procedure! The functionality provided by this procedure cannot be matched by a custom task.
o Set String From Variable: sets a LCD status text field to the value contained in a variable. Allows for Decimal data type, as well as optional formatting features such as trailing-zero truncation, thousands separators, and a special invalid value.

This is only the beginning, there is plenty of scope for improvement/extension. With the upcoming release of Design Center 2.0, who knows what will be possible.

Coming up: a new programming model for home/building automation.

Parallels and Boot Camp

2007-07-08T22:24:00.000+10:00

I did something unthinkable. I got a Mac.

At the risk of offending those other (other?) faith, I find it a great machine to run Vista. Now I have got everyone offside.

I tried running Parallels and Boot Camp as options to running Vista on my Mac.

Parallels is a very smart virtualisation product, that emulates the notion of integrating Windows® windows into the OS X® desktop (“coherence”). You get the giddying feeling of world unification to see the two worlds unite – but there is some smoke-and-mirrors involved. The Vista desktop runs “off screen” and targeted sections of the desktop (relating to your apps) are replicated onto the OS X desktop. Very cool – but you do see some visual artifacts when you drag windows around. I don't think virtualisation can get much better.

Boot Camp is a set of Windows® drivers for Mac hardware. Enough said, that’s all there is really. I have not booted OS X since I installed Vista. The hardware performs really well for Vista (latest MacBook Pro), so no complaints there. The only issue is the keyboard layour – oh and the lack of PCMCIA slots.

My pick: Boot Camp, unless you really are confused and use both OS’s all day long. Parallels is cute, but for power users any form of virtualisation is just a “go slow” layer.

Got to hand it to Apple – they design really well. Just a pity they aren’t in the software business.

For the record, I still have OS X on there for the future if I discover something I need it for (I’m told video editing is much better).

Tip: I use a Wireless Mighty Mouse, when bringing the MacBook Pro back from slumber I need to revive the mouse. This is done by pressing the ‘nipple’ (you know what I mean..) a few times. Or else you can delete and re-associate the Bluetooth® device if you like to do things the hard way.

SQL Server performance on SUSER_SNAME() joins

2007-06-20T13:06:00.000+10:00

I came across a painful performance issue with merge replication on SQL 2000. OK - it is a temperamental beast, but the performance we were seeing was so bad - way too bad.

After digging through the system generated views, the root cause came down to using SUSER_SNAME() in WHERE clauses as part of table joins. Replacing the function with a constant literal solved the problem.

Take the following system-generated replication view:

ALTER VIEW [Perm_VIEW] AS
SELECT DB.*
FROM DB JOIN Perm ON DB.DBID=Perm.DBID
WHERE Perm.UserName=SUSER_SNAME() and ({ fn ISPALUSER('1A9E41A4-55B9-0E55-ACD6-4CC0B419BDE0') } = 1)

.. now, SUSER_SNAME() you would think would operate as if it were a constant string. I mean, it can't change during execution or anything - so you wouldn't go re-evaluating it for every row. That's exactly what it does - as if it were constantly changing the return value.

The problem is that SUSER_SNAME() is desigated as "non-deterministic" by SQL Server. It kind of is, and kind of isn't. What the SQL Server team were thinking that it is non-deterministic between users, but it is obviously deterministic for a single user (and therefore query execution).

SUSER_SNAME() is deemed as a fairly unreliable "unknown quantity" that it must guess the reliability of, the opposite of what it is - a very reliable item to query on.

So, the standard recommendation is to put the result in a variable, and then use that. Duh. Works fine if you are in a stored proc, but less practical in a VIEW. After much research, here is the simplest/most elegant solution:

ALTER VIEW [Perm_VIEW] AS
SELECT DB.*
FROM DB JOIN Perm ON DB.DBID=Perm.DBID
WHERE Perm.UserName=(SELECT SUSER_SNAME()) and ({ fn ISPALUSER('1A9E41A4-55B9-0E55-ACD6-4CC0B419BDE0') } = 1)

This gives SQL help by forcing it to use only a single value returned by the function, that it does not need to re-evaluate. The impact on this performance improvement will vary, but I saw betwen 6-17x improvements in performance. In particular eliminating Index/Table Scans, and making it able to use Seeks instead.

Vantage InFusion Design Center software

2006-12-06T13:50:00.000+11:00

It has been 48 hours since I got a copy of Vantage Design Center software. My house recently completed construction, and All About A/V arranged for the install of a InFusion-based home automation system. To give you an idea of the scope of this system, it contains x dimmer and y relay circuits under InFusion control. It is the first installation of InFusion in Australia.

As a programmer by trade, I was naturally keen (insisting?) on getting access to the design software. I was not disappointed.

My background is a software architect and programmer of more corporate enterprise software (Windows / .NET), instead of hardware-based stand-alone systems such as home automation. In my blog I want to capture all the things I learn along the way, from the perspective of an application software developer.

First impressions as such that InFusion is a very stable platform – I have not seen any evidence of instability or unreliability – which is a must for home automation. I am not sure what operating system InFusion uses, but it looks a little Linux-esque. I haven’t looked into this any further – but I must say I am pretty happy it does not run Wintel. Ouch.

In the surface InFusion is a little black box (blue actually) that does stuff. It has no UI of any significance, but is a control system that does things in response to external events (buttons, switches, movement detectors, temperature, light, clock etc.). It has USB and Ethernet interfaces. DHCP or static IP – no matter. For the record, I am waiting until the DIN controller is released, until then I am using a "borrowed" controller.

What the Design Center gives you is the ability to program the InFusion Controller to do some sophisticated stuff. While it is pretty light in terms of pure programming per se, it hits a broad appeal by giving people who are not veteran programmers the ability to create powerful control scenarios. There will always be pressure to dumb-down or smarten up a tool such as this, but I think Vantage have done a pretty good job at getting decent spread of target skills in the tool they have provided. Being .NET based, using lots of XML under the covers, with a modern UI helps as well. I think the team at Vantage did a great job at representing really well the power of InFusion.

Having said all this, there will be lot of trial and error in getting this all to work properly mainly because there is such a broad spectrum of Design Center users. On the one hand I think the task is a hard one for non-programmers since people do not naturally think logically/procedurally, but as a programmer it is taking me a while to find out exactly where the level of sophistication lies in the design. Here is what I have learned:

Terms

Here are the really important terms in the context of InFusion/Design Center:

Loads: lights, or anything that consumes power
Procedures: pre-defined pieces of functionality
Tasks: created in Design Center to tie together procedures with conditional logic.
Areas: a fairly arbitrary hierarchy of buildings/floors/rooms etc. used to organise all your loads

Tasks are pretty simple from a programmer's viewpoint. Here are the key points I learned early on which I needed before I could understand how it would work:

Blocking

Task steps are blocking - take the following task steps:

Contrary to popular belief, the Motion Sensor procedure above will prevent execution of any subsequent steps until it has completed. That means in the case of Motion Sensor that the light has come on, the Off Delay time has passed, and the load has been turned off. It is not the case that subsequent steps in the task will continue, along with the Motion Sensor in parallel. So that means the log entry 'Test 2' happens after the light goes out. Initially I had been told that they were executing in parallel (in layman's terms), which scared the heck out of me, since the implications of that are.. well.. dangerous!

To take an old favourite, I have seen a sensor/light/fan task designed as below. This task is to turn on a fan according to a sensor, but delay the fan until the light has been on for a period of time (eg. 25 seconds), then run the fan for another period (eg. 1 min) beyond when the light is switched off.

Taking the steps one at a time (in a blocking manner) we can see the above does not do what is expected:

The light comes on, and turns off according to the Motion Sensor procedure (ie. someone has left the room)
Nothing. The if statement tests if the light is off - it "always" is at this point since the Motion Sensor procedure would have already turned it out. In any case there is no code to run as a consequence as there is no indented steps after this if statement.
Wait 25 seconds
Turn on the fan
Nothing. The if statement tests if the light is off - it still is. This time there are steps indented below, so they will run.
Wait 1 minute
Turn off fan

So instead of the intended behaviour, you get instead: light on, light off, delay, fan on, delay, fan off. The confusion here was the belief that the Motion Sensor procedure was not blocking, so the delays and fan load changes would happen while the light was on.

No flow control

One annoyance is the lack of real flow control, beyond basic 'if' statements in tasks. The lack of loops/while statements etc. keeps it simple for the masses - but there are now many things I think I need to dig in deeper for. On the up-side, there are no infinite loops! Simple ones at least anyway...

Enhancements/Bugs?

Don’t provide warnings for task steps that are disabled
Associate dry contact inputs with Areas – it is less convenient to find these things in the dimmer, rather than the location of the PIR etc.
Provide some descriptions for the properties of Dry Contacts. One of the few areas where good descriptions are not found
Provide a warning for task steps which are redundant - eg. an if statement with no indented target steps
View the underlying .amx scripts generated by tasks
Write custom .amx/.sma procedures, and be able to construct 3rd party libraries of such

Questions I Have

Can I attach tasks to load events?
Can I write my own procedures in .SMA scripts as the software currently stands? ie. create now own .SMA file and edit the LibraryInformation.xml to link it in. Will this automatically be deployed to my Controller by the software?

PDF Goes on a Diet

2006-12-06T11:57:00.000+11:00

I have been doing a lot of work on PDF formats – something which I have taken for granted up until this stage.

One point of interest is the development of JBIG2 – an image compression standard, which focuses on scanned text images. Since this is the type of content I am dealing with, it is a perfect match.

Essentially, JBIG2 aims to replace TIFF (in its various forms) for fax-style content. It works by recognising characters in the pages, and representing that, in addition to non-character based content. Essentially this gets a PDF with scanned pages in it, down to a similar size to as if it had been natively produced in a PDF – in the order of 10x compression. In effect it compresses a scanned page in PDF form down to the size of the raw document (eg. Word .DOC) used to produce it.

Nice.

The best commercial product to do all this I found was CVista PdfCompressor - very flexible licensing, and produced the best results in real-world testing for me.

Nine v IceTV court case

2006-10-19T10:25:00.000+10:00

With the current movements in the media industry, most have not yet heard of a low profile case working its way through the Federal Court in Australia. It is between the nations most aggressive TV network - Channel Nine - and a small EPG provider - IceTV.

You can check out more on the background to this case here, but I love the quote from the chairman of IceTV:

"[It is] like Australia Post suing Microsoft for inventing the email,"

I think this quote (which probably came from founder Peter Vogel) really sums up the case. Nine are suing IceTV on the basis that using the program guide, describing what TV shows are playing when on Nine, infringes their copyright - since they publish the guide. That's it. IceTV claim they have a part-automated, part-journalism process that creates the guide without copying the guide published by Nine.

I think there are some great areas for IceTV to move in the future:

Publishing an after-the-case guide, eg. providing a summary of TV shows that have already aired. eg. I am looking at ABC "7:30 report" in my EPG, not knowing if I really want to watch it. I notice in the review that there was a last-minute interview with a leader from North Korea about their nuclear testing, that I am interested in, so I decide to watch it.
Providing advertising that people actually want to watch; specific targeted advertising. I do find using an EPG, that I sometimes do rewind to specifically watch an ad I am interested in. 95% however, I am not.
Real-time EPG, taking into consideration the usual game playing that free-to-air channels use to make sure you cannot transition easily from one channel to another in prime time. If you haven't noticed, channels on occasion begin and end at different times to those advertised. A real-time EPG could work around this by actually having people watching and adjusting as it happens.

If Nine succeeds by bleeding IceTV of its cash, they will have been victorious in plugging the dyke for a little longer, and keeping this country back from the evolution of public media. No wonder viewership is falling.

My take on it is a classic case of a 1980's media organisation, which has been used to being propped up by government protection its whole life, is now under threat from technology. While Nine was happy to collaborate with Microsoft on the ninemsn venture, Microsoft were interestingly unable to launch a free EPG for their products, as they have done in the US.

Keep your friends close, and your enemies closer.

Dynamic/Streaming XML Data

2006-06-09T22:31:00.000+10:00

Up until now XML data has been fairly static and boring. Without wanting to get bogged down in too much philosophical naval-gazing on the subject, for me an oddity in the evolution of XML was the creation of MDDL (Market Data Definition Language) – as financial market data was the area I was working in at the time. The most important and complex systems with market data are ones that require high performance, and dynamic/streaming updates. XML is incapable of delivering this. Of course MDDL is useful for more consumer-oriented "snap-shot stock quote" solutions, but for me the more important applications were being neglected.

To retrofit XML into to dynamic/streaming data world (see my definitions page on the confusion over these terms), extra smarts are required. We just don’t have the pieces necessary right now to make this work. Here's a great presentation from FISD on the scope of the problem, and a bit of a review of the available solutions. One solution is the approach taken by fisdMessage, which essentially works by updating values within an existing XML document (such as MDDL). No structural changes to the XML document are possible with updates (the theory being that this is not desired anyway, as this type of data is fairly homogenised).

While right now it is true that financial data is homogenised to the point that the structure of the data does not change in a fluid matter; but possibly this is a short-term (and dare I say US-centric) view? It is my feeling that the reason the data is currently quite static in structure is because of technology limitations, rather than preference. With current day non-XML technology it is far more difficult to achieve dynamic structural changes – so they are rarely implemented.

XML is very good at handling structure; my belief is that we want to preserve this capability in as many ways as we can. We do not want to throw this away, unless we have no other choice.

In thinking about how to implement dynamic/streaming data using XML led me to design a complementary technology that allowed the familiar and open XML model to be used for dynamic applications. Not financial prices, but all data. The scope of my investigation was:

XML centric: all data content is assumed to be available in, and consumed in, XML form.
Large: the underlying content contains medium to large datasets (1,000's of items).
Fluid: the content is changing regularly, both in content and structure.
Timely: consumers of content are interested in seeing the very latest information.
Resilient: bandwidth varies in availability, and should handle periods of low availability.
Efficient: able to be implemented on the client, with a minimal footprint.

The above could apply equally to: online forums, sport scores, news websites.

A Review of XML Technology

A series of technologies come to mind when thinking of handling changing XML documents: XyDelta, DeltaXML™, XML Diff Language (Microsoft) to name a few. However these were trying to solve a different problem set. They were focused on more static document-based content, where updates were more infrequent, and were not going to be distributed in a time-critical manner. They were more about efficiency, reducing the amount of data traffic when doing substantial chunky mods to existing documents. In addition, they were generally produced by comparing two versions of a document, rather than by the atomic events that caused their changes.

To be continued…

VirtualArray: powerful data structure for large, fluid datasets

2006-06-07T16:21:00.000+10:00

On my investigations into using XML for large datasets of rapidly changing data, I came across a performance issue to do with XML DOM implementations. If you want to find a given node in the node-list (ie. find its index) there is no scalable way to go since the DOM works via linked-lists. You have to “walk the DOM” in order to find the node you are looking for; keeping count of the number of siblings you step over in the process.

Because MotionSet™ works in an event-based manner, a bottleneck occurs when working out the position of a given change. This task requires that the node’s index be determined, so it can be integrated into the existing delta. For example:

<root>
<data1/>
<data2/>
<data2A/> <== this is going to be deleted
<data3/>
</root>

An event is fired deleting the ‘data2A’ element, causing MotionSet™ to determine the index of the new node. In this case the zero-based index is 2. This leads to a delta from the parent node "root" of:

<skip count="2"/>
<delete/>

At the moment, the only way to determine the index of a node is to do a brute-force linear search through all the children. Fine for reasonably sized datasets, but not good news for anything in the order of 1,000's of items.

What is needed is a data structure, and relevant processes, that satisfy the following requirements:

An ordered array of items: ie. order is important, and must be preserved. For example, it cannot be sorted to aid in searching.
Items can be obtained by index, as is normally the case in arrays
Index can be obtained by searching by item
Items are regularly being inserted, changed and deleted from the array

The classic approach would be to use a standard array, and build a hash-table (index) that converted the item value to an index value. The disadvantage with this is that, on average, 50% of the indexes will change in the case of an insert or delete. This means the hash-table needs to be regenerated after each insert or delete, with the inevitable performance penalty.

What emerged (dubbed "VirtualArray") is an array that satisfies all the above.

VirtualArray consists of two data structures:

A modified form of binary tree: this is used to store the data items in a sorted manner, sorted on the index of each item in the VirtualArray. This can be any form of tree-like structure, but personally I like red-black tree.

A hash-table to provide access to a given tree node (see above), by item in the VirtualArray. This can be a simple sorted list in hash order, using binary-search.

The tree is modified in a way that assists in adjusting the indexes of items subsequent to deleted/inserted items in the VirtualArray. This is performed by storing relative index values at each node, rather than absolute ones. To calculate an actual index for a node, all the ancestor nodes in the tree are added together. By calculating the index in this fashion, the number of nodes that need to have their relative index adjusted after an insert/delete is very limited. The down-side is that index values are not readily available, but must be calculated at each point – but given the characteristics of appropriate tree-like structures, this is still very scalable.

So what does this all look like in reality? I implemented a series of tasks, using a set of different storage techniques:

Tasks:

Look up a value by (random) index
Look up the index for this value
Remove an item by (random) index
Insert a new item into a (random) index
Update an existing item by (random) index

Granted, the steps are skewed to the particular requirements we are interested in, so it comes as no surprise the performance is so bad for the non-VirtualArray techniques:

Techniques: (.NET 2.0 types)

List<XmlNode> (simple array)
LinkedList<XmlNode> (simple linked list)
XmlNodeList
VirtualArray<XmlNode>
XLinq (coming soon!)

By measuring the average total time taken to perform the above tasks, using each technique, the following was the result:

Note that the X (size of array) and Y (time taken) are both log scales. You can see that the traditional techniques are roughly linear in performance (as expected), but they have better performance than VirtualArray for smaller arrays. As expected, XmlNode does not perform well with these tasks.

To put this in perspective for those who don't get having a log scale on both axes, the log scale has been removed from both axes below:

The sharp improvements in performance could be related to memory swapping, as they almost disappeared when I performed each technique's test in isolation. Below is a test comprising of only List<XmlNode> and VirtualArray<XmlNode>:

.. and with the log scales re-applied:

From this chart, we can see that 1,000 items is where VirtualArray starts to outperform a simple array. From this, VirtualArray has been enhanced to use a simple array until the number of items reaches a set threshold (1,200). At this point the simple array data is copied into the tree/hash-table structures. If the number of items then falls below another threshold (600), the data is copied back into a simple array. This means that VirtualArray does not sacrifice performance in the lower end of the array sizes.

What does this mean for the XML DOM? Well, in my view - not much for the current XML space.

In order for you to believe that that there is a problem in need of a fix, you need to accept the following:

DOM collections (XmlNodeList) sizes in the 1,000's are a practical performance problem (most don’t go anywhere near this size)
Finding the index of a given node amongst its siblings is something you need to do. MotionSet™ needs this operation, but it's hardly a top-10 operation.

This investigation is taking XML/DOM into a space where it is not currently: large datasets of highly fluid, and constantly changing in-memory information. XML/DOM is currently more at home with medium-size static documents, and either generated on-demand or kept on permanent store.

VirtualArray provides the necessary performance boost to handle large dynamically changing datasets.

Note: VirtualArray is patent pending technology.

Patent application

2006-06-07T15:52:00.000+10:00

A patent application has now been filed, convering a whole bunch of stuff I have been working on in the SuperSet space over the past months, but have been unable to talk/blog about. Time to lift the floodgates!

For those of you that are interested in reading documents that look like they were written on a typewriter, you can view the specification. Normally patent specifications don't get published for as long as they can get away with (at least 18 months), since people want to keep it secret, but not me!

Please contact me if you wish to use the processes described, most of which I'll be posting about in this blog over the coming days. Of course my desire is to license the technology to others, and I plan to issue no-cost licenses to non-commercial projects. For what it is worth, this is not about patent troll tactics (this is our first patent application) – but instead having some form of protection over our technologies that copyright simply does not provide. For the record, we are building genuine commercial products from these technologies - this is not a plan to tax the innovative efforts of others.

Empty XML Nodes

2006-06-07T15:49:00.000+10:00

One annoyance I had with the DOM is that it did not collapse empty ones. That is:

<root>
<a/>
</root>

When 'a' was deleted became:

<root>
</root>

.. instead of just "<root/>".

You may ask – "why do you care" – and that's a great question. In my case, I was comparing documents to see if they had changed back to where they started. If you were to compare a textual representation of the XML, this would fail since the "<root/>" != "<root></root>". Once great suggestion was from Michael Champion (MS) who suggested comparing Canonical XML. Why didn't I think of that?

Anyway.. I wrote the following class to split out this optimization, since you need to be careful when you want to apply it. You use it by doing a:

EmptyNodeOptimizer opt = new EmptyNodeOptimizer(xmlDocument);

public class EmptyNodeOptimizer
{
private XmlNodeChangedEventHandler _xmlNodeRemovedEvent;
private XmlDocument _xmlDocument;

public EmptyNodeOptimizer(XmlDocument doc)
{
_xmlNodeRemovedEvent = new XmlNodeChangedEventHandler(XmlNodeRemovedEvent);
Attach(doc);
}

public void Attach(XmlDocument doc)
{
// attach to document to optimise empty elements
_xmlDocument = doc;
_xmlDocument.NodeRemoved += _xmlNodeRemovedEvent;
}

public void Detach()
{
if (_xmlDocument != null)
{
_xmlDocument.NodeRemoved -= _xmlNodeRemovedEvent;
_xmlDocument = null;
}
}

// Performs a cleanup of parent element when the last child is deleted
private void XmlNodeRemovedEvent(Object src, XmlNodeChangedEventArgs args)
{
// does parent element now have no children?
if (args.OldParent != null &&
args.OldParent.NodeType == XmlNodeType.Element &&
!args.OldParent.HasChildNodes)
{
// make sure it gets marked as empty
(args.OldParent as XmlElement).IsEmpty = true;
}
}
}

HTTP Resumability / Partial Downloads with ASP.NET

2006-05-15T13:43:00.001+10:00

I have been looking for a way to support partial downloads (as IIS has done since 4.0 for files) with ASP.NET. Not an easy task since IIS must execute an ASPX page in order to get the content, it can’t just arbitrarily obtain subsections of the output as is required for range requests.

The best solution I have seen (in the ASP.NET world) has been to implement a custom HttpHandler; this assumes you are getting your data from the file-system which can be randomly accessed. This provides a way to support download resumability on large files for which access is controlled by ASP.NET.

http://www.vbip.com/winsock/winsock_http_08_01.asp

(Health warning: code is written in VB)

ASP.NET Data Binding

2006-05-15T13:43:00.000+10:00

Data updates fail in ASP.NET

Problem: updates in a data binding scenario are not working – no errors, the data does not go through. On inspection (by trap the inserting event) the "Original_[key]" column has no value set.

Solution: with your data viewing control (eg. FormView) set the following attribute DataKeyNames="[KeyColumn1],[KeyColumn2]". For some reason these don’t come through (for me anyway) automatically when using the designer.

ASP.NET Development

I am into building maintainable systems. Especially early in the dev cycle I am not into building mountains of repetitious code and stored procs. While they are great to have, changes inevitably cause you to re-work this early development anyway, no matter how hard you try to get the stars aligned before you start. At least I’ve never seen it work that way. This re-work is a significant cost that should be avoided if at all possible.

For a piece of development I am looking at, I had the usual questions of architecture for an ASP.NET application. I looked at a bunch of approaches, but to be honest the full blown 3-tier/model-view-controller approach was not really an option. I did not have a team of devs to spend 50 hours a week grinding out the code, or the confidence that – after users actually used the system – there would not be significant changes to the fundamental model.

So, my requirements were such that it needed to be very adaptable to changes, with a minimum of manual effort to change, say a column of a table which inevitably finds its way through to business objects and front-end code. Moving down the 3-tier/MVC approach would come with time, as I grew into it – with greater confidence that the fundamental model was solid.

Remember, the worst thing that can happen in a development is – not that your model turns out to be a bit off, and you spend an extra 3-6 months converting the work you have already done; but instead that you live with it. This produces a system that does not match the way the users want it to work, and well.. that creates an ongoing cost.

In this “agile”-ish mindset I investigated a number of techniques:

SqlDataSource Binding

Placing SqlDataSource items on the pages, and data binding from visual controls to those. This is effectively two-tier design, with all the pitfalls. More of an exercise really, this was really only appropriate in the most trivial of cases to act as a temporary measure before upgrading to something a bit better, like in 5 minutes time (read: prototype). The negative is a hit to the database, unless you activate the caching which I didn’t really look into. The idea of having a control on the visual design service directly bound to a SQL connection does not give me a good feeling.

Global DataSet

This technique involved designing a XSD DataSet using the visual designer, and keeping an instance in the global “Cache[]”. On first use this cache is created and pre-populated with the basic fundamental static data – eg. states, business rule tables etc. that in general do not change. This was enhanced with some SQL dependencies to ensure that when they are updated when the underlying data moved. No transactional information is kept here, just the stuff that may be changed once per week at the most. The advantage of this approach is you always have a ready supply of DataTable’s fully populated with the latest data. Then, when you come to drop visual controls – you can bind directly to these global DataTable’s, without the need to created, read, populate them as would normally be the case by using a DataSource. Of course this technique only works for data that is not customised on a per-user basis: or requires any level of sophisticated querying.

This technique also allows you to easily make a copy of the global DataSet, giving you a pre-populated DataSet to modify, and then throw away.

XSD DataSet Queries

The XSD was filled out with custom queries – embedded SQL where possible, or stored procs where some level of programmatic flow was required. I know this is not the best practice, but remember the goal here is to validate the design and user experience – I can always return to change the DataSet to call stored procs (and create more work for myself when things change) once the ground has firmed up.

Multiple Tables in DataSet

This technique I had to search for ages before I could work out how to get it to operate. The theory goes – I have a DataSet capable of storing data from several tables, and I want to display relational data from several tables. The DataSet provides excellent support for this – by allowing you to easily navigate child rows from related tables.

So, I wanted to display multiple tables (using child GridView’s within parent GridView’s). The problem here is that the normal DataSet designer cannot handle (from what I can see) multiple resultsets from stored procedure calls, which is the normal technique of getting multiple tables from a SQL database.

The way to do this requires you to abandon ObjectDataSource, and jump down one level to SqlDataAdapter. This allows you to provide a mapping between resultsets coming back from your stored procedure, and tables within your DataSet:

WidgetDataSet data = GlobalDataSetCopy();
SqlDataAdapter a = new SqlDataAdapter();
SqlConnection connection = new SqlConnection(ConfigurationManager.ConnectionStrings["SqlDatabase"].ConnectionString);
SqlCommand cmd = new SqlCommand();
cmd.CommandText = "GetAllByWidgetId";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Connection = connection;
cmd.Parameters.Add("@WidgetId", SqlDbType.UniqueIdentifier).Value = new Guid(Request["WidgetId"]);
a.SelectCommand = cmd;
a.TableMappings.Add("Table", "Widget");
a.TableMappings.Add("Table1", "WidgetTitle");
a.TableMappings.Add("Table2", "WidgetRole");
a.TableMappings.Add("Table3", "WidgetDocument");
a.Fill(data);

FormView1.DataSource = data.Widget;
FormView1.DataBind();

If you look at the code generated by the XSD designer, you’ll see that this technique is actually used to map single resultsets to a specific table in the DataSet. As far as I can see there is no way to get it to support multiple resultsets – definitely a must for ASP.NET 3.0 I would think. The wizard could guide you through providing this mapping, meaning you could use the TableAdapter infrastructure for these types of queries.

The above is a lot of work, and still leaves you with the issue of providing changes, since they must be handled separately via individual stored proc calls. More work in my view – especially in the early fluid period. It does however provide the best solution from a performance perspective, since a single round trip can handle large numbers of parent records, with all their various types of child records.

Binding DataSources to One Another

This can be enhanced by some very sophisticated data binding where you effectively bind a child ObjectDataSource to the child rows in the DataSet for the parent ObjectDataSource. It looks a bit ugly, but it effectively allows you to put a Repeater within a GridView column; which is very useful:

eg.

<asp:Repeater ID="orders" DataSource='<%# ((DataRowView)Container.DataItem).CreateChildView("CustomerOrders")%>' Runat="server">

See http://www.codeproject.com/aspnet/Mastering_DataBinding.asp

Completely Seperate ObjectDataSource’s

The advantage of using multiple ObjectDataSource’s (over multiple resultsets and custom SqlDataAdapter) is that all the insert/update/delete operations are handled in a unit – you don’t need to worry about them separately. The disadvantage is that there is a round-trip to the database for every parent record, and for each type of child records. Something you need to weigh up depending on the application.

For me, the performance hit at runtime does not outweigh the implementation overhead, and besides I can always convert back to the more convoluted method should the need arise. Rearranging the DataSource’s does not involve messing up the visual controls since they are abstracted by the interface they supply. It also opens up the possibility I could create real business objects, instead of the ones supplied by the XSD designer.

EPIRB and PLB's: emergency transmitters

2006-05-15T13:42:00.000+10:00

A few weeks ago I went on a mountain bike ride (28K or so) into Kangaroo Valley, a couple of hours south of Sydney. One guy had a really bad stack, severing all the tendons in his shoulder. We were a bit concerned about other injuries (crushed hand, bruising all over etc.) and one person in our party considered using his "EPIRB". I had never heard of one.

Basically, it's a distress signal that means that emergency service professionals know you need help, if you're in a seriously bad situation with no alternative. As it turned out, we were able to walk him out (3KM or so) and National Parks drove him to hospital.

Anyway...

This lead me to investigate this “EPIRB” thing, how it worked etc. and this is what I found out. Analogue EPIRB’s (~$300) are single-use devices that transmit a signal on a known emergency frequency. This allows you to be detected and tracked by satellites and aircraft as they try to find you. Here’s how the process works:

Between 30mins-5 hours: the satellite picks up the signal (depending on when the satellite next passes over the region you are in, and relays it through for evaluation, providing an approximate area of about 20km. That’s big.

Aircraft that are in that area (private or commercial) are requested to provide a bearing on the signal. This may never happen, if there are no aircraft.

Depending on the situation, a helicopter crew is requested to deploy to the area to locate, and air-lift or support a ground recovery.

So, the upshot is you could be waiting a long, long time for a response. The major problem is that 97% of all analogue EPIRB’s are false alarms. Kids playing in their shed find one in the fishing boat – apparently they are inundated with alarms during school holidays.

For this reason the analogue system is being shut down and replaced with a digital system.

The digital system is different in that a personalised ID is sent with the signal – which allows emergency response to obtain details on the owner of the PLB. Also the signal is received within 3 minutes, without the delays caused by satellite overpass. Contact with the owner via mobile phone is attempted, and background details consulted to work out what the likely scenario could be. From here the process is very similar to that of analogue, except that the location is a little more specific – but still an enormous area to search.

The practical solution is a digital PLB with GPS. With this device the GPS location is transmitted, meaning that the response team have an exact location within 3 minutes. This allows them to further validate the incident, since the location gives a good guide, and respond quickly with appropriate response (possibly land not air).

So, the moral of the story is – don’t rely on an analogue EPIRB to give you a 10-30 minute response time (I was told 1 hour would be the very best case). My use for a PLB is mountain biking in mountainous terrain, and an anonymous signal in a 20K region is not all that useful. Digital with GPS is the way to go, when they get released later in the year (~$600).

Answer to XML namespace question

2006-04-18T20:37:00.000+10:00

Chris Lovett gave me an answer to the previous question (I promised I'd post the answer) on XML namespaces. Why does the framework create redundant xmlns='blah' attributes when parsing XML into the DOM.

Chris filled me in:

You are correct. Default namespace attributes is redundant. But there’s another wrinkle.

The XPath spec requires that namespace:: axis return actual namespace attributes, so it’s handy to have them in the tree for this purpose also.

XmlDocument doc = new XmlDocument();
doc.Load(@"..\Employee.xml");
foreach (XmlAttribute nsa in doc.DocumentElement.SelectNodes("namespace::*"))
{
Console.WriteLine(nsa.Name + "=" + nsa.Value);
}

ASP.NET Challenges

2006-04-18T20:34:00.000+10:00

I've been doing a bit of ASP.NET lately, which is a first for me really since the original ASP days. Short of the more-than-impressive Microsoft demos I've really only had superficial exposure to it. I have seen so much about the impressive productivity benefits of data-binding and the like – time to find out.

Why am I doing ASP.NET? It's not really my thing to create data-driven websites, but I have a new product in the real estate industry that requires such a platform. (hoping to launch it in the next week)

Lets just say the whole data-binding thing is impressive, but when you use it in anger you get.. well.. angry. I had this "Ground Hog Day" experience of looking up problems I was encountering on Google – only to find everyone else with the same questions, and very few answers. The big ones I'll summarise here.

The first one is where you bind something to an ObjectDataSource, which is then connected to a TableAdapter – and you want to do updates on a table that has a key. You will inevitably find your way to the error:

ObjectDataSource 'MyDataSource' could not find a non-generic method 'Update' that has parameters: [data columns], original_[key field].

Essentially the problem is that read-only fields are never supplied as parameters. Note this error only happens when the key field's ReadOnly is true in the control you are bound to. If you set it to false the field is supplied and everything works fine.

Maybe it is correct, but it sure is ugly annoying. There should be an easier way to produce an Update call on the TableAdapter that does not include the setting of the key values.

And that is exactly the solution – create a custom UpdateX() on your TableAdapter (in the XSD designer) to take the exact list of fields that you are updating. Then bind to that UpdateX() call in the ObjectDataSource. Makes sense, since sometimes you don’t have all the data for a row, so you need a SQL statement to just target them.

As far as improving ASP.NET 2.0 – I'd like to see the ability for DetailsView and similar to bind to a ObjectDataSource via DataTable's not just parameter lists. Then you can edit only the bits you care about (who cares then about which ones you aren't), and you don’t need to create custom update queries.

WPF: Auto-expanding TreeView

2006-04-04T08:59:00.000+10:00

How do you auto-expand items in a WPF TreeView? Beatriz Costa (Microsoft) gave me this tip. Basically, create a style for the TreeViewItem's:

<Style x:Key="AlwaysExpand" TargetType="{x:Type TreeViewItem}">
<Setter Property="IsExpanded" Value="true"/>
</Style>

.. and then use it on the TreeView:

<TreeView ItemsSource="{Binding Source={StaticResource doc}}"
ItemContainerStyle="{StaticResource AlwaysExpand}"
Margin="0,0,0,0"/>

Much easier than responding to change events, and manually expanding each TreeViewItem as it is created. Done!

XML namespaces

2006-04-03T10:42:00.000+10:00

A few tricks I found with XML text/DOM mismatches.

When you read some XML text with a namespace (eg. <root xmlns="testns">), this actually sets the .NamespaceURI to "testns" on the element (as expected), but also creates an attribute called "xmlns" with a value of "testns".

I found that .NET 2.0's implementation will ignore the .NamespaceURI value if you change the attribute. This begs the question: why read it as an attribute at all. If you create the element natively in the DOM (specifying the namespace at creation), it does not rely on such an attribute being there, so why include it..

XML bits and pieces

2006-03-31T15:42:00.000+11:00

I have been doing some QA work on MotionSet (part of SuperSet). A few things have jumped out at me, particularly when reconciling the three manifestations of XML data:

XML text (ie. angled brackets)
XML DOM object model
XML Information Set

The first is text nodes. The DOM handles adjacent text nodes fine - but when they get rendered to XML text, the distinction is lost between that and a single text node. I cannot see any way to preserve the DOM in this instance, when transporting via XML text (an email to a XML guru at MS pending..).

This opened up some other issues with the Infoset, where I discovered the DOM is quite distinct from the Infoset. In one case, I added and removed some child nodes from an element (this is in a DOM). Prior, it rendered as "<root/>" but after it rendered as "<root></root>". I thought the DOM would act more Infoset-like in this case and collapse it back to "<root/>". See the end of this post for a solution. (under the Infoset, these two representations are the same, since it models the underlying data)

So far I had been planning to target the Infoset, since it makes more sense long-term, just like targeting XHTML. They're not quite "here, now" - but they will take over things soon enough; and are close enough to XML text and HTML for people to understand them.

So, you are left with three distinct partially overlapping XML models. Most models have shortcomings of things it cannot do:

XML text: distinguish between adjacent text nodes
DOM: anything?
Infoset: distinguish between short and long form empty elements

It looks as if DOM is the most universal (trying to find some things it cannot do). Though, for mine, it is pretty close to Infoset as long as you don't care about the subtle difference between "<root></root>" and "<root/>".

So, the only real problem facing us here is now to handle adjacent text nodes. Strangely the XML text format looks a bit inadequate when considering this. The only workaround I can see to use XML text to represent the following XML document:

<root xmlns="someother">
This is the first text nodeThis is the second text node
</root>

.. would be wrapping in a XML-based wrapper:

<x:document xmlns:x="somedodgytechnology.com">
<root xmlns="someother">
This is the first text node
<x:delimiter/>
This is the second text node
</root>
</x:message>

.. a bit messy.

As promised, the solution to the problem in .NET's XmlDocument where creating and deleting children to an element split it out into a start and end element tag is as follows. Basically it attaches to the event where nodes are deleted, and does a check on the parent to see if it is now empty. IMO the .NET implementation should do this - can't see any reason not to?

doc.NodeRemoved += new XmlNodeChangedEventHandler(XmlNodeRemovedEvent);

private void XmlNodeRemovedEvent(Object src, XmlNodeChangedEventArgs args)
{
// does parent element now have no children?
if (args.OldParent != null &&
args.OldParent.NodeType == XmlNodeType.Element &&
!args.OldParent.HasChildNodes)
// make sure it gets market as empty
(args.OldParent as XmlElement).IsEmpty = true;
}

Post MIX

2006-03-28T12:37:00.001+11:00

I happily brought home a cold on Friday from MIX06, which took me out for the weekend - and then gave my wife bronchitus. Now the whole family is not going so well :) Oh well, at least I had some pseudoephedrine to bring be down slowly from the highs of Vegas.

Looks like local Australian David Richards started a fire late last week with his article 60% of Windows Vista Code To Be Rewritten. My first reaction was exactly the same as yours I'm sure: only 60%? What about the other 40%?

David likes a good juicy story, and must have followed up with this one to make sure the fire took hold. I hadn't talked to David in a year or so, so I checked his email address I had was still valid - he called me back.

There has been a lot written on the now infamous "It's not going to work" where it was discussed why the current approach to Longhorn needed to be "reset". The "reset" is known as the decision to abandon the approach of creating a completely new Windows built on the new .NET CLR technology. Instead, they decided to keep the status quo and ship the .NET CLR with the OS, but not actually use it - except some of the games maybe. That might be a bit harsh, but you get the drift: it's not like the core API will be managed .NET like it was originally envisaged.

The parallells between the reset and the current angle by David Richards is very strong. It looks from the outside like David Richards is describing an alternate outcome, where MS decided to NOT to "reset". Pursuing this approach is the thing Allchin was talking about when he said "it's not going to work" - and that sounds like what someone looking down a canyon of a 60% rewrite would say.

So if history is telling us that this all happened years ago, why is David Richards talking about this as if it is very recent? Well, as Paul Thorrott said in his article Microsoft do railroad people when they are onto something true, when the PR machine want it not to be. This can have very bad consequences from Microsoft, when something is genuinely not true. I guess that is the key isn't it - we dont like to believe those who try to control the truth. The other key is that a liar wont believe anyone else (U2) - I just had to throw that one in.

60% rewrite? Man I cannot imagine that given a wholesale ship in November. Maybe David Richards has other thoughts on the actual ship date though. Time will tell.

Many at Microsoft are pretty pissed about this article: Owen and Scoble just to name two. All I can say is they will look pretty stupid if they turn out to be wrong, then again I thought that of all those WMD's as well.

The progress of Vista "seems" to be fairly predictable now - the stabalisation, internal (Microsoft) rollout etc. Everything seems to be tracking as did 2000 and XP. In an alternative universe where Bills said "no reset" I can see David's picture coming true (a true disaster scenario). However, Vista seems to be pretty tame.

What I have more difficultly with, is seeing how Microsoft will deliver on Bill's promise (at MIX06) to deliver smaller incremental updates to Windows. I mean, Windows has some serious design problems that will require a large scale rewrite, and many breakages to fix. The lack of security in the Windows messaging infrastructure for starters. I mean you really cannot trust anything you see on a Windows desktop - since it can be manipualted by any other application. Great for desktop utility writers, terrible for InfoCard. I need to read up on security in WPF to see if there is a bridge there someone to provide a legacy (insecure) sandbox for GDI apps. I have seen no mention of desktop security in WPF. Perhaps WPF hasn't been built that far out, fine, but I hope it has been at least thought of.

MIX Summary

2006-03-24T13:55:00.000+11:00

MIXisms

Here are some really cool terms (they are now soo last week) that got more than a few laps at MIX06. You could spin together a compelling discussion at MIX06 with only the following words available:

Mix It Up
Mashup / Mash It
Right click experience
User experience
AJAXness
Single click experience (usually said in a demo just after clicking at least 5 times)
[wildcard] experience (used to describe anything that would have previously been obvious)
GeoRSS
Microformat
Social Networking Site
RSSable
Affiliate program
How will you monetize that?
REST / SOAP / WDSL / AJAX or blah blahblahblah blahblah…

IN SUMMARY - MICROSOFT

Microsoft is a company people like to talk and bitch about. Being the biggest and best (financially at the least) means you do get your mix of opinions. Microsoft is like an economy of its own, with lots of companies (divisions/products/teams) within it. Each is competing (in part) with the others, for status (some say more so) and capital; each has its own story, its own view of the world, and its own plan for getting there.

Microsoft is a company of contradictions. The big one is the fact that it likes to ask:

Question: "Which companies here are developing their core business applications in .NET and taking advantage of the benefits of the Microsoft .NET platform?"

The answer: not Microsoft.

Microsoft's three main products (Windows, Office, IE) are native Win32. Their big pitch for a revolutionary new technology has so far fallen on deaf ears internally, with reluctance to use the new technology for its core revenue. This has at least two major effects: ISV's are frustrated that they become the bad guy who says customers need to install this .NET thing, and well - the MSDN Magazine guys must be pissed. All those wonderful glossy colored diagrams, gone to waste on the cold face of staunch opposition.

Of course we need to balance that with the other attitude that people have towards Microsoft: the company which forces massive software updates down customers throats. Also, to be fair, Vista will fix this - now we just need to wait for everyone to buy a new PC. (I might go buy some Dell shares..)

ONTOLOGY OF MS

Here's my theory: the Microsoft empire stack goes something like this: good-evil-good.

Good: the guys at the bottom are like any brilliant, enthusiastic, professional bunch of software professionals you could ever meet. They are passionate about what they do, and want to deliver the coolest solutions to problems. They are not working at Microsoft to take over the world, rake in infinite amounts of cash, or harm anyone along the way. Most would be just as happy working on some open source project, or at Google or Yahoo - but they like Microsoft the most simply because they can achieve more there. They have the world’s eyeballs. Saying they would be happy working at Apple, I think would be taking it too far.

Evil: maybe a bit of a harsh word, but this is to represent the business practices from the top are clearly very every-man-for-himself. The current new nice, glossy, warm fuzzy-bear-hugging show-me-the-standards personality is there because it is now in Microsoft's best interests to do so. Their position in the market is now best served with this strategy. It is not because they have concluded that their previous attitudes were at all "wong".

Good: at the end of the day Bill is not making money to buy a bigger boat, or boost his ego. He is genuinely focused on improving the living conditions of those less fortunate as efficiently and effectively as he possibly can with the resources made available to him by his wealth. Bill makes sure that the entire company is actively involved in fund raising, and takes the responsibility for administration of such funds just as seriously as shipping software. Even if a few rich American tech companies get trampled along the way.. I mean they wouldn't give it away.

SUMMARY - MIX06

MIX06 was a great conference. I can still see the image in my head of the "designing for 10 foot" session with the glowing Apple logo from the laptop giving the presentation, elegantly perched on the intimidating (and clearly uncomfortable) Microsoft table. I did notice that many attendees did increasingly come out of the closet as the conference went on, by getting their Mac's out for increasing periods. By the end of the last day I think some people had gone and bought Mac’s to hang out and be cool.

The IE team were *fantastic* - friendly, engaging and dying to find something that broke their browser (in a productive kind of way). Fortunately I was able to help them out with that!

Bill was Bill, and plenty was reported in the press on it so there is no point rehashing. But he was a good sport to come down and put himself out for the press's analytical pleasure.

MIX06 embodies MS's confidence with their position in the industry, letting their guard down, but that’s no risk to their complete dominance.

On the constructive criticism side of things:

- Speakers were often teleported away (often to Seattle) straight after their talks, and were not really that accessible. They were around, but there was never enough time to answer the questions from the usual suspects pestering them after the sessions with nerdy insights.
- Developer-designer interaction was not really there, not sure how you would do it: mash-off? I would have loved to find some designer willing to help make by ugly pieces of functionality look slick in WPF..
- WPF was not well represented, I came with a bunch of WPF issues that I hoped to work through face-to-face - but was left with a forum post.
- Dean's "SEVEN" shirt on Ebay was not there as advertised. It was going to be washed first, right?

ONLY IN VEGAS...

I overheard someone overhear in vegas:
1: "you're married?"
2: "yeah."
1: "for real?"
2: "yeah, we have a dog and everything."

MIX06 - Day Three

2006-03-23T03:50:00.000+11:00

INFOCARD

Phishing is a huge problem, is on the rise, and is causing individuals to reduce the amount of online purchasing. The number of known sites is in the order of 7,000 - with the number of sites almost doubling in December. Presumably at least partially due to seasonal factors.

A very different beast from Passport. Basically it works by users creating their own self-issued Infocard, that contains all personal details you may want to exchange with others. At least then you are assured that identity is preserved - in that one user that comes back again later, is the same user. It does not assure that the identity is correct. Each card contains a list of "claims" - that is details; name, address, phone etc. All fields are optional, but may be required by any particular website in order to conduct business.

You can then get the Infocard signed by third parties (professional associations, banks, government etc.) - there is no centralised "Passport.com"-style Microsoft master database. Though I'm sure Microsoft will participates

Users have a list of cards, representing all the different cards you have in your wallet. All your department store, food and book store loyalty cards. Your ATM, Visa, health care cards etc. Then you can choose, in a way, which identity you want to show to a particular business. It is not about making a "master identity" in the same way that Passport did.

A lot of UI security around freezing the desktop while a user manages/uses their cards. Presumably this to ensure that phishing sites don't try to mislead or confuse users.

Self issued: ones you make up yourself, basically a encapsulated Google autofill, if you like.
Managed Card: cards issued by third parties, where they endorse the claims in an Infocard.

Makes extensive use of WS-*, so the comms will get pretty chunky - but it is pretty open. Infocard can interoperate with other systems (yet to be implemented, but MS say there is nothing proprietary in the interfaces..)

There is a substiantial amount of the system that has a Linux-based open source implementation, and apparently the user UI is being implemented as well. All indications are from the standards focused (not just MS) that this will be interoperable. I think it needs to be, not another Passport.

MICROSOFT RESEARCH

This was a pretty cool demo from a company who use MS's IP and use it to analyze raw traffic data.
They take everything:

- Traffic counters
- Taxi cab locations
- Traffic cameras
- Toll booths
etc.

.. and they produce:

- Current traffic conditions on every road segment
- Current travel times for a given route
- Predictive traffic conditions in the future
- Predictive travel times for a given route in the future
- An analysis of the departure time v length of journey, for a given route

WAY cool. It deals with all pattern lengths - daily, weekly, monthly, yearly etc. Believe it used a basiean filter.

VIRTUAL EARTH / MAPPOINT

This was pretty good - if I were interested in mapping parts of America. I guess it would be useful if I lived there. Apparently our (Australian) government is not that interested in taking photographs of the place. Sounds like they haven't been able to get suitable snaps and maps of the colony. Lets hope they can get this data real soon so we can start working out convoluted solutions to pizza finding in arbitrary locations.

MIX06 - Day Two

2006-03-22T10:18:00.000+11:00

AJAX

A repeat of the usual AJAX hacks, issues and contradictions. I'm not a real fan, and have talked to quite a few people about the shortcomings. I need a working model to demonstrate some ideas - who knows what it will end up looking like.

Media Center

There were 6 million units sold in the past year - more than I had thought. Four options exist for writing MCE apps:

HTML: web pages loads within MC, the only option for MCE 2005. Can be used by any webbrowser as well. Ho hum.
WinFX: requires Vista (Home Premium or Ultimate), and is effectively a XBAP (XAML Browser Application Program) within MCE. This creates a seperate app, that is not integrated into MCE other than just being launched. Can be used on any IE/WinFX capable client.
MCML: locally installed Media Center Markup Language application allows you to add functionality to MCE, and is the same technology used for the the default functionality in MC.
MCML Web: this is markup generated dynamically by a webserver (IIS or other), but of course is targeted only to MC.

The Vista MC looks better in terms of functionality, but navigation I think is less intuitive (I must be missing something). The big push was towards the media extender (specialised boxes plus xbox) - so the model is, leave your PC in your office - and have an xbox in the living room. Not a bad model.

MC is obviously going more mainstream than Steve Jobs, or I, had invisaged. As an owner of a MCE 2005, it is not quite "there" for my parents. There must be lots of me out there.

A great session on 10' design - that is how to design an application for use on a large screen, where you are about 10 feet away. I am not a designer, so it was more than a bit depressing (even the bad examples are far better than anything I could produce :)

myspace

The myspace.com guys were nice enough to invite us all to a party at V Bar. A good night. These guys are on top of their world.

MIX06 - Day One

2006-03-21T08:13:00.000+11:00

Day one started pretty well, here are the highy biased highlights:

FIRST SESSION

BillG

Standard format presentation for Bill, with the usual trimmings of information describing what is going on at MS. Press were pretty scathing about something.. not enough Hollywood I think. And definitely not the next governor or president.

MySpace

These guys are growing VERY quickly - like 200K+ new users PER DAY. Man. Had a demo of some new cool stuff which I can't remember any more - so it must not have been too compelling for me, but I am probably too old for their demographic. Featured the only heckler (Macromedia founder Marc Canter), which I thought was going to be about the war, or censorship or something, but turned out to be something far more important: "open up your API". No comeback line was forthcoming.

BillG -O'Reilly Face Off

This was pretty brave of Bill, but he got off pretty lightly. Not a guy you really want to tick off, but it was very good of him to expose himself to something that could have turned out to be far worse. Well done.

One good one was O'Reilly gave him a bit of stick about all the knock-out blows delivered to competitors in the past (eg. Netscape - there are string of them...) by Microsoft. Bill's response was pretty cool - "there were a lot of supposed beat-ups where the other guy just knocked themselves out". Must be a lot of extremely bad swingers out there. There are enough of the "faithful" here to ensure this one will go down OK for Bill.

Comments did cut a little deeper to imply that Microsoft "cut off the oxygen" to players that worked on the Windows platform after they had built a substantial business. The implication being that the reason for the web's success is due, in part, to the fear of ISV's in building huge businesses on Windows. Yahoo, eBay, Flickr, del.icio.us etc. When I thought about it, the life expectancy of a large ISV's building on Windows is not that great. Adobe is the next one on the block if history is to repeat itself. Interestingly Bill did not take the usual line that "Adobe are high end, some of our stuff touches there on the consumer end, but we're not really competitors". Instead he said they would coexist happily as competitors. When Bill says he is going to happily coexist with you - be careful.

O'Reilly finished with a public invitation for Bill to attend a less friendly audience: his Web 2.0 conference later in '06. Bill responded with a typical polite and diffusing comment that got him off the hook without responding. I got the impression he wont be there.

IE7: "we screwed up", "we're listening now"

Kick off with a self inflicted beat up on MS's decisions in relation to IE. While they didn't say exactly what those things were - most knew that MS had pretty much shutdown the IE team/development when they got to 95+% market share. So there was no IE team for (guessing here) 4 years or more. The IE team is back (well and truly), look for 9-12 month cycles next time. And innovation galore, none of this "we won the war - lets just sit back and milk it". They have done a great job, and the smart ones (Yahoo) are leaving them to dominate.

Where do I get a "SEVEN" t-shirt? (with an IE "E" logo)

Improvements:

Many (many) more layers of warnings (including roll-back) when users wind back security settings to get applications to work. Good luck to those call centers who rely on
opening up IE to get Java/ActiveX based products to work. Might have to review those practises, or.. they can just find out the hard way.
In discussions with the MS techs they did say specifically that ActiveX *was* being supported going forward. I talked a bit about all the bugs I had run into over the years with screwy Downloaded Program Files directory contents, and they looked a little blank. (not a known issue) Tough one to repro/track down, but my ActiveX days are over :)
Cool new interface, way simpler
Security: major new features, including a long (longer) list of warnings when downloading ActiveX, green/red cue to tell you of dodgy websites (phishing blacklist, certificate mismatches, suspicious techniques etc.), or certified high-assurance sites (certificates are not available yet, but will be difficult to get).
RSS: the real story is the RSS platform; infrastructure that IE can front-end (very basic), but Outlook and Express will do the same thing as well. Not the death of readers, since
they too will be able to plug-in to this common infrastructure. This is the new, warm, fuzzy and "give me a hug"-ing Microsoft. (that someone forgot to tell Bill about in his presentation). Sounds like they love everyone now, not just developers. The wierd bit is that it is native, with a COM interface - a .NET wrapper in the future. Go figure.
RSS enhancements: new tagging functionality (ebay using it) to allow the new RSS platform to do more advanced filtering, sorting etc.
Some very clever new IE7 tools for developers/designers - basically to debug ActiveX controls, layout (CSS) issues etc. A nice HTML DOM viewer, that does Spy++-esque finding of nodes in the DOM
CSS improvements: lots better standards compliance. When you have 90%+ of the market, you want things to be open.
Open Search: this is quite good, an open standard (A7) for search/research providers so everyone can connect with the built-in IE7 search bar. Even unknown sites can easily advertise their search interface for users to subscribe. Also looks like a nice way to edge-out the Google toolbar, since the search bit is the major part.

Atlas: "this demo is for you"

Gutsy demo with no prefab or prebake
AJAX cross-platform platform thingy blah blah, things I didn't know was:
Major part is a client-side library, so it works with PHP, Apache not just ASP.NET. Usual story... best with IIS but works with the rats and mice.
Supports Safari, not just IE and Firefox.

WPF (Windows Presentation Foundation)

Not that much new really, a video replay of North Face Kiosk demo (cool - but didnt get any of the American "whooping" factor I expected)

Have only found some (little) expertise on WPF here so far, hopefully there are more out there. The branis must still be running hot up north.

Infocard

The jetlag was starting to kick in around here...

An easy to use identity/certificate management system- containing personal details for users, so no more typing all the usual stuff you need to go through whenever you hookup to any website today. Looks good. I presume this is IE only, can't see how it could work with the others.

AJAX: Show me the money

A bit of a recursive discussion on the state of AJAX, with a lot of non-specific generalisations on things. Real smart panel of people giving sometimes opposing views on things, which was great. My feeling was this whole AJAX thing is still very early, and poorly understood. I mean what is web 1.0, 2.0 anyway - the definitions seems to be very fluid.

Future of Internet Explorer

A bit of a beat-up on the MS guys. I mean it is not their fault that the IE team was abanded to the wilderness for half a decade, so it seemed more than a little unfair on them. Don't shoot the messenger kind of thing. Anyway, MS well and truly communicated that they are on the standards bandwagon - the biggest (and unanswered) question from Dean was "do you want compatability or compliance". Unfortunately the panel were not thought-up on this issue prior, and seemed to find a way through to turn it into a bitch about the crimes of the recent past.

For mine, I think the ideal is to have backward compatability with some sort of identifier (a-la quirks mode), but then strict compliance going forward. If I have to choose, I would say compliance - better to break it now before the web grows another order of magnitude. In any case, leave HTML in the wilderness - and do XHTML strictly and properly. That might help to kick it along as well!

Sweet Sunday in Vegas...

2006-03-20T11:33:00.000+11:00

Woke up late this morning (still adjusting), and went down to Shadow Hills Baptist who were genuinely suprised to see someone venture down from the strip. Pretty big operation down there - 1,000 people or so. Tom & Debbie took me out for a tour, lunch etc. and answered a few questions that only locals can answer. Tried to buy some touristy toys for this kids, but this whole town (and everything in it) is labelled "not suitable for under 3 years".

Took a walk down the strip (I really hadn't seen it much yet), there is some crazy stuff there like this massive TV screen with an oval shield the size of a football field. (it actually moves up and down to block the light on the screen, below)

The casionos are bigger than you can imagine, on acres and acres of land. Building is going on over the place

Tour day

2006-03-19T10:06:00.000+11:00

Got up at 3AM local time, left for tour at 5:30PM - it was a blast. Helicopter trip over Hoover Dam, grand canyon etc. I was big (isn’t everything here?) – but I suppose you expect that.

Next was the Hum Vee tour of the dam, desert areas etc. On the way (read: out in the middle of the desert) we met some real friendly Americans and made a promise that I would email Oprah about getting Bono onto her show. Go figure. Don't know how that happened, but "only in America" could you get into that conversation...

Viva Las Vegas (sorry..)

2006-03-18T14:21:00.000+11:00

On Friday... I said goodbye to the kids (plus Julie!):

(had to throw that photo in)

Here’s where the house is up to:

I left Sydney mid-afternoon on Fri, after a stressful near-miss of the fuel gage. Who’s to say it really was as bad as it said – I mean how can there be “0KM left” when we were still moving. Hmm…

Flew into LAX, then onto LV and have setup camp in the Venetian hotel (a pity Julie isn’t here to enjoy), in the main strip right smack in the middle of LV. The hotel has 4,000 suites (currently building to double its size), the place is expanding like there is only a tomorrow of oil. Truly amazing – an indoor replica of Venice complete with genuine-ishlooking (better than you’d think) piazzas and Gondola rides on the 4th floor.

MIX06 starts on Monday – should be great.

My first impressions are, well, predictable. All the usual stuff about the slot machines the second you get off the airplane (they are too heavy to fly), and the occasional monster trucks you see on the streets (people actually drive them), and the cheap gas that runs them (for now).

Met a whole lot of friendly Americans who have been at pains to ensure I understood that LV is life to excess (even for Americans). Best I can tell is that it is the weekend escape from (for?) conservative America.

But boy is this place moving. 4,000+ people per week (taxi told me 6,000+) with a metro population of 1.6 million (the taxi told me 2.5). Maybe the biggest growth is in, well, how much they think they are growing? Catholics, Mormans, Hoover Dam, Lake Mead. Who’s got time for blackjack...

On the work side of things, have put together the building blocks for a XAML/WPF based XML document visualiser. The problem stopping integration of the pieces is there are some bugs in WPF (TreeView?) that are preventing it form working. Hopefully talk to someone (maybe Bill) on Monday about it.

So, today (Fri 17-Mar-2006) has lasted 43 hours - ouch. Since I haven’t really slept for 32 hours, it is time to get some zzz’s, up early tomorrow for a grand canyon tour. If I can work out the time when I wake...