Bo And Luke Make Glass, Wine

image The Dukes of Hazzard was a TV show in the late 70s and early 80s. The plot, at least the way I remember it, always involved Bo and Luke, two brothers, getting tangled up unexpectedly in some nefarious scheme cooked up by the local Mayor, Boss Hogg. A sub-plot always involved Daisy Duke, their sister, wearing shorts and running around in a jeep, but hey, I was a teenager back then. At some point, the boys would end up crossing the state line and so be free of arrest by Boss Hogg. That “crossing the state line” has featured in many hundreds of American TV shows and movies.

It always seemed to me that governments are not unlike the Dukes of Hazzard. Not so much the smuggling of moonshine and driving fast cars, although I guess that happened in some places too, but in the idea that there’s a “state line” and when you cross it, your problem becomes someone else’s or, at the very least, there’s a whole new set of law makers involved. Cross the post code/zip code barrier and whether it’s healthcare, drug availability, housing taxes, bin collection, dog poo removal or whatever, it’s different. In most countries that I’ve visited, the consequences of this are near identical business processes supported by [deliberately] entirely incompatible IT systems across many dozens or hundreds of operations. Needless to say, almost every process lacks the scale to operate effectively and efficiently. The drivers are maintaining local control (or the illusion of it) and ensuring local people take local jobs for local citizens.

I’m always interested in people or businesses that break out of the mould of “we’re different, we need our own process/system/operation/call centre/sales and marketing operation” etc. Working in banking a decade ago, every country-centred business had its own operation and own IT – its own FX books, own securities settlement system, own cash reconcilement process and so on. Those were gradually simplified, rationalised and operated at scale. Sub-prime loans notwithstanding, banks operate pretty efficiently now, at least in their transactional operations.

Two examples of breaking the mould that I’ve come across whilst out and about (@large?) in the last few months:clip_image001

1) Glassmakers in Murano, under pressure from competing (and they’ll say inferior) products from elsewhere in the world (notably, but not limited to, China), have started to merge to gain scale. Visiting one factory on a recent trip to Venice, I saw that they made only one type of product – very modern. I asked about their other products. The manager told me that they had recently merged with 14 other glass makers, with each one deciding to specialise in just one product area. They had figured out who of their masters was best at each product and then given them the job of producing the very best of that product at a volume that the market can support. With the process from novice to master taking 15 years or more, and young people increasing leaving Venice to work on the mainland, there’s also a shortage of talent – and so no longer the ability to support every glass maker producing every type of product. Together, they put their money to work to build a single showroom that displays all of their products. Each one bears a seal of quality and the signature of the master who produced the piece. Prices are clearly displayed – well, clearly until you hesitate whereupon the calculator comes out and an “off-season” discount is proposed.

image 2) Winemakers in Australia, under pressure from the effects of climate change, their appreciating currency, the massive competition in the wine industry at the price mid-point (despite Australia beating out France in volumes, the bulk of the sales continue to be at the low end of the market – and the French are now starting to re-work their marketing and pricing and will, if they aren’t already, gain ground), and keen to show their products in the best light, have started to form alliances. One such alliance, Artisans of Barossa, brings together a dozen individual producers, all of whom make unique and special wines, and who now market their wines as an ensemble. Tastings are arranged head to head – so you can try out, for instance, an out and out Shiraz against a more varied Grenache, Shiraz, Mourvedre combination. None of these wineries particularly needed to come together – their wines are good enough to sell by themselves (and often have the awards to show for it), yet they recognised that together they are stronger: they have scale, can reach wider markets, can pitch each other’s products, can learn from one another, can reduce their costs of marketing, shipping and representation and so on. Taste their wines if you get a chance – and if you don’t, contact me and I’ll tell you where you can buy them.

I know that governments at various levels have tried this – whether it is local, regional and sometimes even national departments – but there doesn’t seem to be, from where I’ve stood and looked, the same willingness and engagement. Sure, the pressures are different, but the thinking should be the same.

If a local government is recognised as having the fastest, most efficient housing benefit process, why wouldn’t councils in the area (hey, even the country) say, “I’m not that good at HB, but I am good at business rates – why don’t I give them my HB and I’ll take their BR”. I understand that charters would have to change, I understand that system modifications would have to be made – but surely those are not beyond the wit of man if it simplifies and rationalises the processes. Not every process is unique and special – or, in fact, not every process is necessarily unique and special – I see that they often end up that way.

We could take this to a national level – and I’ve rambled about this before. The department of give, and the department of take, for instance. Payment scale and receipts scale. Could it work?

How many armies does an e-government need?

clip_image001Whilst @large over the last few weeks, four apparently unrelated events fused together in my mind to create an idea for governments that might make for both some fun and some real business benefit. These are the four events:

1. Since I first heard about it, probably when I was 10 or 11, I’ve wanted to visit the Terracotta Army near Mount Lishan in China. The nearest I’ve got so far is the exhibition at the British Museum. I’ll take that for now – seeing the original Tutankhamun show in London was the pre-cursor to seeing it all for real many years later after all. There are many astonishing things about this army – the scale of imagination to originally envision it, the incredible craftsmanship to produce such individualised warriors (coupled with an enormous army of people to make them), the bureaucracy and managerial process to create it (I won’t dwell on the facy that most involved were probably killed right after completion), the damage done to it not long after the first emperor died and, now, the reconstruction effort that means we can at least see some of the pieces pretty much as they were in 210BC – this last thought is only truly appreciated when you see the stills of how things were when they were found: millions of fragments piled one on top of the other with little to differentiate them. It felt like there were 100s of people in the Reading Room at the British Museum on the day I visited, snaking in long lines from exhibit to exhibit but I suspect they restrict each visiting slot to 50 or 100 people at at time. It’s truly an impressive draw, although one that leaves you longing to see the entire spectacle.

2. A few weeks ago I was amongst the first to know about a newsworthy event – and I found out through public sources rather than through some devious internal channel. It turned out to be a big story but I suspect few realised it at first. I happened to think of going to wikipedia to see what it said about the event. It was silent, entirely unaware, it seemed, that anything had happened. I took the liberty of adding my footprint to the armies of those who have gone before, and edited the appropriate page with the updated information. I sat back, pleased that I had added a [very] little knowledge to humankind. Within 15 minutes, seemingly dozens of others had updated the site, refining the information I posted, adding citation and links to other sources. The space that this news topic occupied could initially have been seen as very niche, yet a veritable army of people were apparently looking for something to happen so that they too could be editors of their own newsfeed.

3. I went to a meeting with some people who know do, loosely at least, some of what I used to do in the Cabinet Office. Of course, they’ve more than moved on from what I was up to – it’s around 2 years since I left. But a lot of the topics we discussed were ones that I’d spent time on before; ones that I’d commissioned work on, even paid money to allow government to action them in perpuity. I took along a document that my team had produced, with a vendor, in late 2002 when we were looking at rebuilding ukonline (now directgov) for the 3rd or even 4th time since its launch in early 2001. The document was bristling with great ideas on how to engage the citizen more, how to expose more of government to the outside world, how to structure websites and transactions so that they’d have the most impact and what areas to concentrate on first. It was a great piece of work and whilst we’d acted on some of it, I was sure that more than 50% had been left undone for time, money or capability constraints. In truth, armies of consultants, IT vendors, outsourcers and business process experts compile hundreds or even thousands of such reports every year for government as it merrily spends around £3 billion/year on consultants.

4. Lastly, I was looking for some figures to tell me how much use was being made of Freedom of Information requests. When I first thought about this law, in 2000, I was expecting it to be the offline equivalent of the 1901 Census website – something that would knock government out as it responded to potentially millions of both frivolous and fact-seeking requests filed by armies of citizens and, especially, journalists. As far as I can tell, it’s done nothing of the sort. But the more I hear about FoI the more concerned I am about whether we’ve taken the right approach in the UK.

So taking those four un-related things into account, I wondered:

  • What if government took facebook into the inside? What if we ditched every intranet there ever was in every government department and allowed everyone to create, instead, a facebook page for themselves? The same tools and applications would be available; groups joined would be centered on areas of expertise & experience (desired or actual) and room to play would be allowed to – no point in making it all business, there needs to be some kind of trade. Straight away, links would form between people doing similar jobs in different parts of the government (or different parts of the same department but spread around the same country); experience would be shared; job-postings would be easy to find and could be matched by a talent inventory that could draw on all 4-5 million public sector employees (that number could be anywhere from 250,000 to 7 million depending on how you cut things). Now I’m no great fan of facebook – truth be told, I don’t really get it – but I get its potential, in a slightly different context, to replace the intranet – to be a place where people look up contact information, find people that might know something that they need to know, exchange holiday photos, date, arrange to meet or whatever they need to know.
  • What if government took a licence for wikipedia and built an internal version? What if that site became the place where all reports from every consultancy that’s ever worked for government was published? Where people edited topics that they were interested in and added statistics, links and sources that were verified by the armies of others that were also interested in those topics? What if this became the hub of knowledge were people found out how to do their job, what they could do to develop in their job, where they would find information from others doing the same job, where they could see what consultancies and others had recommended could be done to a given process, function or organisation in another, related part of government. Or even a completely unrelated part of government. Many of those reports, the many hundreds every week, month or year, end up gathering dust in a cupboard somewhere. The very best are 50% implemented with the remaining actions getting swamped by the pressure of time or money, or the clean sweep of a new broom coming in with different ideas. That leaves perhaps a billion and a half worth of ideas left unimplemented every year. That’s a lot of intellectual property left on the shelf. And let’s not wonder aloud, at least not here, how much of those reports are repeats of what has already been bought and paid for by a government department somewhere else.
  • Next, what if we took every FoI request – and its response – and published it online with a simple search application, driven by google or windows live or any other engine- so that before you asked your question you could see what else had been asked that was similar; you’d then either just use that information and not bother to ask your own question or you’d refine yours to get a better take. Smart journalists would use the search tool to bring together previously unrelated questions and draw even more conspiratorial conclusions. Smarter ones would phrase their next question to take advantage of the freely obtained knowledge that they already have to find something new. Government would respond, one would hope, by getting smarter about its operations and processes and would use this leverage to drive greater change and efficiency.
  • And lastly, maybe all of this would be turned inside out and put online, not just FoI requests, but reports and consultancy work that government had paid for, so as to act as the single greatest source of pressure for change and, dare I say that ugly word, transformation (the single best example of which continues to be Optimus Prime in Michael Bay’s recent Transformers film). The deluge of information would be enormous. The fragments of data would require an entire army to stitch it together into meaningful conclusions. But, let’s be honest, government itself is never going to have a big enough internal army to do this stitching but, the outside world, those who want to be part of an open-source government, now maybe they’d have the willing, the time, the intellect and the energy to sort, distill and publish the very best pieces – and government, of course, would pay for such pieces once and once only. Sadly, the name YouGov is already taken by a very clever chap called Nadhim Zahawi, but maybe he’d be open to offers. Failing that, we could always go back to me.gov, the vision of access to government coined in 2000 following the [necessary] demise of open.gov.uk.

This way, the vast body of knowledge that government accumulates year in, year out would be available not only to all of government but to all those with interest in what it says about where their taxpayer pounds, dollars or even, one day, renminbi. After all, it was that first Emperor of China, who unified the country, standardised currency & axle lengths and introduced many other reforms (and yes, I know he killed the 700,000 people who worked on his tomb, but bear with me – the metaphor nearly works).

Over the next 7 years, some 40% of government’s workers will retire. They’ll take an awful lot of knowledge with them. Not all of it will be useful, but figuring out which is and isn’t is a job for a distributed network of staff and citizens who can argue amongst each other, for a while at least, about relative value – promoting those items that their successors need to hold on to and relegating those that they don’t. And, in 100 years or 500 years, what better place for those who come after us to look for how things were done back in the early days of the 21st century.

The nice thing about these projects is that they could be started individually and cheaply. There’s no need for a huge infrastructure, no need for a complicated requirements gathering process, no need for expensive outsource deals. There just needs to be a bit of willing for a few senior folks in a few key departments who want to give it a try – who want to be bold (but not too bold) and take a step in a new direction. Along the way there would be pitfalls, there would be screw-ups but there would be successes too. And those successes would quickly build as more players came to be involved. Just starting one of these projects – say, facebook as government’s intranet – might go further to creating some joined up government than anything that has gone before.

Tick box to skip a year

aminny Much to my surprise my entry to the London Marathon in 2008 was accepted in the ballot.  The last time I got a place like that was for Paris in 1999.  Every other marathon I’ve always run under a charity’s golden bond scheme.

The acceptance form has a useful box which allows me to defer my entry until 2009.  With my knee still not working properly after I tore the meniscus back in March, despite an apparently successful operation, this looks to be my only option.

I’m going to give it a couple of weeks before I send in my deferral but I’m not confident of getting from zero to marathon shape on a dodgy knee in the time available. In fact, I’m not sure I could pull off 200 yards right now.

The Data Spiral

Here’s an extract from my Government Enterprise Architecture paper from September 2003:

The principal now is that data can be exposed to many viewers – internal staff, third parties, intermediaries and the citizen/business themselves. The number of data sources has been dramatically reduced, perhaps not to one but to a few at least. This has been achieved principally through abstracting the original back end systems using clever technology known as web services and through creating a set of consistent and reusable components.

clip_image002[5]The journey to such an enterprise architecture is lengthy – even achieving such a vision in a single department is a huge challenge. It may be appropriate to think of progress being made along 4 axes, not necessarily with equivalent speed. The axes are business process, business application, business data and technology infrastructure – note that the focus is on business involvement and leadership, especially around such important areas as data.

The model might look like the figure at right. Progress is made by moving out along any of the axes, with the time to make progress and the potential for cost saves increasing the further out you move. Although, progress need not be equivalent against each axis at the same time, there will be points when the next level of change can only be achieved when enough, dramatic progress has been made across each axis.

That slide doesn’t look too clear in the blog and I can’t find the original to paste in.  I’ll keep looking for it and then expand on this post.

But the recent HMRC (and everyone else’s) data problems reminded me of some of this – I was trying to create a future model for government technology where the citizen would be put back in control of their data, there would be common (not consistent) processes in government and an integrated suite of technology built with re-useable components.

Governments were, and are for the most part, in the middle of the middle: individual processes custom built each time, data held within individual applications, multiple over-lapping business applications and multiple technology infrastructures even within single departments.  No wonder data exchange is hard.

The essence of commitment

I had a strange conversation with a guy this week.  I needed him to sign up to do something.  He didn’t want to do it.    His way of telling me this was to say that he “didn’t want to commit in case he had to decommit later” – of course this was by email not an actual, whatchamacallit, a conversation.  I’m wondering if that should be de-commit.  Or, actually, I’m wondering if there’s even such a word.  Surely “commit” means just that – to commit to do something means you’ll do it, come what may. Much more than a maybe, somewhat more than a promise, as good as a guarantee from a reputable manufacturer?   We’re talking about saying you’re going to do something and then doing it.  His name, in case you’re wondering, wasn’t John Kerry.  Although you can imagine how he’d have explained it to me had he committed: “I committed to do it before I de-committed from doing it.”

That narrowly beat a conversation earlier in the week that included two words I never thought I’d hear next to each other: “hardcore strategy.”  I have no idea what that means. But I think I need to add it to my CV.  Along with the phrase “always committed unless I need to de-commit, in which case I promise to inform you in writing no later than 30 days after I’ve already de-committed.”

25 million green bottles

iStock_000003735726XSmall There are, as you’d expect, 1001 stories about the loss of 25 million records relating to children and their parents.  Child benefit is one of the most “taken up” government benefits – something like 98% of parents (umm, sorry, children) receive it (versus perhaps 80% for child tax credit). So there’s certainly a large number of people affected – the figures of 7.5 million households and 25 million people total look about right.  I’ve seen this called “DataGate” by the Independent.  Perhaps “Shutting the DataGate after the horse has bolted” may be better.  The story definitely isn’t over and I’m sure, barring any other major news developments, it will hold space in the first 2 or 3 pages of newspapers for several weeks and several more instances will doubtless come to light.

If you have a child under 16, your personal detail (name, address, bank account, date of birth and national insurance number).  It’s unclear whether if you used to receive child benefit (i.e. your child is now older than 16), your data was still available on the system, but I suspect not.  Likewise, if you are one of those who are generally off-system (certain members of the military, the police and so on), I suspect that data was held elsewhere – so those who talk about the risk of protected identities being compromised are probably wrong.  It is, sadly, one of the hallmarks of IT the world over that data is held locally in each application for each purpose – so this kind of data exists in dozens of applications across every unit of government, whether central or local, state or national, metropolitan or federal.  When we built the Government Gateway, we looked hard at the data we needed – for instance, to post the PIN, we needed an address; but, once posted, we didn’t need it anymore.  So we issued a query to the relevant government back end system, got the address, and then dispensed with it as soon as the envelopes were printed.  But that was relatively easy to do in designing a new system from scratch.  Most systems have been around a lot longer.

Let me state two things up front:

1)  Loss of sensitive data is not just a UK government problem or even just a UK problem.  It’s prevalent all around the world, in corporates and govenrments, and made ever easier by the increasingly wide access to email and the Internet – and, of course, by the ever increasing number of systems that store all the data that they ever need right in their main database.  It’s almost like we should be surprised if our data isn’t out there in the wild world.  Never mind worries about putting some personal information on Facebook, your data is already on several other sites, for anyone malicious or maligned to access.  There’s a reason that whenever you see people in a film going into a secure nuclear area, there are two of them and they each have a key that has to be turned simultaneously.  Putting control in the hands of one person can be a recipe for disaster. This latest issue comes on top of:

    • An event just a couple of months ago when a disc being sent to Standard Life and containing details of 15,000 people was lost (sadly also by HMRC)
    • 94 million Visa and Mastercard accounts exposed at TJ Maxx
    • Bank of America’s loss of backup tapes containing credit card information for 1.2 million Americans
    • The exposure of the records of 800,000 people at UCLA
    • Reed Elsevier’s loss of personal information on 300,000 Americans
    • Transaction data for 180,000 customers of Ralph Lauren
    • The use of unsecure email to send out classified nuclear secrets (that’s a link to the story by the way, not to the actual secrets)
    • Choicepoints loss of 163,000 individuals records (and the accompanying ID fraud)
    • Hackers in Ohio Universities systems took 137,000 records of students and alumni
    • The loss of doctor’s personal information on an NHS website
    • The loss of 26 million records for US veterans
    • and, golly, I’ve just found this extraordinarily comprehensive list of data breaches.

2) This isn’t a problem about why weren’t the CDs encrypted or why wasn’t the data sent by some other, presumably safer means, it’s about several lengthy failings in process: who can access the data, how easy is it to get a full database dump, what controls are there on writing data to CD, who needs to approve what and so on.  In the technical world that most of us operate in we’re used to a window popping up and saying “hey, stupid, are you sure you want to delete that entire list of folders and files?”.  There is no “are you sure you want to send this data by post dummy?” dialogue box, but there would have been checks and balances before it got to that stage.

It must have been a long chain of events to get to this point.  A full download of every data item in any of the government’s big systems isn’t the kind of thing that can be just asked for – I’d go as far as to say that it’s a one time request requiring special work (although it’s possible in this case that the extract had already been prepared for some other reason in the past – and, if that was the case, perhaps many of the usual controls would have been bypassed in this case.  Imagine the conversation “you need an extract? Well, normally that would take us 3 months but I just happen to have one over here, only one previous careful owner, that we took in April 2007”). 

I’d bet that there isn’t a requirement in the specification of any government system anywhere in the world to be able to “hit f12 to dump database to two CDs”, password protected or not.  So my assumption would be a change request is raised, the IT supplier (probably EDS as the Child Benefit process and accompanying systems used to belong to DWP but were transferred 4 or 5 years ago to HMRC but I don’t think they were absorbed by the CapGemini contract) does a quick check to see how long it will take, the change request gets approved (not as quick to get done as it is to write – perhaps a month or more), the data gets offloaded at the next convenient point in processing and then copied to two CDs by someone technical.    Lots of people get involved in this process.  There would even have been a discussion about the cost of removing some fields, hashing out others, creating dummy data and so on.  In the end, it sounds like we’ve got a very big spreadsheet secured by a password when you try to open it.  I’m not even sure that old versions of Excel can handle that many rows so maybe it was just a word file.  That’s a lot of pages.

My guess it that encryption wasn’t asked for because the person doing the asking wouldn’t have known much about that and the people receiving the data would have known even less, and the technical folks would have wondered about it but would have been busy and so moved on. PKI isn’t part of the default desktop installation of any where in government outside spooksville.  I could get into this a lot more but it’s a long time since I worked at the Inland Revenue and even then I wasn’t that close to the systems involved here – and I’d be speculating.  Doubtless someone is already working on a report and it will come out under FoI or through the persuasive nature of various journalists and, I’m sure, a series of Internet message boards.

As far as I understand, no one ever actually asked for a “full copy of the entire child benefit database”.  The NAO asked for a sample of de-sensitized data.  Typically that’s a few tens of records with personal identification information removed – certainly the NI record hashed and probably the bank details removed.  When I did a stint in audit back in my banking days, a typical sample was 30 records – statistically, that’s enough to give you a sense of whether everything is in order when you’re doing a substantive test.  I’m not sure what NAO were trying to prove – maybe that only appropriate data was stored (perhaps that only parents with children under 16 were in the system?) or perhaps that the fields contained the right data and in the right format (post codes matched what they were supposed to) or maybe they were testing that the population claiming matched the expected population claiming.

Putting aside then the issues of should the data even have been floating around 0r what process breakdowns were there, here’s a take on the technical aspects of how data should be shipped around:

Most people – as did one commenter on an earlier post – will be asking “why on earth is data being shipped on CD in this day and age?”  A perfectly reasonable question. And one that when you look at the other ways that were probably immediately available, you might briefly think “oh, I see why they’d do it that way” … right before you clap your hand to your forehead.  Don’t think that government (generally, not just the UK) are endowed with the latest hi-tech gear available to one and all.

Two CDs is a fair chunk of data.  At least 1.2GB based on standard format of 600MB a disc.  Not much compared with the capacity of the average ipod (even my iphone has 8gb, I think the entry level classic is now 80gb) or even the average memory stick (2gb is a common size for Vista ReadyBoost).  But a lot of data to ship around nonetheless.

Let’s take email as one option – most people would consider that first:

  1. Email systems in government generally have very small mailbox sizes. A few tens of megabytes is very common, even as much (as little?) as 200mb would be uncommon.  This is not like google where you get a couple of gigabytes or more on signup.   Trying to send 600mb would bust both sender and receiver.
  2. Bandwidth between departments is relatively small.  More accurately, there’s lots of bandwidth along the backbone  that links departments, but individual links to that backbone are typically small – 1.5MB/s, sometimes less (and are set as a function of the size of the department – I’d expect NAO to be one of the smallest (and I’m actually pretty sure, but not certain, that they’re not on the GSI), HMRC to be one of the largest).  Network performance in offices is load dependent and likely to be slow making uploading an attachment of 600MB to the server interminable.
  3. Many government staff don’t have access to email at all (if they are routinely processing citizen tax transactions, it’s felt there’s no need).  Likewise, even fewer have access to the Internet.
  4. Firewalls on the email systems limit attachments to 2mb, sometimes 4mb, rarely much more than that (there are exceptions but they are rare)

But had these all been overcome, the file would have moved between HMRC and NAO within the secure network of government departments known as the GSI.  Risk of interception would have been low (the GSI is regularly penetration tested and is built to a high standard).  But, realistically, this wasn’t an option for anyone in HMRC. Government email systems are just not built for files of this size – and I believe that even those that the rest of us use day to day would fall over after trying to digest a file of 1.2gb.  My entire PST file in outlook is only about one gigabyte now (and it has 2 years of email in it, the rest is in archives). With all these issues – and the continuing sense that e-mail is somehow unsafe (like all things on the Internet) compared with “sending 2 CDs by post (!) – I would not be at all surprised to hear that CDs by post is the default choice for exchanging even relatively small amounts of data between departments, agencies and 3rd parties (such as pension companies and banks).

Sometime in 2002 the team I ran in the Cabinet Office built, on behalf of the Criminal Justice folks, a secure email system.  It was the brainchild of the same guy that thought up the Gateway as a pan-government authentication system and, I think, ukonline (which was known originally as me.gov).  It was designed to allow lawyers working on criminal cases to exchange, securely, documents between their offices and the courts (and each other).  Remote users could use a web-based email front end or their own outlook client and everything inbetween would have been encrypted and secured.  At the time we deployed it, the common way to send such data around was to fax it (you remember the way it used to be done – you’d phone them up, say “stand by the fax machine”, then they’d put the phone down and go to their fax, nothing would happen because it was out of paper, or it was already receiving someone else’s 100 page fax, all on that slightly fuzzy thermal-style paper).  It was a comedy and needed to be sorted, hence the requirement for the secure mail.  This solution was made available to the whole of government, but take up was low.  I’m not sure that this would have been any better – it would have had the same limitations of bandwidth, firewalls, and so on.

In our own team, and before the secure mail system, we also used various commercial products to exchange secure data (the systems we built and ran were at least restricted and were sometimes higher).  They were based on hosted servers.  But the same issues of bandwidth, firewalls and so on would have applied.  On top of that, both parties have to be connected to the  secure system – so there has to be a set up process: passwords, keyfobs and so on need to be exchanged in advance and kept current. All of those things complicate the issue enormously – especially when such exchanges are not routine and day to day.   What usually happens is that they fall into disuse, the processes breakdown and then rather than take the time to set them up again, people look for a quicker way – popping 2 CDs into an envelope and putting them in the mail for instance.

So, no, email isn’t a viable alternative for large volumes of data.  In fact, uploading and downloading to websites via secure spaces, even when encrypted and super-protected, probably isn’t a viable way of shifting data around outside of your own secure network within the building, except when you’re talking about project-type information and using sharepoint or similar tools – and when you’re moving data that you wouldn’t mind someone else finding by accident if you haven’t set up your server security quite right.

Lots of companies offer solutions to these – the usual products chasing a problem to solve.  There will be lines of them queuing up to offer their services to governments (globally) and their IT suppliers over the next few weeks.  They will offer super-duper-extra-double encryption, they’ll say that they can identify rogue data being sent by email and divert it, they can check staff activities on the Internet and make sure they’re not doing things, they can spot people trying to download data off a system and copy it to their iPod and so on.  Of course, they spot the problems they’re design to spot; not the ones that happen off the beaten track or where the procedures are deliberately over-ridden.

But, on the face of it, had this data been copied to an iPod and hand-carried to where it was going and copied on to another iPod, we might never have known about this.  So iPods to come equipped with a government-approved fingerprint reader as the next step?  Or maybe personal memory sticks with dual control – sender and receiver fingerprint readers.

This is an undeniably serious problem.  There may have been many serious breaches as noted above, but few have stretched as far as the child benefit data.  The solution isn’t, however, simple.  And it isn’t about secure ways of exchanging data – at least not initially.  There’s nothing to say that had this data not arrived at the NAO securely, it wouldn’t have been left on an unsecure laptop and then been stolen from the back of a car for instance.

So:

  1. All of the processes around access to patient, customer, taxpayer, citizen etc data in every department, agency, non-departmental public body and local authority are going to go through a rapid review.  New standards will be enforced: senior management sign-off, dual control (keys round the neck and everything), IT supplier held accountable for where data is put and so on. This will take time and still things will be missed and it will happen again – let’s not hope that it’s on this scale, but it will happen again.
    • Lock down data exchange now.  People come to the data, not the data to the people. Until better processes are in place, this should stop the problem from getting worse.
  2. All staff should be taught the “green cross code” of using computers. The very basics need to be re-taught.  For that matter, the code should be taught at schools, colleges and libraries.
  3. The spooks should lead a review of deploying encryption technology to departments holding individual data so that all correspondence is encrypted automatically in transit using appropriate levels of protection for the job.  This will be expensive.  The alternative though is to make encryption optional – but because you can choose, sometimes people will choose not to (because it’s too slow or something) and the problem will recur.
  4. Systems being architected now and those to be architected in the future will look at what data they really need to hold and for how long and will, wherever possible, make transient use of data held elsewhere.  The mother of all ID databases would be a good place to start.

All of this will take time.  In the interim, managers in the line of fire are going to have to use common sense and check and recheck when they’re asked to provide information to anyone.  Social engineering is alive and well after all.