Big head, long tail. Not that I’m stealing Chris Anderson’s theory of the web, just re-awakening a description of government on the web that the OeE used in mid-2003 and one that we probably borrowed from Clay Shirky.
When I first worked in e-government, there was only one strategy for getting government online. It was driven by the the “2005, 100% online, citizen focused, joined up government” vision which, arguably, is as much a strategy as you need, but there wasn’t anything more explicit than that.
The result was thousands of government websites, very frequently replicating content that was already available elsewhere, still more often providing content that was less than authoritative or was quickly out of date as few had realised how genuinely hard it is to keep a website up to date and relevant.
During 2003 I regularly tracked the number of occurrences of a few government products across the entire .gov.uk domain as a proxy for the degree of duplication. I repeated the test today to give some comparison For instance:
| Product | June 2003 count | July 2007 count | Growth |
| Disability Living Allowance | 9,000 | 122,000 | 13x |
| Tax Credits | 11,400 | 193,000 | 17x |
| Housing Benefit | 49,000 | 1,160,000 | 23x |
| Self Assessment | 22,700 | 401,000 | 18x |
This is a far from perfect proxy but I think it plainly illustrates a staggering rate of duplication, even just within the .gov.uk space. To try and refine the measure a little, I looked at the rate of growth of the same terms within the site of the nominal owner of the product:
| Product | Owner | 2003 count | 2007 count | Growth |
| Disability Living Allowance | DWP | 1,010 | 1,660 | 1.6x |
| Tax Credits | HMRC | 2,730 | 28,800 | 10x |
| Housing Benefit | DWP | 2,180 | 3,900 | 1.8x |
| Self Assessment | HMRC | 2,820 | 41,700 | 15x |
This shows that whilst the products certainly occur more frequently now than in 2003 within the owner sites, the growth is far higher across the wider government domain for 3 of the products (and a little higher for the last, self assessment).
My argument, often expressed in this blog, is that this is wasteful for government and confusing for the customer. In July 2003, there were 2,643 government web sites. I don’t have a current total but the last time I looked, about a year ago, the total had increased to over 4,000. I suspect it’s not far from that now – given that very few sites appear to have been closed since then.
One of the consequences of confusing customers appears to have been that few used online government services.
At a Public Sector Forums conference in March 2003, I shared this slide with the audience. It showed that individual government sites had low market share and low loyalty when compared with market leading sites (such as the bbc and amazon). When aggregated together, however, the potential market share for government was significantly higher – although loyalty didn’t budge much. In short, joined up government online has the potential to reach a big chunk of the market
To illustrate this point further, here are the current market share figures for the top 10 government web sites for last week, sourced from Hitwise via Public Sector Forums (the comparison figures are those from July 29th 2006):
1 Met Office www.metoffice.gov.uk 13.42% (versus 11.48%)
2 Jobcentre Plus www.jobcentreplus.gov.uk 9.89% (versus 10.21%)
3 Directgov www.direct.gov.uk 7.18% (versus 4.51%)
4 HM Revenue & Customs www.hmrc.gov.uk 4.31% (versus 4.49%)
5 Vehicle Licensing Online www.vehiclelicence.gov.uk 2.55% (versus 4.39%)
6 10 Downing St www.number-10.gov.uk 2.28% (not listed)
7 Driving Standards Agency www.dsa.gov.uk 1.77% (versus 2.15%)
8 DVLA www.dvla.gov.uk 1.73% (versus 2.1%)
9 National Center for Biotechnology Information www.ncbi.hlm.nih.gov.uk 1.57% (versus 2.13%)
10 National Archives www.nationalarchives.gov.uk 1.55% (versus 1.84%)
Local government websites, individually, all have market shares of less than this – which makes sense. But, like I said, upfront. Tadpole. Big head, long tail. Or maybe that should be long tale. Here’s another view of the tail, from a conference in May 2003, this time looking at pages of content per site (and showing that the bulk of the content is in just a few sites – something like 80% is in less than 5% of sites – although this isn’t a perfect measure as some sites hide content behind passwords or database engines).
Back to the top 10 though. Downing Street’s site makes a rare entry to the top 10, perhaps driven by the change in PM this month (although they were, I think, at the top during the petition against road pricing excitement). I’m fascinated to see NCBI in there – and even more surprised that it is a consistent top 10 entry. Directgov is, though, the only site to show a significant change in market share – increasing nearly 60%. In October 2005, Directgov’s market share was lower still, at 2.8%. It looks like the killer government website might be a site that finds jobs for weather forecasters
Only two government websites make it into the top 100 UK sites according to Alexa. Jobcentre is at 83 and Transport for London is at 92. So a market share of around 10% is good for only 83rd place. Incidentally 10% UK share is 0.01385% of global market share.
There are, then, two strategies for dealing with this.
- The folks at directionlessgov (oh how they must have laughed when they came up with that name. I’d probably have gone for indirectgov – there’s plainly “direction” even if there are those who disagree with it) believe that the solution to the vast number of government web sites is to provide great search, by harnessing google. They’re widely quoted as having spent �60 and 75 minutes developing the site, not to mention harnessing the $168 billion market cap of google. It’s been tuned since then and looks really nice – and it’s also added a post code search capability which is very smart.
- My approach and now also that of the Transformational Government programme, is to aggregate content into “one” site and drive traffic to that through changing the balance of government marketing to promote it heavily. Government is complex – it’s like the most incomprehensible film you’ve ever seen with a convoluted plot (The Good Shepherd anyone) with threads that diverge and never get picked up again. Without a guide, you’re lost. You sit and scratch your head wondering what it’s all about.
Today, both strategies are needed – but should that be the case always? Without a guide, such as the search engine offered by directionlessgov or the aggregation of direct.gov, you’ll struggle to find what you need in government. I like what the directionless guys have done and government departments, like many corporations, need people to thumb their nose at it and show it there are other ways – so these comments aren’t a dig – but it doesn’t solve the real problem which is that tens and probably hundreds of millions of pounds are being spent supporting, maintaining and promoting a vast array of government web sites that rarely see any visitors who almost certainly have very little loyalty – because, for the most part, they don’t get the answers that they need.
The problem with search engines is you have to know what you’re looking for – what the search engine does is tell you where you can find it. So a search for “tax credits” will absolutely take you to the tax credits site (actually, in google, the number one return for this is directgov) but if you’re a new parent and you want to know what government can do for you, what would you search for? Searching for “new parent government uk” takes you to a content aggregation site – parentscentre.gov.uk. Go to direct.gov and you’re one click away from a whole section on what government can do for you as a new parent. Likewise, you’re a click away from everything you need to know about employment, motoring, travel and whatever else you need.
Direct.gov has, of course, a search engine. But it’s deliberately hobbled. It’s job is to keep you inside the direct.gov website so it only returns entries from that site. When we built direct.gov there was a huge internal debate about whether it should do that. I was, for a long time, pro a wider search. With ukonline.gov.uk, we’d had a full pan-government search engine (using a product from inktomi). As I’ve said before we tried to do a deal with google – who, at the time, were extraordinarily difficult to deal with. We ended up talking to Eric Schmidt and Sergei Brin directly before we got any traction. We explored a few options, including using their API and rebadging the results (as directionless does) although the problems with that are several fold:
- We couldn’t promote specific campaigns by influencing the search results to ensure that our chosen campaign hit the top spot (although there are ways to do this with sponsored links, it hardly made sense to pay for such a “benefit”). I can’t even remember, to be honest, how developed ad words were when we were doing this in late 2003.
- Newly posted content wouldn’t be available through search immediately (we’d have to wait for google to index the site and we had no control over how frequent that indexing was)
- Google makes money through advertising and, in theory, had we used them we’d have had their ads alongside our content (directionless strips those which I’m sure is fine for a few dozen searches a week but not so for several hundred thousand or even a few million)
The final deal we worked on was using a “google appliance”, effectively google in a box of its own (one that was made up to look like swiss cheese I think), that we’d host alongside direct.gov and therefore have full control over. We would have been the first overseas user and google had no UK presence, so would have helped us maintain it remotely. The price, in the end, made us baulk – Tom has commented that it would have been money well spent, even for a million or so, but at the time there was no way we were going to do it. So we looked at other deals and ending up working with Verity, now owned by Autonomy (I don’t know if the new direct.gov still uses Verity). We spent a lot of time with Verity, working towards our original aim of searching all of government. At the same time, the direct.gov (actually, it was called the Online Government Store at that time) folks were building their strategy of having a “closed search engine”. Oddly, it was Verity’s inability to perform at the scale we needed – it just couldn’t return the search results quickly enough when we had a full index of the whole of government – that aligned us with direct.gov’s strategy. Verity, despite their best efforts and the involvement of the guy who wrote the engine, couldn’t solve the problem. And so we ended up, both by design and by default, with an engine that searched only within the walled garden of direct.gov.
Plainly direct.gov isn’t all of government. In it’s early days it wasn’t even very much of government, but it has grown now to involve 18 different government departments, all of whom assign people to work co-operatively to join up disparate pieces of government policy and process in a way that makes sense for the customer. It is, of course, still a veneer. Government isn’t joined up behind direct.gov – and it won’t be for a long time but the take up of direct.gov shows that there is demand for it to happen.
As it aggregates more and more content and especially when it bolsters that with transactional services inside the same brand, then the need for a pan-government search engine decreases – because all of the content is housed in one place, in a consistent and sensible way that is easy to navigate. It will never do everything right – the directionless folks note, in parliament no less, at the responses you get if you put the word “goat” into the search engine (google takes you to defra’s goat pox page, direct.gov takes you to a page about pregnancy, and not about goat pregnancy – where it advises you not to eat goat’s cheese).
Who knows what you’re after if you type in “goat” – maybe you’re looking for the law on having a goat as a pet, maybe you’re after a picture of one, maybe you want to know what grants are available if you want to be a goat farmer; maybe it’s a typo and you really meant “boat”. Government websites aren’t going to learn to read minds. And no single site will, any time soon, hold all of the combined knowledge of government that is distributed over those 4,000 sites and however many hundred million pages.
But, for 80% of the people, it’s reasonable to expect that a single well designed and architected site could do 80% of what they need. The rest probably need a good search engine that looks across all of government – and for that, direct.gov could do far worse than link to directionlessgov (after agreeing a deal with google that lifted the cap on search volumes perhaps). I even wrote the pitch for a pan-government search engine, posting it on this blog at the end of 2005 (I wrote it 2 or even 3 years earlier I think) – I ended that post with “Anyway, great search is a necessary part of driving e-government adoption. I just believe that without significant consolidation of websites, it won’t deliver what’s needed”. I still believe that.
In the meantime, direct.gov is doing something right. It’s volumes are growing at a hefty clip, reaching just over 4,000,000 a month in January 2007 (and I’ve heard numbers up to 5 million since then):
In early 2003, I proposed a plan for dealing with the problem as I saw it. I haven’t much wavered from this plan as the ultimate way of dealing with e-government strategy, although I’ve probably written it in a dozen different ways at various times over the last few years.
- Fewer websites not more. Kill 50 websites for every new domain name.
- Less content not more. Delete five (or fifty, or five hundred) pages for every page you write.
- Solve the top 50 questions that citizens ask … and structure your content around those first. Then do the next 50 and the next. The people who know these questions are the ones that answer the phone in your call centres, the ones that write in to your agency and the ones that visit your offices for help; likewise, they visit accountants, advice bureau, charities and so on.
- Test search engines to see how your site ranks – both from a mind share side and for individual queries.
- Impose rigorous discipline on use of “words” – plain speak.
- Impose even more rigorous discipline on the structure of the content, including metadata so that it’s easy to read – by people and by search engines.
It seems to me that executing against this plan will drive out cost from government (freeing up money to invest in new services and capabilities), encourage greater take-up (if every government letter had a single site name printed on it, pretty soon it would stick in everyone’s mind), spark the necessary initiatives to join up the back ends (customer pressure would soon call for that to happen) and so drive out back end efficiencies in government. It would even reduce the environmental footprint of e-government. Wins all round.
I for one have been the benefit of your wisdom over the years of setting-up and running the Directgov (or the Online Government Store as you correctly note). On a couple of points:Latest I have from Directgov is May this year – 5 million visitors, still 3rd behind jobs and weather (and we\’ve always known that!) and ranking 88th on the overall UK rankings.Under the new government shared \’Club\’ infrastructure, the Autonomy K2 (Verity) engine is still used in very much the same way as you describe and for many of the same reasons. There is much talk of the ‘pan-government’ search but I have been sceptical simply as it is often used as a smoke screen to prevent the rest of government from addressing the bigger issues that you outline (too many sites, lack of focus, huge duplication etc.). I do however agree that we should recognise that Government is unlikely to fix its online estate to the extent required in the near future and therefore a broader search is necessary to meet needs of the customer. Whether the folks at Directionlessgov could contain themselves to help rather than poke remains to be seen…Transformation Government, if carried out properly, will address the points you raise. This is as long as:- a holistic strategy is put in place that goes beyond just Directgov and BusinessLink and addresses all government audiences correctly- Government can avoid falling foul of \’protectionism\’ of those sites and servies that while are often are great ideas, do not serve the customerPersonally, I am glad to see the concept of Directgov turned to a reality that is really working and improving. I believed it in the beginning and still believe it can make a real difference.One area it doesn’t cover however is health and the online NHS – what do you think?Gary Ashby
Really thoughtful and helpful analysis. Why the hell aren\’t you still working there?
Did you not have time to write a shorter blog?
Not sure.I think you are shooting indiscriminate thought sperm here. Is that what blogs have become, the thinking man\’s centrefold?
On the pan-government search malarkey:The much-missed Open.gov.uk and the not-very-missed UKOnline did actually provide this very helpful functionality, under the much-maligned \’DotP\’.However, the Cabinet Office removed it – without consultation with users – because they didn\’t want folk to leave the site.So it should be simply a case of switching it back on. Whooops – DotP\’s been scrapped.
Wow – The Good Shepherd – it\’s really as bad as The Good Shepherd?Ouch – perhaps they could learn some lessons from Breach – a vastly superior film.
http://www.jobserve.com/W2AFE8593510725A3.job\”Experienced Technical Architect required to work on a large NHS project and be RESPONSIBLE FOR THE DELIVERY and not just the design on the product in the London Cluster. \”Hahahaha!So the technical architect\’s design isn\’t implementable.Laugh, I nearly went to the Sudan.
Maybe one of the reasons government finds this whole online strategy hard to formulate and even harder to deliver is a flawed concept of its relationship with the people it is serving. Having worked in gov for some time I think we can boil government down to these sets of services1) We provide government information2) Government provides us information3) We pay or receive money as a result of 1) and 2)4) We require some physical service (health care, housing, bin collection)Your article talks about take up and shows some real boasts on hits by direct.gov. But unless you understand the underlying reasons for the hits then it is not possible to see if 5 million hits is impressive or represents a very poor position.The issue is that government thinks people want to have a relationship with them. On the whole they don\’t. There are things we have to do inclusing taxes and arguably benefits (people need money and gov is the only place for some to get it). What government has focused on 2) above by taking the vast array of services, forms, confusing guidance and providing a clever portal to get to it. And in some respects, as a stepping stone this is an admirable thing to be doing. But it tends to be at the expense of the real transformation that is required to turn government services into the modern services we all desire. In essence, I don\’t want to have to transact with government or have a \’relationship\’ good or bad with HMRC. I want the number of things I have to do or send or receive with gov to be reduced and this can only be done by joining up departments in a radically different way to the way they are now. In a world of dwindling funds this work is not going to get done. Short sighted strategy. If they do not have the money to put this right now, they certainly will not have the money in the future when the treasury makes the next round of cuts.Government is a pig and the reaction to the real problem is not to tackle it but to simply put lipstick on the pig. The IT suppliers out there are purveyors of fine lipstick and will certainly be there to help government do the wrong thing (again). Direct.gov is a fine example of lipstick (one of the better shades in gov at the moment) but it is still not transforming the underlying layers where the real problems sit. This is only a problem solvable at a political level and I do not see where the right level of briefing is occurring.
For me this boils down to four things (in no particular order):1) Understanding what is common2) Proper integration3) How you measure success4) Politics1) If each area of government specialises in a particular area, why shouldn\’t they deliver services specific to their speciality? However, rather than blindly reinventing the wheel, there needs to be an understanding about what they have in common with wider government. e.g. The vast aspects of a customer or citizen across government are common, each specific area of government will however have a specific relationship, or piece of information important to them. They should use what\’s common and specialise to meet their own particular needs.2) If government begins to understand what integration actually means and stops listening to vendors who don\’t, progress can be made.3) Success is broadcast across government without any real measure. Is a site that is delivered late, albeit technically working, and sort of meeting the requirements sucessful? Should we not measure success of a website by the behaviour it drives in the customer – you do something and stick around to do something else?4) The politics in Whitehall can be debilitating, why should the CEO of the agency comply with the CEO of the executive agency when they are both the same grade?With solutions in these key areas, Government is ready for transformation in any guise…
Without wanting to criticise Neil, although it\’s probably going to sound like it, I feel I should add the following observation.I\’ve looked at many a monolithic organisation with problems modernising, and they all have basically the same problem. (Some more than others.) This problem can be loosely defined as, having an organisational structure which consists of computers helping various sections do bits of work. The solution has always been IMHO, to reverse the problem and try to get the organisation to the point where the Enterprise Solution is the work, and the people help the computers maintain the consistency. The Enterprise solution needs to completely understand the business. In no other way, can people be made dispensible.Obviously some government departments will find it an easier task, to modernise than others. Departments whose entire model is based around a data registry for instance, already have the data centric model to start with.It\’s interesting to not the increase in job adverts reverting to SSADM (predominantly a data model centric architecture rather than domain model,) and short iterative development cycles.Were I prime minister, I\’d give a single website (say http://www.gov.uk,) the lead, and all departments would have to do what it said, rather than the department telling the website what it did.Until the department\’s in sync with the web, the web is an add on, and the only way to make the department in sync, is to make the department follow the web, and not vice versa. I recognise this will never happen, for several reasons. 1. Empire building.2. The government finds absolute contracts with the public a nuisance, whatever it may say.3. The government doesn\’t want to make it easy to get money out of them, it would prefer it if people who could afford to not be bothered to claim due payment, didn\’t bother.Regards,Ian
I should add to my earlier comment.\”Empire building.\”This was not a value judgement, but rather an observation. I find it no more possible to avoid empire building in careers where it\’s a job for life, (an advantage in quite a lot of civil service posts,) than it is to say, avoid violence at the weekend in Garrison towns.One cannot expect someone to know he\’s stuck with his position for years not to try to increase his control over the environment – any more than one can expect the man who is both interested in a career in, and subsequently trained to commit extreme violence on our behalf, to be able to switch it off when he walks out the camp gates.Ian
So many comments …… this is a short blog, you should have seen the first draft. … thought sperm? you\’ve lost me… NHS etc, why not? but i think they\’re working too many other issues right now. still, with a new health secretary in place, now is the time to present ideas that might prove the quick way to the solution to those problems… the good shepherd as bad as the good shepherd? you\’ve lost me (that\’s twice now). haven\’t seen breach, but the reviews look good.- neil/ian, agree with you both. good ideas. just got to pick one really and then execute against it. ideas are the cheapest commodity we have in government, staying the course and realising the vision is very, very expensive and entirely not a commodity (what does that make it?)
Gary ashby: Whether the folks at Directionlessgov could contain themselves to help rather than poke remains to be seen…Gary, many of us on the web frontline for egov have tried more than once \’to help\’. you end up thinking a slap rather than just a poke is needed.Alan\’s main arguments: * Fewer websites not more. Kill 50 websites for every new domain name.if all you are doing is untying the replication of government bureaucratic structure, fine. if you are destroying communities of interest, no. \’kill\’ is the wrong attitude and hardly customer-focussed. what is a \’website\’ anyway? * Less content not more. Delete five (or fifty, or five hundred) pages for every page you write.this is ridiculous. people want more depth, as shown by poynter and how the web\’s developing. people have always wanted all the detail about a product, including the shipping detail. now they want user comment. * Solve the top 50 questions that citizens ask … and structure your content around those first. Then do the next 50 and the next. The people who know these questions are the ones that answer the phone in your call centres, the ones that write in to your agency and the ones that visit your offices for help; likewise, they visit accountants, advice bureau, charities and so on.what % of the total is the top 50? what about the long, long tail? if you answer them do you actually save even more? this sounds like the FAQ approach to help and, ultimately, usability. you\’re still mediating through officers, not customer-led, plus you can get most everything you need from existing metrics. plus it\’s tasks you should focus on, which is different from questions. * Test search engines to see how your site ranks – both from a mind share side and for individual queries.I\’ll say. what\’s \’mind share\’? * Impose rigorous discipline on use of \”words\” – plain speak.Fine, but that\’s not necessarily going to help your findability. plus don\’t assume all your audience are sun-reading age. * Impose even more rigorous discipline on the structure of the content, including metadata so that it\’s easy to read – by people and by search engines.That\’s right, that\’s a start. You like \’rigorous\’ and \’kill\’ a wee bit too much though maybe?Sorry to be a bit rough on you Alan but having just read Tadpole strategies it sounds like you\’ve wrecked a relationship between government and Google — which we could do with right now.Paul
Ah Paul, you sound like a died in the wool civil servant .. bless. that\’s not an insult, it\’s just a comment on your state of mind. I like words like \”kill\”, \”eliminate\”, \”get rid of\” because they\’re extremes – they provoke a reaction. You might prefer shades of grey, \”reduce\”, \”downgrade\” and so on. For years those words have held sway and the situation on the government web is a direct result of that.As to ruining a relationship with google, it would perhaps be best for you to be even a tiny bit informed. We worked hard to get a deal together with google – first to use the engine in API form, then to use their own hardware hosted at their end, then to use their hardware hosted at our end. We couldn\’t make a deal that worked, despite direct contact with Sergei and Eric. Google had other things on their mind at the time – they weren\’t public, were growing fast, there was no gmail, very little in the way of adwords and so on, so they were working on all of those whilst talking to us. we\’d have had no support locally and limited support from the US – they didn\’t have a corporate division much less a government division. All those kind of things are important to government when signing a deal. So, we passed. Doesn\’t mean the deal won\’t happen again – i\’d be surprised, in fact, if there aren\’t folks in direct.gov engaged in conversation with google every few months, touching base to see what they\’re up to and whether there\’s an opportunity to do something together.sorry to be rough on you too – but extremes are designed to provoke emotion. shades of grey just let everyone smile at each other as if it\’s all understood and everyone thinks the same thing.more generally, i don\’t much disagree with your observations on my action list .. if you track back through my previous posts, you\’ll find that i have espoused many of those views myself. some of these actions are derived from earlier ppts – when it depended on the audience, the projects that i was working on at the time, the maturity of our own offering and so on. nice blog by the way …