Measuring the health of the open source economy

One of Michael Meeks' graphs from his fascinating analysis of the health of the OO.o project

Here’s a fascinating thing. A really detailed analysis of a single very important open source project, Open (OO.o). The author, Michael Meeks, a prominent Open Office hacker, has tabulated and analysed the application’s CVS logs (CVS is the system that stores and organises the code contributed to the project by its many developers). What we have here is a quite fine-grained set of direct, numerical indicators of the health of Open – in particular, of the engagement of the contributing developers (Meeks acknowledges in the post various weaknesses in the data). How many developers are involved, how much code is contributed and which organisations are most engaged. Meeks’ conclusion is that engagement is falling and that Open is sick. He thinks the project needs urgent remedial work to get it back on track.

The reason I find this so interesting is because it offers a preview of the way we’ll monitor the health of all sorts of projects in the future. Open source, as a way of working, is spreading to other areas of activity, some quite remote from software development. If it really catches on – in business, education and the media, for instance – we’ll presumably be able to analyse this kind of data for many different kinds of work. A university might select an open source physics course or a manufacturer an open source component design by comparing graphs like Meeks’ for competing projects. And Meeks’ hand-rolled analysis will inevitably mature into a measurement and presentation dashboard for the whole open source economy.

We’re also learning that data, once exposed, is quickly acquired, tabulated, visualised and compared. Open data is much more useful than the closed stuff. Can it be long before I can fly through a 3D virtual world of open source projects floating in space, coloured and shaped to represent their various critical attributes?

Freeing content at the BBC

I had a bit of a whinge over at Speechification earlier on about the BBC’s content archiving policy. I find it frustrating to say the least that Heather Couper’s epic history of astronomy, Cosmic Quest, which has been running on Radio 4 since May, will now be withdrawn from the public domain all together.

The BBC’s standard line here—and it’s not an unreasonable one—is that the Corporation can only afford to buy ‘first run’ or otherwise limited rights to shows like Cosmic Quest and that if it was obliged to pay for ‘in perpetuity’ rights the additional cost would block the purchase of other good stuff and thus ultimately limit the choice provided to licence-fee payers.

This is undoubtedly true but also defeatist and essentially an inadequate response to the changing imperatives of the network era. The BBC needs to be braver and more committed to change. Here are a few things that could and should be done to unlock more good content for public use:

The BBC should free access to content that has limited (or zero) secondary value. That’s not to say content that’s no good: just stuff that can’t easily be sold on or exploited after it’s been transmitted. Lionel Kellaway’s brilliant Radio 4 programme about Rooks (a favourite of mine) is an asset of great beauty and immeasurable value to its listeners but, let’s face it, hardly any value in an open marketplace for audio content.

The BBC should be a rights innovator: hybrid methods of preserving public access to assets and commercial value to creators and license-holders should be developed and tested on real content. Not easy and not possible without compromise on both sides but the Corporation is uniquely placed to drive innovation that’s beneficial to UK licence fee-payers.

The BBC should be a copyright activist. Legislators should be lobbied to help redesign copyright law to preserve access to orphaned assets: content that’s not being exploited but can’t be freed because it still has a nominal owner. The Corporation should fund work to design use-it-or-lose-it laws and other innovative devices that emphasise access and public benefit over predatory and unfair protection.

The BBC should set targets for freeing content. By defining and prioritising categories of assets that should be freed, the BBC could drive the accumulation of a big pool of useful material held in the public domain permanently. Announcing in advance that certain content categories—perhaps whole channels or strands—are in future going to be purchased for permanent public access would encourage creators to get on with it and adjust their pricing and commercial terms for the new climate.

The goal should be to define and then grow the pool of free-to-use, public domain content archived at and not to apologise for the inflexibility and intransigence of rights-holders and exploiters. The potential gain for UK Plc and UK citizens could be enormous. The “there’s nothing we can do about it , guv” response must be made a thing of the past.

The BBC common platform debate

Last Wednesday’s common platform debate at Broadcasting House was a hit. We talked for nearly three hours plus time in the pub afterwards. Mike covered it (live) over at Techcrunch UK (and I know the event was recorded in some form) and other bloggers have written it up (although at least one was actually watching the football!).

The topic—the BBC’s role in a post-broadcast public service ecology—is clearly going to be a very rich and productive one. Here is my summary of the event’s interesting bits, organised under useful headings. I was chairing the event so I wasn’t able to take proper notes—this is mostly from memory. Please chip in with your own recollections if you were present.

There’s a lot the BBC could be doing.

We’re ten years+ into the network revolution and the BBC’s impact so far has been a raft of quality content on multiple platforms (like this weekend’s quite awesome Glastonbury coverage) and very little else: hardly any of the kind of gutsy innovation the nation and the economy need. We’ve seen cautious incremental change when the circumstances (demographic and economic change at home, rampant growth in our most important competitor economies, environmental change of unknown scope…) demand courageous leaps in the dark.

The BBC is the nation’s most important machine for the production of consensus: nervous, go-slow adoption of common platform goals just won’t do. Tom Loosemore, who used to work at the BBC and now divides his time between Ofcom and The Cabinet Office, provided a handy seven-item list of things the Corporation ought to be sharing which I will now crudely paraphrase and expand (Tom would want me to point out that I’ve added quite a lot to his list in case any of it is traced back to him, causing him to be thrown from a high window in Downing Street):

1. Research. The Corporation produces (and pays for the production of) huge amounts of proprietary audience research, much of it hardly used. This should be made available to business and community, preferably in a useful, nicely-tabulated form.

2. Code. All code written at the BBC should be published under a suitable Open Source licence (Azeem Azhar was conveniently on hand to rehearse his BBC Public Licence idea from 2002). There’s really no excuse for this. It’s not obvious which licence would apply but if there isn’t one out there, one should be invented.

3. Data. The BBC produces and buys lots of data: from TV listings to electoral data. Sadly, much of it is not owned outright and some has even been stupidly given away (like the TV listings gifted to Red Bee on privatisation). What data the Corporation does own outright, however, should be made available freely.

4. Tools. The BBC should be obliged to give away or at least develop nationally-useful tools. Tom’s examples were geolocation and some kind of UK blog search tool. Others came to us in the pub afterwards but I’ve forgotten them.

5. Incubation and investment. The BBC could ‘do a Channel 4’ and seed a whole layer of productive and profitable new media production and technology businesses. Production quotas should be enlarged and a framework put in place to support startups and small businesses in the sector (analogous, I suppose, to the cost-plus budgeting methods used in TV production).

6. Traffic. This could be huge. The phrase ‘trusted guide’ has been current at the BBC since John Birt discovered the Internet in 1995 but it’s never been given meaning. Institutional caution has stopped the Corporation from linking to more than a handful of external sites, and always via a forest of disclaimers. It was pointed out at the debate that the addition of MusicBrainz to the BBC’s music sites adds something like 3 million external links so that’s a start I guess!

7. The Internet. The BBC shouldn’t attempt to augment, enhance or wall off any part of the Internet. The BBC’s endorsement of net neutrality is vital. The Corporation should build on freely-available net tools and services and avoid duplication at all costs.

Rights are a big deal. And they’re not going away.

Outsiders are impatient with the BBC’s rights regime. They want uncomplicated one-stop access to BBC content (and not necessarily for free) but instead they get a spaghetti of overlapping rights owners and regimes. Only the simplest and most directly-owned content can be shared easily: one example cited was Melvyn Bragg’s In Our Time: live speech recorded in a BBC studio with expert contributions paid-for outright and no music. But a tiny proportion of the Corporation’s daily output belongs in the In Our Time category. Even historic and archive material is often encumbered by multiple rights owners (or potential rights owners) and the BBC’s own commercial arm has its own agenda and often has first call on original content.

Two lessons. First: if you’re a tech or media entrepreneur and your business idea needs access to BBC content, think again. Building-in any kind of dependency on the BBC’s rights regime will scare away investors faster than bird flu. Second: the BBC could simplify the regime and persuade (read: force) rights owners to participate, especially for old and potentially neglected material that is unnecessarily encumbered. The key here may lie with legislators who could help by reasserting the BBC’s larger purpose and agreeing to compromise the inalienable rights of content owners and creators a little in return for a more open environment (pay attention Cliff Richard).

BBC new media managers are not the problem

On the evidence of Wednesday evening’s showing, slow progress in building a common platform in Britain is not the fault of the managers building it. Tony Ageh, James Cridland, Jem Stone and other BBC staff present are evidently all passionate drivers of change (and I know many others like them inside the Beeb). In many respects they were the most forward-thinking people in the room: Tony Ageh’s suggestion that he’d like us to be able to right-click on any asset at to get a pop-up detailing its ownership and rules for use was particularly inspiring. Presumably many years spent wrestling with a highly-regulated, historically-cautious tax payer-funded monolith has produced a pragmatic approach to achieving change but there’s evidently no absence of enthusiasm for it.

Channel 4 is ready to help

It’s not all about the BBC. Jon Gisby, Channel 4’s new Director of New Media, spoke with what I’d characterise as cautious enthusiasm about the station’s status as a ‘convening brand’ (that’s a technical term. Look it up) and about its potential to provide a £50M springboard for UK tech and media businesses via the fascinating and so far enigmatic 4IP fund. More than one person used the analogy of Channel 4’s seeding of the British independent TV production business 25 years ago. Gisby nodded.

I’d certainly like to see Channel 4 function as a cheeky and innovative counterweight to the BBC in building out the common platform. The potential is clearly there in 4IP.

The BBC has a history of engineering leadership

All sorts of technical innovations were invented at the BBC and then spread into the wider industry. James Cridland put it like this: “we agree in technology and compete in content”. An organisation accustomed to sharing its production and distribution tech but closely guarding the programmes made with it will inevitably find it hard to shift to a new model. One where technology leadership has been replaced by open source collaborative methods and where content is freely shared.

Tech entrepreneurs don’t care much

On the evidence of our debate there is no real clamour for access to BBC resources from the UK tech startup industry. In fact there’s significant mutual ignorance. Startups don’t know how to access the BBC’s fund of good stuff and the BBC doesn’t know who might want it or in which formats. Entrepreneurs heard at the event said things like: “who do I talk to about access to historic news content?” and “how do I get a commercial agreement for use of programme metadata?” I think for most people—adventurous tech startups included—the BBC is part of the woodwork, practically invisible. The idea that it might function as an enabler for enterprise or community is not widespread. There’s work to be done there.

There were many other fascinating strands to our debate—the BBC’s monopolistic behaviour in some categories, market failure in others, the absence of true APIs and other easy methods of calling on BBC assets, public value tests… to name a few. I’ll try to return to some of them here. Others have covered the debate elsewhere. I’d like to see a follow-up meeting soon. I’m also planning to put up a web site (probably a wiki) for the further discussion of these matters. Do let me know if you’d like to help.

Thanks to Mike Butcher for putting the considerable weight of TechCrunch UK behind this. It was his initial post that got things going in the first place. Thanks to all the panelists, all of whom have important jobs to do and better things to be doing with their Wednesday nights. Thanks in particular to James Cridland who organised the venue, catering and lots of other stuff for us, as well as being a trenchant panelist and defender of the BBC’s honour! Thanks finally to attendees and bloggers who made the thing lively and interesting!

Government content? exciting, huh?

Mike Butcher blogs a deal between the UK Government and MSN to offer Government content to MSN users. Mike draws out an analogy with the recent discussion of Azeem Azhar’s BPL idea. I think the comparison is valuable but the big difference it that so much Government content is paralysingly boring so it’s unlikely that the content industry will throng to repurpose it. Most interesting is the potential for MSN (or anyone really – preferably Google) to offer a better interface to Government content than the Government currently seems able to do. See my whinge about NHS Direct’s dire interface to its own extremely important and useful content.

BBC Online under the microscope

A long overdue enquiry into the BBC’s investment online should be a good thing for all parties but it must strike a delicate balance. If it turns into a mugging for the corporation orchestrated by its competitors it will not serve the interests of industry or citizens. Likewise, a whitewash that leaves the BBC’s hugely out-of-proportion investment unexamined will not answer vital questions about the proper role of a public service broadcaster in the networked era. If this enquiry is real, it presents an unlikely-to-be-repeated opportunity to straighten out the regulatory and funding context for BBC Online and to set some goals:

  • Hub for a new online content industry. The BBC’s massive investment in online content and infrastructure should stimulate a new downstream ecology of content and application creators – an online ‘indie’ sector like the one brought into being by Channel 4 in the 80s would be a good thing.
  • An engine for participation. BBC Online should invest in a new generation of content and applications that promote participation, connection and creativity amongst wired citizens – not just programme support material and one-way content.
  • Public service online. If BBC Online is to assume a statutory public service obligation (as it should), then the burden should be shared by other online businesses – and until the funding climate for private sector net businesses recovers, Government money may have to be made available to support them.

BBC Online should not fear the inquisition. It’s likely to be critical and may close off some of the department’s activities but the current uncertainty is more damaging. Lambert’s report on News 24 is hardly flattering but, by taking the project seriously, it has secured the channel’s future nonetheless. A close examination of BBC Online should have the same effect.

Expanding on the three goals:

  • Hub for a new online content industry. Azeem Azhar thinks he knows how to jump start this downstream ecology. He’s proposed the application of the GNU General Public License to BBC Online’s content and code. This is fresh thinking from the Internet’s intellectual property laboratory and I hope it’ll be a part of any serious inquiry into BBC Online’s future. To learn about Azeem’s idea, start here. I blogged the idea here and here.You could probably achieve the goal of opening up a secondary content industry around BBC Online using established methods like commissioning quotas but Azeem’s idea cleverly exploits one of the net’s efficiencies.
  • An engine for participation. An ambitious cadre within BBC Online is already working on applications like this. This serious-minded group could do the job if required to. I think a benefit of an enquiry would be to throw the spotlight on good work already being done by the corporation.
  • Public service online. I get more grumpy by the day about the way the Government continues to evade its obligation to help define what a ‘public service’ BBC Online should look like. The latest evasion is going to be embodied in law – the Communications Bill, now at second reading stage, explicitly forbids Ofcom to regulate the Internet, thus making it almost impossible for Ofcom to make a useful contribution to the debate. I’ve blogged this before here, here and here.

Problems with Azeem’s BPL idea

Azeem’s BPL idea will encounter many obstacles on its way to the mainstream:

1. How far downstream does the BPL go? If you require content and app developers to embed the BPL in all derivitive product (as the pure GPL requires), there is no limit. This will alienate businesses who don’t want their work to inherit the BPL. It would be better to allow developers to use BBC material without publishing their own source – a sort of one-way GPL that would permit bigger, more conservative organisations to play.

2. The whole thing is going to be extremely hard to explain to almost anyone, let alone to BBC Governors, management, regulators and media. It’s easy to imagine the project going nowhere if entrenched interests succeed in characterising it as something geeky, something to do with computers – or, worse, as some kind of weirdo collectivism, detached from reality – “meanwhile, back in the real world.” How would it play in The Daily Mail and the rest of the Conservative media, already hostile to the Beeb?

3. Competitors – many badly knocked around by the crash – will only approve if the effect of the BPL is to reduce the BBC’s overall share of audience. The scheme should be engineered to achieve this, not to cement the BBC as the sole source of quality content and code in the UK or as the hub of an emerging content network.

4. As a starting project Digital IDs are tricky. Anyone issuing hard IDs like the ones envisaged by Azeem will be perceived as an arm of government. No one would believe for a minute that there were no Government-mandated back doors. It might be better to stop short of hard authentication and start with credentials: ‘I’m over 18’, ‘I live in the UK so I’m entitled to get BBCi content for nothing’, ‘I’m under 14 so I can enter the CBBC Chat Rooms’… These simpler IDs, if widely adopted, would be a trojan horse for the real thing.

Redefining ‘Public Service’ at BBCi

Azeem thinks we should try to apply open source thinking to the BBC. He thinks the Beeb’s online content and code should be freely published under the GPL – the radical constitution of the copyleft movement. The effect of this – if it worked – would be to bring into being a thriving new ‘creative commons’ downstream of the beeb, built on the BBC’s stock of content and application logic. This might just be the boost that UK Online needs to beat the bust and overcome the natural pessimism produced by nearly three years of market misery. More important, it might also represent the first serious attempt to update the definition of ‘public service’ for the networked era.

So why is this interesting? Isn’t open source just a geek fad? Actually, I think it might help us advance the debate about the BBC in the digital era. Arguments about the BBC’s role – the charter, the license fee, public service vs ratings etc. – are especially dry and boring these days. With Dyke in charge, Labour in power and OfCom barred from regulating the Beeb directly, the corporation is more-or-less bulletproof. Even Rupert Murdoch’s ‘untouchable‘ outburst struck a plaintive note. Open source might short-circuit these old-world arguments and help us get a productive argument about public service in the twenty-first century going again.

Azeem’s idea is focused not on ownership (privatise it, usually – yawn) or on output (cut it back to an explicitly public service core, privatise the rest – double yawn) but on creation. By promising to stimulate the online creative economy in the middle of the nastiest crash in recent history, an Open Source BBCi might bring to life a whole new ecosystem – like the independent TV production sector that rallied around the new Channel 4 in the eighties. If it works, we’ll have ourselves a useful model for the redefinition of public service in other areas of the Beeb’s output and perhaps for Government investment in interactivity – ‘Broadband Britain’, UK Online and so on – in general.

Networks need diversity

GPL zealots play with fire

Open source lobbyists are pushing laws through various legislatures to prohibit Governments from purchasing software from companies who won’t publish source code. Bill Gates, naturally, sees this as a threat and will roll out his biggest guns to prevent it from happening. In this scenario, pointless and destructive conflict is inevitable. The zealots (and the monopolists) need to understand that monocultures are bad for almost everyone. The networked economy’s complex ecology needs diversity. Open source and proprietary code will co-exist and the tension created will be productive.