Now posts ↓

Wednesday 25 April 2007

Questioning stats

The BBC carries a virtual press release from Comscore, Web counting tools 'need change'.

Key bit:

In comScore's study, an analysis of 400,000 home PCs in the US found that a hardcore minority of web users are clearing their cookies from their computers on a regular basis.

This causes servers to deposit new cookies which in turn could lead to an over-estimate of unique users to a particular website.

It found that 7% of computers accounted for 35% of all cookies, which extrapolated could mean the size of a site's audience is being overstated by as much as 150%, said comScore.

"It is clear that a certain segment of internet users clears its cookies very frequently. These 'serial resetters' have the potential to wildly inflate a site's internal unique visitor tally, because just one set of 'eyeballs' at the site may be counted as 10 or more unique visitors over the course of a month," explained comScore president Dr Magid Abraham.

Perhaps unsurprisingly, comScore offers a very different approach to audience measurement - using the panel-based system favoured by the TV and radio industries which relies on using a representative sample of net users to gauge behaviour.

The problem being not just the claims but this very strong statement in response to these claims of 'the potential to wildly inflate' by the industry body, the Interactive Advertising Bureau (which isn't on the BBC):

The goal of the IAB and the entire Interactive industry is simple: to achieve transparency in audience counts and to revise out-of-date methodologies.

For the Interactive industry, one that is committed to delivering accountability, integrity in audience measurement is a fundamental necessity. But, despite a multiplicity of reported discrepancies in audience measurements, comScore and NNR each has resisted numerous requests for audits by the IAB and the Media Ratings Council since 1999.

In order to establish the source of these discrepancies, and to find the potential solutions, the IAB is asking that both comScore and NNR obtain audits of their technologies and processes by the Media Rating Council (MRC).

‘We simply cannot let the Internet, the most accountable medium ever invented, fall into the same bad customs that have hindered older media and angered advertisers for decades,’

Which kindof undermines any numbers from comScore but also puts the industry's approach in context - comScore has their approach to flog.

This, I'd suggest, should also feature in a BBC technology correspondent's considerations:

Or this, from Steve Rubel:

Comscore Clings to a Page View World

I have no reason to pick on the fine folks at comScore Media Metrix. However, despite some recent indications that they want to change, it seems as though they are clinging to the days of yore when hits were all that ruled.

Consider this analysis published yesterday by Ars Technica. The piece reports: "comScore has said that they are working on new metrics that will also take into account the trappings of Web 2.0, including interactive AJAX-driven web pages which do not necessarily generate page views." That doesn't sound like bad news, right? Wrong.

Further down in the piece Dr. Magid Abraham, President and CEO of comScore Networks, added: "While page views will not altogether cease to be a relevant measure of a site's value, it's clear that there is an increasing need to consider page views alongside newer, more relevant measures." Abraham, however, doesn't say what that solution is. The reason could be such metrics could have severe ramifications for comScore's business model, which feeds off a hit-driven economy that's dying.

Comscore needs to wake up and realize that we're in a Long Tail world where top 10 lists matter less. Marketers want to know about the influence circles within the niches that matter to them - and those niches are often tiny. The time is now for comScore to open up to the little guy.

Quantcast is going to eat comScore's lunch. They recognize that partnering with the crowd is essential to measuring it. Comscore seems to slow to adopt to this model and it's highly possible they will become irrelevant in this world if they don't change fast.

What the BBC's report also mentions is debate about time spent on sites:

RuneScape - 6hrs 32mins
Electronic Arts Online - 3h 07m
Bebo - 2h 37m
Facebook 2h 28m
eBay - 1h 55m - 1h 53m
Adventure Quest - 1h 35m
Fox Interactive Media (MySpace) - 1h 11m
Club Penguin - Ih 10m
Cartoon Network - 1h 09m
*Source: Nielsen
But again, someone has something to flog:
Page-views metrics discriminate against sites with audio and video content and Nielsen/NetRatings argues that metrics based on the time spent on a website could be a more accurate method.

You can understand why the IAB sound so annoyed with NNR/comScore ... all this noise, which the Beeb adds to, distracts from the actual standards agreed by all.

No comments:

Post a Comment