My work on critiquing the British Medical Journal (
BMJ)'s research article last week on search and suicide has now been published
on PublicRadar, in an article form.
I'm republishing it below as it's more digestible than
the original post.
Been much boosted by positive responses from several psychologists who use the web, SEO and SEM full-timers and others who know this stuff far better than me.
As well, I was blanked for a week by the BMJ but am now waiting for either a response as to why they won't publish my criticism or if they actually will.
Unfortunately the BMJ PR went around the world and was published in a lot of media. They really need to unpublish the research article and issue PR explaining why but as one psychologist correspondent told me, "the BMJ is in a very conservative world where they still think that hiding or ignoring negative comment works".
How this serves medical practitioners, let alone the suicidal, completely eludes me.
~~~~~~~~ April 17th, 2008 PublicRadar
Last week the British Medical Journal (BMJ) published research headlined ‘Suicide and the internet‘.
Recent reports of suicide by young people have highlighted the possible influence of internet sites. Lucy Biddle and colleagues investigate what a web search is likely to find.
Those recent reports are of the Bridgend, Wales suicide cluster. Shock horror newspaper headlines often pointed at the web as a reason why a bunch of young people might have killed themselves or at least encouraged them. Of course, virtually none of the media reports looked at the research which has repeatedly shown that the media itself, particularly when it describes methods, has been demonstrated to encourage so-called ‘copycat’ suicide, but that’s for another day.
What Biddle and colleagues from Bristol, Oxford and Manchester universities claimed to find, and what the BBC and media around the world subsequently and dutifully reported, was that it’s search engines that encourage suicide. Methods are “easy to find” - and they’d proved it.
“The three most frequently occurring sites were all pro-suicide, prompting researchers to call for anti-suicide web pages to be prioritised.”
The trouble was the research was flawed in its methodology and Biddle et al aren’t experts in how search actually works. Worse, ways in which pro-suicide websites can be countered online are relatively easy and the methods used well established - so how this ‘prioritisation’ might happen wasn’t explained.
Starting with their idea of “most frequently occurring sites”, this is nonsense as the vast, vast majority of searchers don’t get past the first ten results and most of those don’t get past the top three.
So to rank, as they did, the ‘top ten’ searches as having equal value is false. This alone discounts their findings.
They also counted from searches on four search engines. Yet fully 86% of UK searches are via Google - they’d again counted each search on each search engine as having equal value.
As well, they gave each of ten search terms equal value when those terms have vastly different uses. Using a keyword suggestion tool, you can see that ‘Suicide’ has a daily UK search number of 7788 whereas one they picked, ‘Most effective methods for committing suicide’, has 0. This tool also throws up terms they didn’t use, such as ’suicide poetry’.
You can actually see this vast different in usage recognised by commercial sites as well as church and other small charity bodies who pay for advertising next to search results (’Cremated ashes made into glass: “Keep the memory”‘). They will naturally only pick the terms of most value.
Most tellingly they failed to understand that two terms they used ‘Methods of suicide’ and ‘Suicide methods’ are exactly the same term because ‘of’ is discounted.
In their paper they state that their ‘Search strategy’ was:
“To replicate the results of a typical search that might be undertaken by a person seeking information about methods of suicide. We conducted searches using the four most popular UK search engines and 12 broad search terms—a total of 48 searches. The terms entered were those likely to be used by distressed individuals, determined partly from interview data collected in an ongoing qualitative study of near-fatal suicide attempts and by using search suggestions provided by the engines upon entering terms such as ’suicide.’”
There isn’t any further detail on just how they could know what search terms were actually entered ‘by distressed individuals’ as opposed to ones without distress or how relevant ‘interview data’ would be in working that out.
If they had even bothered to ask some of the search marketing/search optimisation specialists probably around the corner from them, or possibly even within the same universities, they would have realised that their methodology doesn’t show anything. But as a result of this article being in the hallowed BMJ we now have web-bashing headlines around the world.
I would suggest that this article devalues the BMJ itself as a source of scientific information unless it is withdrawn. There was nothing scientific about this study.
It is notable that in a comment on my blog Graham Jones who runs the Internet Psychology web site said that he’d met some people who were connected to the research and “they were suitably embarrassed in private when I pointed out the simple flaws in the research”.
Biddle told the BBC that:
“This research shows it is very easy to obtain detailed technical information about methods of suicide.”
Her research did not demonstrate that finding pro-suicide websites is “very easy”. But, yes, if you are determined to find it you will find it. Just like the determined can find bomb making recipes and the Chinese in China can find rants against the Chinese government.
But this is not how most people operate, which you can see from what gets typed in most frequently.
Contrary to assumptions, I haven’t seen any evidence that shows that kids and teens are that much better, if at all, at finding things online using search engines than anyone else.
What I can say is that, using Google Trends, which covers searches going back four years, for general searches for ’suicide’ (tweaked to exclude unrelated Iraq/Afghanistan ’suicide bombers’ searches), the trend is clearly down. There’s some good news you won’t read in reporting.
Apart from no numbers on what the actual usage is of ‘pro-suicide’ sites, another point is whether the websites which charities and government create are actually helping kids and teens. I don’t know but I’d like to - there’s nothing about that in this research, why some kids and teens might be turning to these pro-suicide websites instead of ‘official’ ones.
The BBC quoted Marjorie Wallace, chief executive of the mental health charity Sane, saying the proverbial ’something should be done’. There’s actually a lot which could be done. For example:
1. How about running text ads next to more search terms than just ’suicide’.
2. Or employing some Search Engine Optimisation specialists to make sure that your pages come up first. They might even do it for nothing or just the publicity.
3. Or working with other charities to make sure you cover every possible term and intervene via content and ads on other sites or through social networks (simply creating a page on Mind’s website, already high-up results, which is titled ‘How to commit suicide’ would immediately help).
4. Or fixing your own websites, not only to make them more appealing to your target audience but also to fix errors such as the first result on a search for ’suicide’ on Sane’s website being ‘The National Suicide Prevention Strategy report’.
On one term you can see how smaller, more agile bodies aren’t moaning but are learning how to use the web to their own ends. ‘How to commit suicide‘ includes a top result which redirects people to an anti-suicide web page.
My issue with the research and its reaction is that the complainants are making no effort to ensure that their pages turn up tops on such search terms as ‘How to commit suicide’ . There are no excuses for this and to behave as if this is someone else’s responsibility - let alone ISPs - is childish and pathetic
Simply put, it is not ISPs but health practitioners, charities and government who are not doing their job properly online. They are failing the very kids and teens they claim to be helping and looking for someone else to blame - there’s an abdication of responsibility.
This is exactly what happens when you set up online walled gardens and fail to relate to the wider web - I am not seeing the NHS or government portal directgov anywhere in these results - and that ‘can’t be bothered’ ‘it’s all too complicated’ mentality apparently dominates the charity sector as well.
Hardly surprising when the ‘National suicide prevention strategy for England‘ contains no mention of either the web, the internet or even chatrooms.
Here’s what could be done:
1. Talk to the search engines, they are very interested in getting results right and can and do ‘tweak’ them. They won’t ‘censor’ sites or stop indexing the whole web but they will help and advise on improving positioning.
2. Don’t talk to the ISPs! Talk to the search engine experts such as the Search Marketing Association.
3. Talk to social networks about teaming up with them and others to create widgets and other tools so kids and teens can help others.
As well, these people would potentially do it for free or cheaply or for publicity. It would be very straightforward to out-manoeuvre the pro-suicide websites - what resources do they have vs. what resources do you have?
Research on how search may contribute to actual suicide may well be valuable, but it needs to be done by specialists who can use the tools established by the industry to track and analyse which sites are the most dangerous and where the traffic to them is coming from - it may well not be primarily search. That could be done. But the best course is a concerted effort by charities and government to direct ‘distressed individuals’ to websites which can really help them.
I am afraid that none of these people are listening though. What they are instead developing is an righteous effort, like has happened in Australia, which will end up in a censored Internet for all of us - and no real help for those they claim to be helping.
~~~~~Postscript: Published in BMJ with one edit.
