The question of how much phishing costs business in an important one. I am personally interested in it because I have recently been speaking of my research in phishing at various conferences and also because my clients often reach out to me for advise on the topic.
In December 2007, Gartner issued a press release stating that phishing cost businesses approximately $3.2 billion (based upon survey results of about 4,500 adults).
I concede that I have often leveraged the Gartner claim of $3.2 billion to make my case on why phishing is a major problem - it immediately gets the audience to understand and accept the fact that the phishing ecosystem is an incredible menace that must be dealt with.
In January 2009, Cormac Herley and Dinei Florencio of Microsoft Research published A Profitless Endeavor: Phishing as Tragedy of the Commons. In this paper, Herley and Florencio systematically and methodologically state the case for why they think the cost of phishing is not $3.2 billion annually as claimed by Gartner, but more around $61 million (section 4.4).
Even though I have leveraged the Gartner research in my presentations, I nodded in agreement as I read the hypothesis and the reasoning offered by Microsoft Research. I remembered going through the vast amount of underground message boards where phishers and scam-artists convene, noting how much of a constant struggle it was for the phishers to monetize (including cases where phishers attempted to scam other phishers), and wondering how it is that such a struggling system could correlate to a $3.2 billion loss.
The factor of difference between the Gartner and Microsoft Research numbers is x50. That's right: fifty.
It is important to understand that the claims by Gartner and Microsoft Research are scientific claims - they should be based upon reason and evidence. Even though it may not be possible to arrive at the hypothetical end-solution of an absolutely exact number, the goal of the exercise it to tend towards a reasonably accurate estimate.
My appreciation of the Microsoft Research publication stems from the scientific discourse utilized to make the arguments - reason and evidence, systematically presented. What did Gartner have to say in response? Here is a quote from a Dark Reading article:
Avivah Litan, vice president and distinguished analyst of information security and risk at Gartner, says the researchers' paper is more of an academic exercise than reality. "They are assuming their economic theories apply here -- there is no hard evidence that they do," Litan says.
I am extremely puzzled by this stance. Why wouldn't economic theories apply? After all, we are debating dollar figures here, aren't we? Herley and Florencio walk through well-reasoned arguments for their calculations. If Litan isn't in agreement, I feel she should be more specific on her stance - what exactly doesn't she agree with and why?
Here is another quote from a recent zdnet article:
"It's very misleading for the authors only to look at the phishing industry without looking at the malware business," said Litan. "In fact, it renders their entire economic argument meaningless."
Even though Herley and Florencio do not specifically mention malware in their paper (except for once, in a different context, page 8), their hypothesis on the dismal economics of phishing still stands (I am assuming that Litan was referring to malware served by phishing sites).
The original Gartner press release makes extraordinary claims: $3.2 billion is not an estimate that should be taken lightly by anyone. Extraordinary claims require extraordinary evidence (quoting Carl Sagan). As I read through the Gartner press release, I felt that the claims were unsupported because, besides the fact that a survey was conducted, it does not reveal the methodology used to arrive at the specific claims (and then there's Herley and Florencio's bias argument, section 4.2.1).
The Gartner press release, seems in essence, to be based on authority: here are the facts and they are true because we are a reputable brand and because we say so. There are only a few people that can get away with forming arguments based on authority - one of them is pictured below.
In all seriousness, I am sincerely excited about the ongoing conversation on the real cost of phishing because it is something that advances our knowledge. The point is not for Gartner or Microsoft Research to have the final say - the goal is to have a conversation so we all arrive closer to what is true. Here is a quote from a Richard Dawkins' speech that illustrates my sentiment:
A formative influence on my undergraduate self was the response of a respected elder statesmen of the Oxford Zoology Department when an American visitor had just publicly disproved his favorite theory. The old man strode to the front of the lecture hall, shook the American warmly by the hand and declared in ringing, emotional tones: "My dear fellow, I wish to thank you. I have been wrong these fifteen years." And we clapped our hands red. Can you imagine a Government Minister being cheered in the House of Commons for a similar admission? "Resign, Resign" is a much more likely response!
I am excited that we are indulging in such important conversations in the information security, but I sincerely hope that we keep ourselves in check, and that we continue to press for critical thinking and encourage scientific discourse.
Getting back onto the original topic, it could very well be that the Microsoft Research paper includes errors, yet for now, I have not come across any well-reasoned counter-arguments that influence me otherwise. I would welcome any additional comments from Gartner - but be aware, they would have to qualify my "is this argument based on reason and evidence?" filter before being accepted.
