Posts Tagged ‘analysis’

Attention All Analysts

Jul 2009

10

To whom it may concern,

 

I have noticed that for a long time (but especially recently), you’ve been picking up dirty habits. Considering there is no good system for vetting you, I would like to address these two issues here. Additionally, I’ll give a couple prime examples of your large blunders as well as a current example.

 

extrapolating Attention All Analysts
(Source: xkcd)

 

Extrapolation is Dangerous
Ex. 1: The value of radio stations continued to rise in the 1990’s. Corporate behemoths like Clear Channel saw the growth and thought they could buy up all the stations and have control over the airwaves. Due to the rise of the internet and the changes in the market, radio is dying and Clear Channel’s creditors are trying to force the company into bankruptcy.
Ex. 2: In the late 1990’s, AOL was a media super giant. Its valuation was so high that it allowed it to purchase Time Warner in January of 2000, when the sky was the limit for AOL with subscriber growth through the roof (deal finally closed in January 2001). Since 2002, AOL’s subscriber base has not seen one quarter of growth. Time Warner has announced that AOL will now be split into a separate company by the end of 2009, fully breaking ties with AOL.
Ex. 3: Facebook and Twitter have each seen exponential growth. Facebook recently turned down a valuation of $8 billion, and Twitter was valued at $250 million in January (Facebook tried to purchase it for $500 million).

 

correlation and causation Attention All Analysts
(Source: Stephen R. Johnson)

 

Correlation does not imply Causation
Ex. 1: People who drink coffee have health problems. [No, people who drink coffee also often smoke and are workaholics. Those people do have health problems.]
Ex. 2: Eating breakfast is highly correlated with success for students. [No, people who miss breakfast are often absent or tardy students, or parents who make their child's breakfast are more involved in their lives (read: schoolwork, success)]
Ex. 3: As ad spending on a product increases, so do its sales.

 

Analysis doesn’t just take into account numbers and graphs with lines pointing in one direction or another. A good analyst will look at the other (sometimes hidden) variables and shed some light on whether the data shows mere correlation or actual causation. To that end, an analyst has to keep in mind that there are thousands of variables that will change over a one, two, five year period, which is what makes extrapolation so dangerous. Predicting Facebook’s pageviews in 2013 is no more certain than predicting AOL’s subscriber base or the radio industry’s growth. Just because a pattern exists doesn’t mean that it will last. Or that it isn’t just coincidence. Keep that in mind the next time you write a report.

 

Please. Friends don’t let friends extrapolate or imply causation. It’s just poor form.

 

Sincerely,
Jaremy

 

Links from @jaremy Tweetstream May 17-30 2009

May 2009

31

I missed last week’s update, so here is the past two weeks’ Tweetstream links.

 

CategorySiteDescription
Gaminghttp://bit.ly/bEenuTiger Woods PGA Tour iPhone review
Gaminghttp://bit.ly/p7BInMass Effect 2 teaser video
Gaminghttp://bit.ly/pjH3BinFAMOUS comic-style marketing campaign on IGN
Gaminghttp://bit.ly/17QPkWLionhead Studios to present at Microsoft's E3 presentation
Gaminghttp://bit.ly/13RcTFTeam Fortress 2: Meet the Spy
Gaminghttp://bit.ly/f7iIHDuke Nukem Forever's development team canned
Gaminghttp://bit.ly/envQQE3 Floor Plans
Gaminghttp://bit.ly/37ovmPopCap VP heads to Blizzard
Gaminghttp://bit.ly/1aw8sOTrailer for Call of Duty: Modern Warfare 2
Gaminghttp://bit.ly/XYi38Natural Selection 2 trailer
Gaminghttp://bit.ly/1aBrqqDell exits the gaming scene
Gaminghttp://bit.ly/W3nZNLeaked videos for Assassin's Creed 2, Splinter Cell: Conviction
Technologyhttp://bit.ly/11ybwvApple considering enabling background apps on iPhone
Technologyhttp://bit.ly/iJTNYYou can close apps in Windows 7 with Mouse3
Technologyhttp://bit.ly/tNvbnGoogle Wave preview - revolutionary or just cool?
Technologyhttp://appft.comThere's an app for that - very funny Apple parody website
Technologyhttp://bit.ly/4Ri5MGoogle automatic email message translation
Technologyhttp://bit.ly/18xIHdFans trying to bring back Polaroid film
Technologyhttp://bit.ly/169McCZune HD official announcement
Technologyhttp://bit.ly/Uvb8nMicrosoft Bing hands-on
Analysishttp://bit.ly/11BpELMaster your own social media data flow
Analysishttp://bit.ly/19AJOeRandy Johnson has won 249 games since turning THIRTY years old
Miscellaneoushttp://bit.ly/1PD46Danger Mouse's new album will be released entirely online (through torrents)
Miscellaneoushttp://bit.ly/XZtTnRunPee website tells you when to pee during movies
Miscellaneoushttp://bit.ly/1as8iHA mug for pessimists
Miscellaneoushttp://bit.ly/kOPpjWatch MST3000 24/7 at Justin.tv
Miscellaneoushttp://bit.ly/FpIxGFormer employees' recycled business cards

 

Technorati Profile

 

But who analyzes the analysts?

May 2009

25

sales But who analyzes the analysts?Every day there are dozens of analysts that predict huge sales numbers* or that make claims like Facebook Could Surpass Google by 2011. These predictions make it out to multiple media outlets, are blogged, tweeted and mentioned by hundreds of other outlets, and are discussed at great length. Additionally, there are thousands of analysts who make their living by making their own predictions.

 

The problem is that by the time the numbers actually come out, the predictions have been forgotten and the results rarely get reviewed.* So what’s to stop an analyst from making bold predictions when there are little to no repercussions? Aside from common sense, it doesn’t seem like very much.
*Full disclosure: I am an analyst. Predictions I’ve made are rarely openly vetted after the fact. Yes, I am aware of the irony of this rant.

 

I would love to see a website that takes analyst reports (whether the topic is gaming, social media, finance or sports) and actually vets the analysis. This is not to say that there aren’t analysts who do a good job of making conservative estimates based solely on facts. However, for the general public, it’s hard to differentiate between factual analysis and predictions based on a large number of suppositions.

 

To analysts: it’s time to stop sacrificing credibility for popularity. Anyone can make an outrageous assumption for the sake of appearing in headlines. That shouldn’t be the motivation. To news outlets and blogs: if you’re going to go through the process of printing analysis, you owe it to yourselves and to your readers to report on the results after the fact. If analysts are consistently wrong or off-base, their reports shouldn’t be published just for the sake of being newsworthy.