Tuesday, April 3, 2012

Social Media Infused Economic Trend Analysis


In the article, The Spy Who Tweeted Me, Sharon Weinberger explores how the Intelligence Community wants to monitor social media in an effort to find trends that will aid in predicting the future. Its proposed uses are to predict political events or economic disruptions. Although this is already done, this project will be the first to incorporate social media into the mix.


The idea, coming from IARPA (Intelligence Advanced Research Projects Activity), is to mine big data from social media websites in addition to more traditional sources. One of the proposed predictions that may be fathomable from this economic trend analysis are societal events like a popular revolution. Initial research has found that social networks, financial markets and internet traffic are useful in early detection of macroeconomic trends and political crises.


  • · Outcomes can be predicted through certain patterns
  • · Requires in depth thought on what contributes to the outcome
  • · History often repeats itself and this method may be able to learn/utilize it


  • · Not all data is publicly available
  • · Requires high computer power
  • · Can be time consuming
  • · Can be difficult to find the right indicators
  • · The outcomes are hard to interpret with such big data


Economic Trend Analysis is a complex analytic tool that is growing in usage and those interested in it. Incorporating social media data may boost its effectiveness at finding patterns and trends related to economic outcomes. However, the data chosen must be public in order to keep the public happy that their privacy is not being breached. Furthermore, the US is only looking to use this technique externally and could be useful in spotting the early warning signs of an even such as the Arab Springs uprising.


Weinberger, Sharon. The Spy Who Tweeted Me. Retrieved from; http://www.wired.com/dangerroom/2011/09/social-media-spies/


  1. The New York Fed has already announced that they are mining search data to forecast economic trends. The Fed said because there is a substantial lag in getting economic data they miss important trends. Using search data they hope they can catch trends quicker.


  2. There's a similar thing called Sentiment analysis, which some prediction markets have been using to (rather effectively) predict things like sports games' outcomes and such. This -looks- like basically sentiment analysis for economic trends, which I suppose is the next logical step for the IC to step into a future filled with social media that isn't going away.