Making Sense of the Chatter on Twitter

The formal definition of the word trend goes something like…  “a general direction in which something tends to move,  a tendency or inclination to veer in a specified direction”.  There are countless organizations from television to retail giants to the corner grocery store that seek out information on current trends. The Neilson Ratings are an indication of the trends of what is most popular on television. Businesses seek out trends in buyer behavior to determine the next wave in fashion, or what items to stock on shelves.  The elusive goal, that Holy Grail is hitting the mark on the next one, doing so can allow business to reap profits. The long-standing challenge of course is accurately gauging and measuring trends.

#TradingTwitter has been a hot topic recently. The chatter on twitter, blogs, even CNBC has reached peak levels. The idea of trading based on trends is not new by any means, nor is the idea of leveraging twitter, news, or other sources of economic and social information. What has changed is the concrete evidence of the correlation between the global psyche as represented through twitter and our markets. Specifically, Professor Johan Bollen’s work at Indiana University marking the predictable interrelationship between the indication of twitter’s public mood and the DJIA.  So convincing are their six dimensional tracking techniques of the human state-of-mind that  DerwentCapital is soon to launch using their modeling.

As with most new ideas (or trends) there are of course the naysayers.  There are those that contend that abusive behavior on twitter could lead to market manipulation.  While the notion of news-based manipulation is certainly possible, understanding twitter reveals itself as a model based on the social or historical sciences not one based on the classic scientific method of empirical evidence such as physics, chemistry or molecular biology. As such it should be judged, critiqued and leveraged in an entirely different manner. The techniques used in the goal of attaining new knowledge in any scientific field include methodology, causation and prediction.   The chief method to gain new understanding and knowledge in the classic sciences is through repeatable laboratory experiments. Chemical engineers, physicists and biologists alike obtain quantitative data through controlled repeatable experimentation, often varying just one or a few parameters between runs. Using such techniques they attain new insights and predict future outcomes.  Sound familiar? These same empirical cause-and-effect techniques are employed by quantitative firms in the alpha discovery process. Whether engaged in high frequency, long-short market neutral models or other trading styles all generally follow this classic experimentation process.

In the social and historical sciences laboratory experimentation is of little importance.  It’s impossible to reverse time and replay great migrations, ice ages, extinctions, or repeat horrific weather events or natural disasters such as earthquakes and their economic and financial impact.  Instead the science of society and history is one of observation and causality. Jared Diamond a well-known author makes this distinction,  “… the concepts of ultimate cause, purpose and function while meaningless to physics and chemistry are essential to understanding living systems and human activities in particular“.  The Treaty of Versailles when viewed myopically signaled the end of hostilities of the Great War, yet on a grander scale it was actually one of many events in a causality chain that eventually lead to an even greater world war just a few years later.  No one could have predicted that outcome in 1919.

The tools and techniques to judge human behaviors and determine sentiment have attained a high-degree of sophistication yet most are targeted to achieve a specific goal, such as retail buying patterns, or determining if a company’s earnings report is good or bad news.  To that end, news agencies such as Reuters and Dow Jones now provide machine readable news. The idea of course is that such news can be used to predict market movements, leveraged by the appropriate trading models the alpha is self-evident. However, relying on the information from these news agencies is based on a trust model.  Firms have confidence in the credibility of the information because they trust in the reputation of the source.  Any single news story, its influence on the markets and a trading firm’s belief that the news is accurate could mean the difference between grand returns or another failed attempt to capitalize. It’s easy to see that this trust model runs deep. There is the reputation of the news agency, the honesty of investigative journalism and the expectation that a story will be reported without undo bias.

Twitter transcends this news media reporting model. The information flow is not directly targeted at reporting a news story in the classic sense but to portray an emotion from the voices of millions. Untold numbers of opinions, perspectives and feelings. With the right tools in hand, a means to gauge the human psyche in action.  I believe this was the initial goal of the Indiana University research project. Twitter content simply defines trends from shear statistical numbers, “What was the level of confidence in our markets after the May 6th flash crash?” No one person can have significant influence to manipulate, they are statistically irrelevant.  Granted, there is a minority with followers in the millions, their views and opinions have the potential to have undue influence. But that old-style herd mentality is difficult to achieve in the virtual world, there is a boldness created by the anonymity of the Internet.  Resulting in a stronger sense of individualism and fearlessness to be outspoken. It’s much less likely for the majority to get swept up by the mentality of the few. So tools to observe and measure twitter’s content and detect causality of trends can hear the collective voices of millions not just those of a few.

Professor Bollen’s groundbreaking research to gauge the human psyche on twitter represents the cutting-edge in social science, techniques that are entering the fray for understanding what makes our markets move. Social sciences and trend analysis will not replace classic empirical experimentation of quantitative research but they are viable tools to leverage in the unceasing quest for alpha.

Once again thanks for reading.
Louis Lovas

For an occasional opinion or commentary on technology in Capital Markets you can follow me on  twitter, here.

About Louis Lovas

Director of Solutions, OneMarketData, with over 20 years of experience in developing cutting edge solutions and a leading voice in technology and trends in the Capital Markets industry.
This entry was posted in Algorithmic Trading, Analytics, Complex Event Processing, Equities, Foreign Exchange, Futures and Options, HFT Regulation, High Frequency Trading, OneMarketData, OneTick. Bookmark the permalink.

2 Responses to Making Sense of the Chatter on Twitter

  1. Sean Moore says:

    Louis: Really like your comments on social sciences, human behavior, and markets vs. the scientific method. Also, you mentioned the potential problem of “twitter abuse” to manipulate markets. Twitter abuse, or twitter bombs, to manipulate political elections has already occurred and has been analyzed and documented by Prof. Metaxas of Wellesley College — see, e.g., http://www.boston.com/news/nation/articles/2010/05/04/conservative_group_used_tweet_strategy_against_coakley/

    Certainly twitter bombs for market manipulation has already occurred. If you would like to talk to Prof. Metaxas about it and possibly generate some material for a future blog, I can put you in touch with him as he is a personal friend.

  2. Louis Lovas says:

    Thanks Sean for the comment, interesting article on the twitter bomb. A real concern certainly, aspects that comes to mind are reaction latency and validation. From a trading perspective, one of the main goals is to be ‘first to market’, high frequency trading focus on this exclusively. However, leveraging twitter is more the purview of portfolio traders, those holding a long/short position for more than a day. As such the reaction and validation of twitter content can have a longer latency thus giving time to weed out bombs and other spurious noise. I suspect the tracking techniques of the Indiana U project already took this into account.
    Also, recent news on twitter growth: http://tinyurl.com/4h3wvrq

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s