By Dr John Bates,
chief technology officer,
follow Dr Bates on Twitter here
UK hedge fund Derwent Capital has bet 25 million pounds that Twitter can predict swings in the Dow Jones Industrial Average with 87 per cent accuracy. While this sounds super cool, it is my view that predicting stock market moves using Twitter alone is about as reliable as a witness in the Raj Rajaratnam insider trading trial. However, do not interpret from this that I’m not a Twitter fan or think that value cannot be gained by analysing Twitter. I am and I do. In fact I believe that Twitter analysis can provide a view of global sentiment that we can use to learn all sorts of fascinating things about how the world is feeling on particular topics. My beef is the claim that Twitter can be used to predict the movements of the stock market.
The first problem with predicting stock movements is that Twitter is not secure. I could tweet anything – eg “US declares war with France”. If that is real then we should certainly react – but I could have made it up. If I have a million followers, they may think it’s hilarious to retweet it. But how should our prediction algorithm respond?
The second problem is that, for mainstream news, it is highly unlikely that Twitter can provide business critical information to traders quicker or more efficiently than a real-time news source such as Reuters or Bloomberg. When an event occurs and the news media puts out a story, then interested parties retweet that story. This may be an effective way to share news and information among the masses, but from an algorithmic standpoint it is a lagging indicator. So often tweets are reactive rather than proactive.
The third problem is in using global sentiment as a predictive indicator. Tools that can analyse all tweets to gauge global sentiment on particular topics, such as the economy, are incredibly exciting. The results can represent a snapshot of a “global consciousness”. And taken at a global level, it does not matter if some people tweet made up facts, as long as 99 per cent of the population does not; The global view removes the noise. But the problem is that by the time we have built up a picture of the global sentiment, it is a trailing indicator and the real-time sentiment may have changed. It is not necessarily a predictor because a real time event, such as a war, a market crash or a good economic news announcement, may occur – which completely changes what the market is thinking.
Twitter is a great tool for judging whether the public is fed up with the royal wedding, or for getting your latest personal opinion out to the masses. However, alone it is not a credible source for predicting market swings. Analysing sentiment globally is one powerful indicator – but alone it is not enough. Successful investors use market sentiment as one indicator of how the market is going to perform in the future. Combining that sentiment with the ability to respond to predictions of how it will move or to real-time events is often the key to success.
Sometimes betting against market sentiment can provide fast, substantial gains. For example, we have recently seen that market sentiment was leaning towards the US dollar outperforming the Swiss Franc. In this instance, due to upcoming elections, many investors were advised to ignore the sentiment and bet that the Swiss Franc would surpass the US dollar by the end of that week. Those who participated in the trade booked 545.2 per cent gains. However, while those gains are sizable, they’re only part of the story.
It is becoming more important than ever for traders and investors to know how to track sentiment. The globalisation of the world economy has increased complexity, as well as uncertainty, at all levels of investment. All this is reflected in the correlations between different markets. An event-risk in one market quickly cascades in other markets. For instance, when the North Koreans initiate an aggressive move, the yen weakens. Closer to home, when the Irish run into financial trouble, the euro weakens and risk appetite around the world shifts into risk aversion. The challenge is to be able to detect both direction and points of inflection in global sentiment.
In terms of combining sentiment analysis with response to real-time events, the good news is there are sophisticated real-time technologies on the market that can monitor and respond to market patterns, such as Complex Event Processing (CEP). These patterns may be trading opportunities as well as risks and threats, such as unusual market movements or risk thresholds being breached. In fact CEP is now being used to analyse twitter sentiment as well – combining the best of both sentiment and the ability to respond to events. I envisage a future in which social media is combined with a broad range of data, including news, weather patterns and other forms of behavioural finance data, to create a more acute and comprehensive predictive model of markets across several asset classes. People have not even figured out a killer way of using news data yet – we are still exploring the tip of the iceberg.