Big Data, Small Data and Fast Data: Using data to drive better decisions

By Sumanth Suresh | 13 July 2015

Data in Decision Making

In a world where competition is nimble, digital and disruptive, speed and accuracy of decision making is critical for organisational success. In the quest for timely decisions, organisations stand the risk of taking incorrect decisions by relying solely on quick thinking, intuition and judgment of individual decision makers. The question then arises as to how to achieve enterprise-wide decision making that is fast, accurate and scalable. While data-supported decision making improves the likelihood of making optimal decisions, a key challenge is the amount of time spent to arrive at a data-supported conclusion, particularly in the context of the “Big Data” phenomenon.

The universe of data is exploding. Digital data that is created and copied annually is expected to grow tenfold, from 4.4 ZB in 2013 to 44 ZB in 2020. This data explosion is driven by a number of factors, key among which are rapid proliferation of mobile devices, expansion of sensor-based technologies capable of recording and transmitting information (including wearable technologies), sharing of user created rich media content and the emergence of the “Internet of Things”.

The term ‘Big Data’ is typically used to describe the collection, storage, processing and retrieval of data which arise out of various factors of data production. According to Gartner, “Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.”[ii] With these characteristics, the skills and infrastructure that are required to manage and leverage Big Data go beyond traditional organisational skills that are rooted and developed within individual functional units such as Marketing, Product, Finance, Payments, Operations, CRM and Engineering.

At Western Union, we’ve found the data explosion to be very real. To give an idea of the scale, our C2C business spans across the online and offline world. Our transactional website (in 25 countries, including 21 in Europe) and our mobile app (in the United States) are visited by millions of visitors each month. In the retail world, there are nearly 500,000 point-of-sale Agent locations in over 200 countries and territories. We also have a strong social media presence on Facebook (with over 5M likes) and on Twitter (with over 435,000 followers), where our customers engage, share content and communicate with the company. In addition to transactional, social and digital data, we also work with risk and payments data, consumer survey and research data, contact center operations data, macroeconomic data and people migration data.

Framing a data strategy

A 2013 study by Bain and Co. shows that companies that use analytics best are five times more likely to make decisions faster than competition, and are two times more likely to use data very frequently when making decisions. Data and analytics, if implemented in the right way, can address speed, accuracy and scalability of decision-making. The value of data as an organisational asset is realised only when there is a coherent data strategy which looks at the organisation structure, infrastructure/tools and talent in an integrated way.

For a viable data strategy, there needs to be an organisational mandate to embed and integrate data and analytics into decision-making across business functions. The commitment of leadership teams to seek and utilise data based insights is critical for the development of an organisational structure that maximises the value of data. Furthermore, the emergence of Big Data demands tighter integration of business functions from a people and IT assets perspective.

Various teams need to work together to understand and identify the cross-functional value of data. Identifying key data gaps across the organisation and taking a collective decision for data gathering ensures that only data with information value is collected.

A single data environment housing multiple data streams is imperative for holding data that is easily accessible, consistent and supports collaborative usage. This enables the democratisation of data within an organisation, supports the building of predictive tools and data visualisations, and is foundational to the scalability of data-supported decision making. While designing the environment, it is important to keep in mind that not all enterprise data comes in the form of Big Data. There are several data forms such as user survey data, macroeconomic data, market research data among others, which add rich context in generating insights.

We are leveraging the Hadoop ecosystem to house data from our multiple data sources. Hadoop provides a software framework for distributed storage and distributed processing of big data sets, using hardware clusters that are relatively inexpensive. Within Hadoop, we stitch together data across various data streams, to provide additional dimensionality and context to the business user. For example, with the integration of our marketing campaigns, website and customer data, our digital marketing teams not only know which campaigns are producing customer signups, they can also see the quality of the signups through the website conversion funnel, at which step they drop off on the website and the value of the customers that go on to transact. This enables them to decide quickly which campaigns to continue and which to fine-tune. Our product teams are able to watch closely the interaction of various marketing channels by specific customer segments, platforms and send-receive markets to identify website conversion opportunities. Our business teams use mixed models incorporating macroeconomic data such as GDP growth and unemployment figures along with internal business drivers such as pricing and marketing allocations to drive decisions on media mix optimisation. Meanwhile, our risk and payment teams use velocity variables to enable real time risk scoring. 

Whilst a single data environment is a necessary foundational infrastructure, the data goes only as far as the skill and talent of the user. It is in this context that the role of a data scientist acquires significance. Compared to the traditional role of a statistician or an analyst, this role requires diverse skills spanning programming, statistical analysis and communication. A data-supported organisation needs to make concerted efforts to hire, nurture and retain talent in data science to be able to maximise the value of their data. 

With a well-organised data strategy, organisations can begin to leverage data as an asset that provides value to business stakeholders, enabling timely and accurate decisions for their business.


By Sumanth Suresh​, Senior Manager, Customer Strategy, Western Union

Become a bobsguide member to access the following

1. Unrestricted access to bobsguide
2. Send a proposal request
3. Insights delivered daily to your inbox
4. Career development