General sentiment toward Big Data in 2013 had a substantial impact on the discussions of analytics among business users, culminating in the mistaken idea that if enough data is analysed by the right people, anything can be predicted, according to International Data Corporation (IDC). This article is copyright 2014 The Best Customer Guide.

The reality of the situation, however, is expected to become apparent as more and more organisations attempt Big Data and analytics initiatives. While the opportunities are real, it still involves a lot of trial and error and experimentation.

IDC believes that the drive for heightened customer engagement will be strong enough to compel organisations to push through these challenges, and has consequently compiled its top ten list of big data predictions for 2014.

"People don't want to feel marketed to, they want better levels of service. Organisations that are stuck in the old mentality of only maximizing wallet share will risk brand reputation and losing out to those organisations that personalise services based on what they know about that person," warned Craig Stires, Research Director for Big Data, Analytics and Security Software at IDC Asia/Pacific.

Emerging trends in Big Data have spread quickly into both mainstream and niche markets, and the key trends can be grouped into three domains:

  1. Key industry opportunities for customer and citizen engagement;
  2. Technologies gaining broadest adoption, if not media attention;
  3. Market and regulatory impacts of enriching and monetizing data.

"In the near term we can expect to see strong interest in the insights encoded in geolocation information, when tied to personal behaviour. Efforts to enrich and leverage these insights will push all legal boundaries and expectations of privacy. Expect this to be a discussion that spreads across the markets in APEJ. More mature markets, like Australia and Singapore will be most prominent in leveraging Big Data and analytics for heightened customer engagement," said Stires.

Drawing from IDC's own research, the key Big Data predictions and trends include:

  1. Geolocation matched to individuals will see Telcos and Retailers attempting personalised services, while pushing the boundaries of consumers' expectations of privacy Two different standards are in place for the protection of consumer and citizen privacy, government-defined legal regulations and individual expectations. Legal regulations are extremely limited, particularly in the face of data marketplaces, blurred lines of what people meant to share, and mass accumulators of otherwise disconnected personal data. Telcos, retailers, and to some extent banks, stand to fundamentally change their relationships with customers, based on the kind of insights afforded with joining and enriching various sources of personal data. If done correctly, they can move to a model of personalised and relevant services, and away from more traditional blast marketing of products.

    The game changer on the privacy side will be geolocation information. When it comes to geolocation data, the majority of people have no idea how their data is being collected and shared. Mobile phones are currently the biggest source, with network triangulation, GPS, and WiFi all providing streaming updates of where a device is, and by association, the owner of that device. As more cases surface of who has access to that data, and how it is being used, there will be an increase in awareness that individual expectations of privacy are being overrun.

  2. High profile data breaches will drive regulators to draft specific new policy measures Individuals are generally unaware or misinformed about how much of their behaviour data is being collected, and by whom. Recent revelations about government spying are offensive, but only mildly surprising to most. But, discovery that companies have been collecting, buying, enriching, and profiling our very personal lives elicits a stronger response. Organisations that have built highly personalised profiles of individuals do not always treat those enriched views with the level of care needed.

    In 2014, IDC anticipates a number of high profile data breaches that will expose how much information has been captured and enriched by some organisations. This will result in damage of brand reputation, a serious effect for an Asian organisation, but more broadly will raise awareness that citizens are not as protected as they assume. This discovery will put pressure on governments and regulators to attempt clearer protection through new policies.

  3. Infrastructure investments will outpace software and services, with storage for Big Data growing rapidly by 45% from the previous period Volumes of data being created are unprecedented in our history. With IDC estimates showing that data will double about every two years, the infrastructure to store and analyse that data will grow rapidly. Particularly in emerging markets, the share of the data will grow from 36% in 2012 to 62% in 2020.

    IDC envisions that storage will consistently be one of the fastest growing Big Data and analytics investments in the coming year and beyond, with at least a 45% growth from 2013. The question asked by organisations that will guarantee a continual growth in data captured is simple: "What is the worst that could happen, if I don't have this data when I need it?"