Did you know that 90% of the world’s stored data has been created just in the last two years? Neither did I… and it’s the most staggering statistic I’ve heard in a very long time.

Apparently we’re creating 2.5 quintillion bytes of data every day (that’s 2.5 followed by more zeros than I can keep in my head at one time). This data comes from all around us – posts to social media sites, digital pictures and videos, purchase transaction records, mobile GPS signals, and the like.

These classes of data have of course hit the headlines in a big way recently, courtesy of Edward Snowden’s revelations about US and UK government snooping, but even before that furore, it’s been hard to miss the buzz over what IT consultants far and wide refer to as ‘big data’. It’s also been something of an obsession for the insurance industry of late.

Insurers, like many other businesses, are amassing huge amounts of digital information and are busy working out how they can use it to improve the way they do things. Specifically, using their own databases in conjunction with information available from other organisations – both private and public sector – to develop more accurate ways of rating risk and combating fraudulent activity.

Third-party-sourced digital data has become more important to insurers for pricing risk as the use of certain traditional underwriting factors has now been limited by legislation such as the EU Gender Directive. Then of course, there’s the introduction of the Consumer Insurance Act, which has shifted the onus to insurers and intermediaries to ask the right questions in order to establish the true risk the customer presents and verify that information provided is correct.

Tapping into data sources such as the electoral register and credit checking agencies should have a positive impact on an insurer’s ability to price risks overall, and verifying customer details at the point of quote using data enrichment should lead to consistently better selection against high risk clients. This is all well and good for the insurer profitability, but what will it mean for intermediaries on the ground?

For intermediaries, real-time ratings should give access to products that help them compete with the direct players on personal lines insurance such as household. It could however be a double edged sword.

It is certainly possible that insurers will place more importance on intermediaries checking information against data in order to present their insurance partners with better and more accurately represented risks. However you can rest assured that quotation systems such as the Source will be working to ensure that such activities are as quick and easy for the broker as possible.

So while this trend certainly could lead to a greater burden being placed on intermediaries, we would expect that to be outweighed by benefits such as improved customer retention due to more accurate risk pricing and fewer instances of having to go back to the customer with referral questions or to advise that there is anomaly and an additional premium is due.

Big data is a subject that no intermediary can afford to ignore, but neither do they need to worry about it as long as technology providers like the Source are embracing the challenge of making it work in your favour.

Kevin Paterson is managing director of Source Insurance