Home

Shining a light on your dark data

Dan Kieran, VP & MD Australia & New Zealand, Commvault

Dan Kieran, VP & MD Australia & New Zealand, Commvault

Industry research suggests that every day, the world generates 2.5 quintillion bytes of data. And 90 per cent of it has been generated in the last two years – this is thanks to the huge volume and variety of sources creating data on a daily basis. Let’s just put that into perspective for a moment; that is the equivalent to the total amount of storage on 57.5 billion 32 GB iPhones. There’s no doubt about it, that is a lot of data and yet so much of it will remain in the dark.

Gartner defines dark data as “the information assets organisations collect, process, and store during regular business activities, but generally fail to use for other purposes (for example, analytics, business relationships, and direct monetising).” Essentially, it is data that businesses are assigning resources to, in order to store, protect, and manage but it isn’t being leveraged to improve overall efficiency or productivity. It may even contain content that puts organisations at risk if it’s not stored appropriately.

The Compliance, Governance and Oversight Counsel has estimated that 69 percent of a company’s stored data has absolutely no value to the organisation. This begs the question; why are we keeping it?

Why keep it?

We are no longer dealing with MBs or GBs of business data, meaning the “keep everything” principle for data management is no longer viable. Managing the sheer volume of data is therefore vital to organisations today. But that’s not always as easy as it sounds. To improve the process, infrastructure managers need to embrace content-based retention policies that empower them to keep only the most important and relevant data available for easy access.

The Compliance, Governance and Oversight Counsel Has Estimated That 69 Percent of a Company’s Stored Data Has Absolutely No Value to the Organization

These policies can also take the latest government legislation into account to ensure best practice across an organisation.

As for the rest? The remainder of an organisation’s data must be efficiently archived to keep systems running smoothly. Techniques such as the implementation of user-defined policies including: file name, type, user, group, keyword, exchange classification, and tagging are in growing demand. Such methods allow organisations to add layers of intelligence into the archiving process that previously did not exist, improving efficiency and the overall safe-keeping of data.

Do you know when to ‘delete’?

While effective data management will always be a business priority, we cannot ignore the elephant in the room which is the growth of data. The fact is, no-one can afford to keep volumes upon volumes of useless stored content, therefore the lifecycle of all business data must be considered — from its creation to its disposal. There is a temptation to want to keep every piece of data, to help empower business decision making of the future. But in reality, organisations must consider the end goal or purpose of their data very carefully and define governance policies for its deletion. By cutting data out of the equation, businesses will reduce their dark data and add clarity to future analysis. What’s more, removing useless data can improve system performance, contributing significantly to overall reliability.

Whilst data deletion sounds time consuming, it doesn’t have to be a manual task. Infrastructure managers have the power to implement automated policies to classify, organise, retain, and delete information which in turn streamlines the entire information management lifecycle. In addition to reducing manual data deletion, data automation techniques also improve compliance and litigation readiness making them critical to any business today.

As data is the heartbeat of every organisation, its effective management is business critical. At the same time however, data management is a complicated business – the quantity of data and ever increasing data sources has made sure of this. But companies don’t need to work in the dark. Infrastructure management is as easy as the processes and systems behind it. As data continues to grow, infrastructure managers must be empowered with the right tools to help them sift through their data in order to govern, protect, and leverage it effectively. Automated processes will be key to delivering high levels of data control and will ultimately help to drive business intelligence and efficiencies whilst positively contributing to compliance and reducing risk associated with data loss and improper management.