Obvious vs. Hidden Costs of Data

Obvious Costs of Data

Data is only valuable if you can efficiently store, access, and secure it. There are obvious costs associated with it including the data infrastructure needed to store and process the data. These costs may include hardware, software and datacenter maintenance for on-premise data operations, or cloud costs when you’re using cloud data platforms. This cost includes not only the storage of the data, but also backups, data pipelines, ETL/ELTs, network costs, and more.

There are also costs around data security. Examples may be firewalls and patch management for the databases you maintain, or security controls you may use for cloud services.

Another obvious cost is that of extracting value from the data by using BI tools. This may include licenses or subscription fees, as well as employee training for better data literacy.

Hidden Costs of Data

While some costs are apparent to organizations, there are some costs that are concealed and these costs can add up quickly. Let’s explore some of the hidden costs of data, and what can be done to reduce them:

Finding And Getting Access To The Data You Need

As datasets increase in size and complexity, data users can spend their limited time and resources searching for data. This cost is significant, as it is wasted employee time, as well as (and often more importantly) delays in data projects, or simply poor results.

In addition, sensitive data is typically restricted and there is an inherent complexity when allowing users to access this data. The wait time for approval before utilization of the data can significantly diminish workflow, resulting in lost productivity.

Reducing this cost is done by having proper metadata available to your data users. Ideally, you would have a data portal or a “data mart” where data consumers can both understand what data is available, as well as get access to it.

Inefficiency Costs

The size and structure of data can result in numerous inefficiencies, each of which has an associated cost that accumulates. Large amounts of data spread across several platforms increases the time and effort for data engineers who need to write additional code to access this data. Your data lake may become a data swamp, causing challenges in accessing and managing data, as well as having security implications.

Another common inefficiency is manually updating metadata and data models in data catalogs, when some of this work can be automated.

Security Costs

Some security costs are often overlooked such as meeting specific security requirements. A good example is handling sensitive data. There are common security policies configured specifically for locations where sensitive data is stored. Periodically mapping and configuring sensitive data is inefficient due to its time consuming and resource intensive nature.

Another typical example is the need to anonymize data based on the role of the data consumer, often done using dynamic data masking. There is a cost associated with implementing such access controls.

Reducing such costs can be achieved by having clear security policies and better collaboration between security and data teams.

Compliance Costs

Finally, there are hidden costs in compliance. Gathering the required information for an audit requires significant efforts from various data teams which increases the cost of compliance. For example, creating reports of new PII locations, the users accessing PII, or the controls applied on sensitive data access can be burdensome.

Satori Helps Reduce Data Costs

Satori helps reduce many of the hidden costs of data. When it comes to finding and getting access to data, Satori simplifies granting and removing access, including capabilities to provide temporary data access and self-service data access using a data portal or Slack integration.

When it comes to security, Satori enables you to have a DataSecOps platform that continuously detects sensitive information, applies security policies across all your data access, and performs that without dependency on data engineering resources.

Finally, Satori keeps you continuously compliant, with audits of all access to sensitive data, including reporting by identities.

Conclusion

Costs of data can be plain to see, but some of them are sneaky or overlooked. This distorts your budget planning, as well as slows down your time-to-value from data and disrupts your business with unplanned projects.