Cost — the neglected Data Team KPI

Cloud has turbocharged the growth of data products. Growth that has a hidden run cost that is rarely considered by Data teams.

Data products are being deployed at an alarming rate — especially in the Enterprise. Data products can take any number of forms; a once-off analysis, monthly dashboard for the exec team, or power mobile apps used by millions of users.

Whilst Data teams focus on launching or improving their data products, one KPI they typically do not consider is the cost of running their data product.

Why cloud should change the way data teams think about cost

Before cloud, the cost of running a data product had minimal focus. Infrastructure costs are generally ignored — it didn’t cost any more to run another product. Until you hit the capacity limits and an upgrade was required.

Cloud changes this. Data products in the cloud have a new cost element — operational costs. Cloud data platforms (e.g. Snowflake) charge companies a fee each time a data product is run.

Take the example of a model being generated for a dashboard. Every day that model is being generated consumes cost. But if that dashboard is never used, generating no value, then that spend is being wasted.

Combined with cloud list prices being typically 10X more than running on-premise, unmonitored operational costs can quickly spiral out of control These unmonitored operational run costs can quickly lead to data costs spiralling out of control.

Cloud cost as first class KPI for data teams

To enable Data teams to own and manage their operational costs they need visibility on how their data products are being used. Data product analytics can track usage and spend to determine when usage, value and cost start to diverge.

A healthy report which consistent usage over time

A dead report. The data set being generated for this report is wasted spend.

Cost per usage is a powerful metric that could be used to benchmark data products and optimise data spend. Data owners can easily identify products with a high cost per usage that require review. Is there a problem with the product that users aren’t telling you? Is it time to decommission the product?

Regularly reviewing and optimising your data products not only saves costs but also has the added benefits of reducing support efforts and having a cleaner data ecosystem

Are you looking at your cloud data costs and wondering how to optimise or stay on top of it? Check out Kada.ai for how we implemented a data product analytics solution to solve this problem. We’d love to hear from you.

Written by Dean Nguyen, CEO of Kada.ai