On September 8th, researchers, data executives, founders and practitioners congregated at the Future Data conference to exchange insights on what’s next in the world of data, from emerging architecture designs to top-notch research into visualization and large-scale data systems.

This timing of the conference was symbolical given the critical time for data, analytics and decision-making. Besides, the means to capture vast amounts of data and make it accessible to anyone feels uncapped. At the same time, despite all the data at our fingertips, there is still a dearth of data-driven decision-making in organizations.

Remember this line from  ‘A Tale of Two Cities’: “It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness”. To much extent, the problem is the modern-day interpretation of Charles Dickens’ novel.

Irrespective of this massive growth of data availability, organizations continue to lack the tools and the competence to make use of it. Some struggle to use even a fraction of their information to influence daily decisions.

However, do not take the frustration for futility. Carefully thought out steps can lead teams to accelerate the pace of decision-making, navigate change more assertively and start putting their first-party data to work to support future product decisions.

About time you Scale Cloud-Native Data Systems

This year marks a shift toward a more cloud-native data architecture. Away from last decade’s investment in data lakes, organizations are now moving toward more structured, cloud-based data warehouses. The initial justification for this migration sure was often cost and scale of storage. But ultimately, companies realize that the real return on investment (ROI) in a cloud-native stack is to accelerate access to data to inform daily decisions.

For enterprises seeking growth, this is a huge opportunity to build a sustainable competitive advantage against any incumbents. Real-time delivery of updates to the warehouse becomes easier than ever with Data pipelines. Concepts like Analytics engineering and other transformation tools can reformat data on the fly and data observability and quality monitoring platforms are emerging to improve the reliability of fragile SaaS and web data.

By getting data to the places where decisions are made faster, small organizations can execute faster and more confidently. The good news is most teams already capture the kind of data necessary to drive these decisions and the talks at Future Data did sketch out a blueprint for building a flexible data foundation:

At your first opportunity, begin  piping your SaaS and customer data into a single, cloud-based data warehouse like Snowflake, Redshift, BigQuery or Synapse.

Render out analytics-friendly views directly in the warehouse. This may require you to break some traditional business intelligence habits, but digging deep to a granular view of customers, transactions and user sessions is the key to better analytics.

Take a step further and enable your team to ask deeper “why” questions about changing metrics with tools that automate the analysis process within the warehouse.

Data Analysis and Insight Generation should be Automated and Accelerated

Now that we have more context than ever about our businesses, it’s increasingly difficult to put it all to use. That’s kind of a drawback of collecting all that data.

This is actually not a new problem. Back in 1971, Herbert Simon said in his 1971 paper “Designing Organizations for an Information-Rich World,” “a wealth of information creates a poverty of attention.” Now almost 50 years later, researchers and innovative founders are rewiring our ability to work with this complex data, both efficiently and effectively.

When you’re attempting your shot to identifying the reasons why customer acquisition costs are increasing, average order values are flat and retention rates are dropping, you have many reasons to savour in your plate. What we need to appreciate is that new augmented analytics platforms have the power to combine the best of machine learning and statistical testing with large-scale data warehouses to comprehensively test hypotheses and “prioritize the attention” of analysts working with the data.

For adaptation, a data team’s best shot at the strategy is to combine the strengths of technology — speed, processing power and iteration — with the key skills of context, navigating ambiguity and domain expertise that analysts bring to the table. This ‘human-in-the-loop’ model will come by as the dominant model for rapid, accurate decision-making in leading organizations.

Leveraging Data to Improved Decision-Making

Understand this; the ability to scale data collection and augment the analytics process is only so good as they’re improving people’s decision making. Now, to accept both sides of the coin, what’s really required to unlock informed decision-making, is that analytics tools need to be more accessible across the business and analysts need to be more integrated into individual departments.

Consequently, the future shines over operating model where companies turn away from a centralized, shared service model for analytics and one where expert analysts are increasingly embedded directly in business units. This is rebound similar to IT-centric models. Only difference is that these cloud-native data platforms are fairly better equipped to support distributed teams and collaborative work.

This leaves organizations with a single option to gain a competitive advantage. They can accelerate decision-making without losing the ability to monitor the impacts of those decisions by centralizing data but pushing analysis back out into the places where decisions are made. This engages more people in the process, improves visibility and eventually serves to capture more feedback on what’s working and what’s a flop.

As the cloud-native architecture adoption grows by the number, companies finally have the ability to track every dimension of their business with fine-grained accuracy. However, unless they accelerate and augment their ability to put the data to use, they won’t see the ROI in the decision-making process.

So here’s what can build you a more sustainable advantage if you’re seeking to enter new market or compete with established competitors- invest in scalable data systems early, build processes that automate & accelerate insight generation and distribute the decision-making process.