Accelerating Digital Transformation With Data Analytics in the Cloud
EBOOK
Read eBook
Swipe left and right to Navigate
EBOOK: ACCELERATING DIGITAL TRANSFORMATION WITH DATA ANALYTICS IN THE CLOUD
2
3 • Introduction 4 • The Need for Speed With Analytics in the Cloud 6 • Destination: Data as a Strategic Asset
Eliminate data silos and integrate Scale, manage, and secure Operationalize and automate Cruise towards optimization
13 • Roadmap to a Successful Data-Driven Organization
3
Introduction
As companies navigate the digital transformation track and confront the impact of a global pandemic, many are accelerating their efforts to reinvigorate growth and boost market share. Some are running into roadblocks as they struggle to unlock the value of data as a strategic asset. Fast-tracking the transition to the cloud and taking a well-mapped approach to data driven analytics are essential. Companies are looking to drive a variety of core business objectives, including targeting growth initiatives and defining new markets. They need insights for designing and building better products and services and attracting new customers through more engaging experiences. Facing novel competitive and economic challenges, organizations are hungry for intelligence that will help optimize and model more efficient operations.
In a business climate completely reshaped by the pandemic, data analytics can play additional, elevated roles in steering more informed decision-making. The infinite scalability of the cloud and the ever expanding palette of new data sources can help companies more accurately forecast demand, identify supply chain obstacles, determine at-risk work scenarios, and unlock innovation and efficiencies in a time of constant change and economic upheaval. “Uncertainty is driving the need for more current data and … the flexibility to change decision models as events change—whether that’s in the supply chain, customer purchasing behavior patterns, HR practices, or financial models,” notes Dan Vesset, group vice president at IDC. “Previous decision models have been broken because of this one-off event, so there’s a need for more data to drive rapid decision-making.”
“Uncertainty is driving the need for more current data and … the flexibility to change decision models as events change—whether that’s in the supply chain, customer purchasing behavior patterns, HR practices, or financial models,”
DAN VESSET
GROUP VICE PRESIDENT, IDC
4
The Need for Speed With Analytics in the Cloud
These uncertain times call for a modern approach to analytics. There is a need for near-real-time capabilities that continuously evolve and scale to meet these ever-changing business requirements and to extract even more value from existing investments and data sets. Through that prism, the cloud is no longer optional as a core foundation for business. It is the right platform to deliver greater flexibility and access to massive, low-cost storage, and to usher in more expansive service capabilities. With a cloud model, there’s no investment in expensive infrastructure or over-provisioning to accommodate peak capacity. The cloud’s pay-as-you-go pricing model gives enterprises greater control over costs: Companies pay only for the storage capacity and processing power they need when they need it.
The cloud’s potent combination of fast provisioning, elastic computing power, and massively scalable storage ensures the highest levels of agility and flexibility. At the same time, the deployment model shrinks time-to-delivery of critical systems, which increases companies’ opportunity for innovation and competitive advantage. Rather than standardizing on a single cloud vendor or committing to a cloud-only architecture, organizations are adopting hybrid multi-cloud strategies that leverage the best of multiple environments. By doing so, they mitigate the risks of relying on one platform or vendor, ensuring greater flexibility and promoting business resiliency.
WORLDWIDE SPENDING ON BIG DATA AND BUSINESS ANALYTICS
$189.1
SOURCE: IDC BIG DATA AND ANALYTICS SPENDING GUIDE, AUGUST 2020
BILLION
$274.3
2019
2022
5
Similarly, when organizations move analytics workloads to the cloud, they also are best served with hybrid multi-cloud approaches that balance the benefits of multiple clouds with those of on-premises systems. Such a strategy ensures they can tap into cloud benefits such as ease of use, simplified configuration and management, and reduced maintenance while accommodating existing on-premises applications that support vast amounts of critical operational data. Opportunity and demand for analytics workload transformation are enormous. Fueled by the need to invest in modern technology that promotes innovation and competitive advantage, IDC projects worldwide spending on big data and business analytics will jump from $189.1 billion in 2019 to $274.3 billion by 2022.
Despite significant investment and demonstrated organizational intent, progress in building data-driven organizations continues to be difficult and slow. According to NewVantage Partners’ 2020 executive survey on big data and AI, 73% of respondents said this was an ongoing challenge, and only 38% confirmed they are successful at creating a data-driven organization. Less than half (45%) believed they were successfully competing on analytics.“ The goal is to be better, faster, cheaper, but we’ve never come close to that,” says Wayne Eckerson, president of Eckerson Group, a research and consulting firm specializing in data analytics. “The cloud brings a ray of hope that we can start to achieve that with data and analytics.”
ONGOING CHALLENGES WITH DATA
STRUGGLING WITH CREATING A DATA-DRIVEN ORGANIZATION
SOURCE: NEWVANTAGE PARTNERS
STRUGGLING TO COMPETE SUCCESSFULLY ON ANALYTICS
6
Adopt a holistic approach.
Traditionally, individual business units collect, own, and leverage data for their specific needs. For example, salespeople collect data on prospects and customers, applying insights to recommend specific products, identify cross-selling opportunities, or provide hands-on customer service. In contrast, product development is more interested in data that can direct future products and services or shed light on warranty issues or problems in the field. Although these specific functional areas aptly use data for their own purposes, these efforts benefit a narrow slice of the business rather than the whole. Instead of wasting resources and restricting the value of data on initiatives at the business-unit level, companies must address end-to-end requirements across multiple functions, taking aim at business objectives that reflect the entire organization.
Like business-unit focused data, siloed data has limited value because it’s often repetitive and lacks the uniformity to allow fast, informed decisions and actions. Instead of sharing a cohesive data set, departments and stakeholders duplicate data, which means they do not work from a consistent data set. That scenario often creates confusion and can compromise agility. Whether data silos spring up intentionally or unintentionally, the limitations are the same. The risk is creating decision silos where an action taken in one part of the organization isn’t adequately balanced against an opportunity or risk factor impacting another part of the enterprise, IDC’s Vesset explains. “You don’t get a holistic view across the organization. You might get insights into customer behavior, but not the supply chain,” Vesset says. “That doesn’t allow you to provide the best customer service because you promise something on the front end, but can’t fulfill it on the back end.”
Say goodbye to business-unit, siloed data.
FIRST GEAR
Eliminate Data Silos and Integrate
Destination: Data as a Strategic Asset
To maximize the value of data, organizations need to throttle up a new approach to data analytics. By following this route, they can create a foundation that can continuously scale to meet changing business needs. Here’s a roadmap for getting started.
7
Embrace the cloud.
Empower users by democratizing data.
Just as the cloud is now the preferred platform for many business-essential workloads, it’s also fast becoming the modern track for analytics and data-driven workloads. The benefits are numerous, from ease of scalability to manageable, cost-effective pay-as-you-go pricing models. While the cloud was an early destination for big data analytics workloads, companies have taken a bit longer to feel comfortable shifting decision support and data management workloads. These tend to rely on historical and operational data, which remains a mainstay of on-premises systems. Nevertheless, the cloud is gaining traction across all areas of data analytics. By 2022, public cloud services will be essential for 90% of data analytics innovation, according to Gartner. As organizations migrate data and analytics
A successful data-driven organization doesn’t limit access to data analytics insights to a select few business users or a cadre of data scientists. Data value is created when more people and business processes gain frictionless access to all the right data, analytics, models, and advanced functions based on their specific requirements and in the context of proper security rules. As part of a modern, cloud-based analytics framework, users should have access to the data they need in the environments they prefer, using fit-for-purpose analytics and visualization tools that help make them better at their jobs. The optimal data analytics framework should be designed with a holistic approach that takes into account the mesh of users, data, platforms, and applications. The platform should accommodate an unlimited number of users and be optimized for concurrency, with the ability to handle tactical and strategic queries with sophisticated workload management at scale.
Such disconnects are mitigated with an integrated, enterprise data repository and set of reusable analytics. With such an approach, data becomes accessible and usable to everyone who needs it—from sales reps trying to close a deal to customers looking to confirm order status and time to delivery. In addition, IT and business stakeholders spend less time on deduplication and rework, shifting attention and resources to higher-value activities, which fosters business agility.
workloads to the cloud, they must go beyond a simple “lift and shift” strategy of moving existing systems. Instead, they must create a new, more powerful foundation that can keep pace with data growth and the speed and diversity of analytics innovation.
8
Pursue a hybrid multi-cloud approach.
Even with the cloud’s flexibility, ease of management, and resiliency advantages, on-premises systems will persist. As a result, a modern analytics framework must adopt a hybrid strategy that melds the flexibility and agility of the public cloud with on-premises systems that are still ground zero for current and historical operational data critical to the broader picture. While conventional, nonvirtualized on-premises systems also will continue to exist, organizations should embrace cloud-like concepts and use virtualization and automation technologies to modernize on-premises systems. They also should consider vendor-supported or managed services to power up and administer on-premises private clouds.
A multi-cloud strategy is another important piece of the modern analytics architecture; this helps avoid a single point of failure for mission-critical business systems along with other risks related to availability, reliability, and security. One big concern is vendor lock-in or reliance on a single cloud provider or environment. Embracing multiple clouds as part of an overall strategy also provides greater flexibility to match data and analytics workloads to the cloud platform most appropriate for their specific performance requirements along with other capabilities.
“Most organizations were not born as cloud native and have existing on-premises infrastructure,” Vesset says. “Because of that, it’s financially and physically impossible to migrate to cloud all at once. It could be a multi year or multidecade process, and during that time, hybrid is necessary.” A multi-cloud strategy is another important piece of the modern analytics architecture; this helps avoid a single point of failure for mission-critical business systems along with other risks related to availability, reliability, and security. One big concern is vendor lock-in or reliance on a single cloud provider or environment. Embracing multiple clouds as part of an overall strategy also provides greater flexibility to match data and analytics workloads to the cloud platform most appropriate for their specific performance requirements along with other capabilities.
9
Deliver data at scale.
SECOND GEAR
The mission of a modern data analytics strategy is to allow organizations to adapt dynamically to changing business needs while giving employees, partners, and customers the freedom and flexibility to ask complex, data-driven questions. Because there are no set parameters on questions or the amount of data required to answer them, scalability lies at the heart of modern data analytics. Organizations want to avoid over-configuration and investment in expensive unused capacity for their analytics platforms. At the same time, they can’t risk under-configuration that would prevent systems from performing as desired. What’s needed is a platform that can support hundreds if not thousands of users and millions of queries every day. To deliver true scalability, a platform should do the following:
For data to become a true high-yield asset, it must be delivered at multidimensional scale across eight core dimensions: data volume, data latency, query data volume, query complexity, query concurrency, query response time, schema sophistication, and mixed workloads. Although all dimensions are not required for every query, predicting the right multidimensional mix for scaling each workload is difficult; one dimension risks scaling at the expense of another. In response, organizations should embrace a platform that enables each dimension to scale independently, providing flexibility for decision-making without technology-driven constraints.
Separate computing and storage with elastic scaling.
Integrate with cloud services and ingest modern data sources.
Support integrated data management and scalable analytics.
Embed dynamic resource allocation and workload management.
Scale, Manage, and Secure
10
Treat data with consistency and rigor.
Data must be treated with the same rigor and consistency applied to other critical enterprise resources. This means putting processes and practices in place to manage reliability, quality, and governance across the entire lifecycle. Ensuring IT and business alignment around data management practices is critical to retaining data’s value. This includes embracing master data management and rigorous data quality practices. Defining and assigning key metrics against data’s value, usage, and contribution are important, too. By monitoring and understanding those metrics, organizations can drive data consumption and frictionless levels of utilization.
role-based access and fine-grained security controls. The platform should also support vigilant security monitoring, including intelligent scanning and correlation of all security-relevant events.
Put the proper security mechanisms in place.
As mission-critical data and analytics workloads migrate to the cloud, security moves front and center. Safeguarding data and intellectual property is key to maintaining customer trust and competitive advantage. It’s also part of meeting compliance and regulatory standards and protecting the enterprise from damage resulting from a security breach. A modern data analytics platform should employ a multitiered approach to securing data assets and managing vulnerability. Strong data encryption and authentication are central to a well-formed security model, as are capabilities supporting
THE 8 CORE DATA DIMENSIONS
• DATA VOLUME • DATA LATENCY • QUERY DATA VOLUME • QUERY COMPLEXITY • QUERY CONCURRENCY • QUERY RESPONSE TIME • SCHEMA SOPHISTICATION • MIXED WORKLOADS
11
Create value from data.
To succeed as a data-driven organization, businesses must go beyond the one-off insight and leverage data across the organization. Instead of using data for a single purpose, the goal is to create processes and practices that operationalize data, models, and insights by integrating them directly into common business practices and systems. By late 2024, three-quarters of enterprises will shift from pilot projects to fully operationalized AI, driving a five-fold increase in streaming data and analytics, according to Gartner.
“We need to move from artisanal development to industrial development, and that’s what DataOps can do. If we are scaling out use of data, we need to build scalable processes and have lots of people building things from the same pipeline.”
WAYNE ECKERSON,
For example, DataOps, AnalyticsOps, and model management are gaining traction as effective mechanisms for building and automating pipelines that ingest data, prevent quality degradation, and perform ongoing monitoring of usage and query speeds to protect against drift in model forecasting. “We need to move from artisanal development to industrial development, and that’s what DataOps can do,” Eckerson says. “If we are scaling out use of data, we need to build scalable processes and have lots of people building things from the same pipeline.”
THIRD GEAR
Operationalize and Automate
PRESIDENT OF ECKERSON GROUP
Leverage automation capabilities.
To do so, data must be streamlined, managed, and automated at enterprise scale. Embracing new practices modeled on agile software development is key to this transformation.
12
Embrace a culture of continuous improvement.
There is always room to improve analytics models or processes. Organizations must evolve their cultures and practices to get behind constant refinement of models, ongoing cleansing and reevaluation of data sources, and recalibration of core business metrics. “Your data teams need to have that as a mantra, and DataOps is a vehicle to achieve that,” Eckerson says.
Employ proof of concepts.
POCs are a critical stop on the cloud data analytics journey, but they have to be done right. The entire picture of the data analytics environment to be migrated, including all workloads, concurrency profiles, and user assessments must be considered. Data must be pruned so that only that which is useful to existing and future workloads is migrated.
“There’s still inertia of making decisions based on experience only—believing you know better than the data that is guiding you. For most, the data culture is not quite there, and data literacy among employees is not high enough.”
DAN VESSET,
Ensuring that data becomes a high-value asset is a team sport. It’s not the sole responsibility of the data analysts, data scientists, or even IT. By employing agile development practices, relationship managers, and a federated Center of Excellence, organizations can bring the appropriate parties together to ensure that data analytics remains a core priority reflected across business processes and enterprise culture.
FOURTH GEAR
Cruise Toward Optimization
Once a modern data analytics platform and core enterprise processes are in place, there’s no room for inertia. By studying, learning from, and mimicking best practices from cloud data analytics pioneers, companies will ensure the most direct route on their journeys.
Foster better alignment between data people and business people.
Tap into expertise.
Sometimes, the most informed data analytics experts reside outside of an organization. Guidance from subject matter experts and third-party providers who have helped others on the journey to data analytics in the cloud can be indispensable.
13
DOWNLOAD OUR WHITE PAPER HERE.
ELIMINATE DATA SILOS AND INTEGRATE
The Teradata Advantage
Find out how to transform without compromise through the power of multidimensional scalability
Teradata Vantage is the cloud data analytics platform that unifies data warehouses, data lakes, and analytics into a single connected ecosystem. Built for a hybrid multi-cloud world, Vantage solves the most complex data challenges at scale.
With data-driven insights as the direct path to new efficiencies and innovation, organizations must make the leap to the cloud while transforming analytics processes. A hybrid, multi-cloud strategy is the most pragmatic way to lock in the cloud’s benefits and accelerate the value of intelligent data.
Your Roadmap to a Successful Data-Driven Organization
Adopt an enterprise approach Say goodbye to business unit, siloed data Embrace the cloud Empower users by democratizing data Pursue a hybrid multi-cloud approach
TM
SCALE, MANAGE, AND SECURE
OPERATIONALIZE AND AUTOMATE
CRUISE TOWARD OPTIMIZATION
Deliver data at scale Treat data with consistency and rigor Put the proper security mechanisms in place
Create value from data Leverage automation capabilities
Embrace a culture of continuous improvement Employ proof of concepts Foster better alignment between data people and business people Promote data literacy Tap into expertise
CLICK TO LEARN MORE
