Data Management: The impact of coronavirus and the new normal
22nd April 2020
Coronavirus will force organisations to juggle a range of unique challenges over the year and beyond.
We may be at the start of a long journey to tackle the rapidly unfolding COVID-19 pandemic. Although the actual duration remains a subject of intense speculation, there is little doubt that, as a society, we are truly embarking on a marathon rather than a sprint.
The impacts of the crisis fall broadly into three categories: medical, social, and economic. Although the medical impact is of course the most severe and distressing, for this article, we will focus here on the latter two areas. We have identified a number of emerging trends relating to data strategy and architecture with implications across a broad swathe of industries, service organizations, and governmental bodies.
Digital Transformation in Turbulent Times
Early estimates of the economic impact of the pandemic suggest a harder hit than that of the Great Depression on the 1930s. In the face of turbulent times, business strategies will likely turn to self-preservation and cost reduction across a wide range of industries. But this does not mean that operations can cease for long and organisations will need to look to the future.
Despite the emerging medical crisis, businesses still need to crunch numbers, analyse trends, gain insights, and make decisions. In fact, in a time of increasing uncertainty, they need to make better decisions with greater urgency and wider import at higher levels of risk. This demands — more than ever – the highest quality, well-understood data, delivered at speed and in considerable volumes.
The focus for digital transformation in the coming months will likely shift from market domination to basic survival. Analytics will turn its models away from predicting customer satisfaction or upselling and toward optimizing production and overcoming logistics and distribution issues as global supply chains become brittle or break completely. An opportunity presents itself to data strategists and business owners: this relative low period for operations is a good time to ensure that data strategies are adequate for the inevitable ramp up on the other side of the coronavirus crisis.
Other implications are still emerging. With business focus shifted from marketing magic to logistics and operations, data quality requirements may finally come to the fore, especially in data lakes. Real-time data may finally be limited to areas with real-world impact and value, rather than instant gratification of consumer desires.
Redefining the Data Strategy
Over the past decade, many organizations have developed data delivery and analysis pipelines where the early stages are performed by data scientists and analysts working on premises with data from highly structured warehouses and more fluid lakes. They depend on a physical infrastructure consisting of a mix of powerful local or remote storage and processing connected through high-bandwidth, secure networking. Cleansed data, prepared information, and pre-baked insights are made available to decision makers who may rework them lightly or more intensively on their laptops or smartphones in downstream self-service work.
Ongoing social distancing and home working by data scientists and analysts disrupts this pipeline, shifting the heavy-lifting processes to widely distributed, largely unsecured home offices on public networks. This presents obvious hardware, software, network capacity, and security infrastructure challenges to IT departments.
Of more significance are the data strategy and architecture questions it raises. Where should the data be stored? Who is responsible for its governance and how will that be enforced? How will these new multilocational pipelines be built and managed? How will problems of duplicate data and change synchronization be addressed?
A simplistic analysis might suggest that businesses with cloud-centric data strategies will be in a stronger position than those that have been slower to move from on-premises approaches. After all, the argument is that cloud data is highly distributed and widely accessible whether from office cubicles or kitchen tables.
However, this leap to the physical solution masks significant governance challenges when sensitive data (in terms of privacy, commercial confidentiality, timeliness, consistency, and other factors) is distributed across on-premises, cloud, and now remote workstation stores and processors. A revamped distributed data architecture — extending existing data warehousing and data lake concepts, embracing extensive data virtualization, and data cataloging — will be needed in short order to address these challenges.
Now is the Time to Reassess Data Strategies and Processes
It’s early days and these ideas are highly speculative at this stage. However, there’s little doubt that all our current business and IT certainties are in flux as a result of the Coronavirus pandemic. This is no time to panic, but it certainly is a good time to reassess your data strategies as you set up your home office.
We are here to help
During this time of uncertainty, Data Clarity are committed to helping organisations. Our data auditing service help companies understand their current data management strategies and make recommendations to prepare you for the challenges of today and tomorrow.
