THE NEW PARADIGM: COMPLEX ADAPTIVE SYSTEMS

As the world continues to evolve towards sophisticated software systems, we are seeing the emergence of very large and complex adaptive systems where the value is enabled by the creation of models and analysis of data, rather than by mere software per se. Given the rapidly rising value of data in all Smart Systems, it’s truly amazing how misunderstood data management’s role is.

This new paradigm—complex adaptive systems—is driving data, information and, more importantly, their interactions towards real-time, state-based, context-driven capabilities that integrate people, processes, physical equipment and knowledge to enable collective awareness and better decision making. Aggregation, transformation, modeling, and management of data from sensors, machines, equipment, systems and people is becoming the holy grail of machine learning.

A rapidly growing number of new software start-ups are focusing on data management tools and infrastructure. This encompasses everything from automated ingestion and pipelines to transformation, storage, modeling, data exchanges, data catalog, components, and more. Much of the growth and value creation within the software infrastructure arena has been mostly about data. Our analysis and forecast places the value of data management tools at someplace north of $50 billion and if you look at new venture investment, funding for the new data tools and solutions has risen to over $10 billion.

SMART SYSTEMS ARE REALLY ABOUT THE DATA MODELS

As networks continue to invade the “physical” world, traditionally unique components and interfaces between and among electronic as well as electro-mechanical elements are becoming more and more standardized. The implications of these trends are enormous, and no product development organization or its suppliers will be able to ignore the forces at play. Product and service design will increasingly be influenced by common components and sub-systems. Vertically defined, stand-alone products and application markets will increasingly become a part of a larger “horizontal” set of standards for hardware, software, communications and data.

As it becomes easier to design and develop smart systems, competitive differentiation will shift away from unique product features towards how the product is actually used, how the product fosters interactions between and among users in a networked context, and, most importantly, how the data from the product will inform these new insights. Even though we have been steadily designing devices and products with more and more intelligence, this information has gone largely unleveraged and unharvested.

Machine data of the real physical world can offer extraordinary business advantages to the companies that understand how to organize that data and model the behavior of the physical world. The ability to detect patterns from large scale sensor and machine data is the “holy grail” of smart systems because it allows not only data patterns but a much higher order of intelligence to emerge from large collections of ordinary machine and device data.

Smart Systems technologies are combining with new innovations in data and information architectures to work together in unprecedented ways to solve more complex business problems than previous generations of computing.

BUSINESS MODELS THAT DRIVE DATA ARE DIFFERENT ANIMALS

Before delving into the new thinking that makes all this possible, let’s talk about why it’s necessary at all. Don’t we already have big data and analytics tools? Aren’t these tools helping us to manage and analyze all this data we keep hearing about?

Almost everyone will answer with a resounding “Yes!” But consider this analogy from Buckminster Fuller: Suppose you are traveling on an ocean liner that suddenly begins to sink. If you rip the lid off the grand piano in the ballroom, throw it overboard, and jump on it, the floating piano lid may well save your life. But if, under normal circumstances, you set about to design the best possible life preserver, are you going to come up with the lid of a grand piano?

Today’s so-called data management tools are like that piano lid. In a period of great change and tumult, that strategy has worked in the sense that it kept us afloat. But that does not make it the best possible design, or qualify it to be something that we should plan to live with forever.

Today, software infrastructure professionals speak a great deal about “data management” tools that can be made available anywhere, anytime, for any kind of data and information. However, the tools we are working with today to manage and analyze data coming from intelligent sensors and machines were not designed to handle the diversity of device data types, nor the massive volume of datapoints generated from real-time machine and equipment interactions. These challenges are diluting the ability of technical organizations to efficiently and effectively organize the data to model it and analyze it. The fragmented nature of software offerings available today to transform, model and analyze data make it extremely difficult, time consuming and costly to get results.

Today, many people refer to machine learning and AI solution opportunities as the “software” business. Since machine learning requires diverse software tools and development work, many industry participants believe that AI/ML solutions are a natural extension of the software business. While we would not dispute that AI/ML development resembles the software business, we believe these solutions are ushering in new business models that could prove to be quite different from those that have succeeded in software.

No doubt, data modeling and machine data intelligence will transform how we think about and conduct business. But we believe that the underlying business and operating model will have many economic characteristics that differ greatly from today’s software SaaS models.