Hub-and-spoke model: An alternative to a data network

Information network is a topical issue in the data and analytics community. Presented in 2020 by Zhamak Dehghani in his article “Information Mesh Principles and Logical Architecture”, data mesh is a new distributed model for organizing analytical teams to deliver data products and is designed to solve both centralized and decentralized data problems. But is this approach really the best approach for today’s businesses?

Organizational models for analytics

Over the years, we have seen both centralized and decentralized organizational models for delivering business analytics. While both models have their advantages, each has some serious drawbacks that make them inadequate to meet the needs of consumers who are hungry for today’s information.

1. Centralized model

The data warehouse allows businesses to store information in a single, selected location so that, in theory, everyone can find and request their information with confidence. With central control over the information platform and standards, data can be consistently identified and delivered reliably.

However, in practice there are several major problems with this approach. First, the data must be compiled and loaded so carefully that only IT has the skills required to set up the data warehouse. This builds IT as a bottleneck for new data integration. Second, the IT team often has difficulty translating business requirements into technical requirements because they do not understand the business, and therefore further strengthen the bottleneck, disappointing their customers. Finally, business users struggle to analyze thousands of database tables, which makes a centralized database attractive only to the most sophisticated users.

2. Decentralized model

With the frustration of the end user and the growing popularity of visualization tools such as Tableau, business users have taken matters into their own hands with a decentralized approach. Instead of waiting for IT to deliver data, business users created their own data extracts, data models, and reports. By decentralizing data processing, business users have been freed from IT and have avoided the problem of “missing in translation” in relation to a centralized, IT-led approach.

In practice, however, this approach, like the centralized approach, has identified some key challenges. First, with no control over business definitions, business users have created their own versions of reality with each board they author. As a result, competitive business definitions and results have destroyed management’s confidence and trust in analytics results. Second, the decentralized approach has led to an increase in competitive and often incompatible platforms and tools, making it difficult or impossible to integrate analytics across business units.

Information network

The data network is designed to solve the problems of both models. It recognizes the distribution of today’s data and allows all users in the organization to access and analyze business concepts from virtually any data source without the intervention of expert data groups. It relies more on people and organization than technology, which is why it is so attractive. Mesh’s distributed architecture decentralizes ownership of each business domain. This means that each domain controls the quality, confidentiality, freshness, accuracy and relevance of the data for analytical and operational use.

However, the data network approach favors a fully decentralized organizational model, with the complete abolition of centralized command. I would like to offer an alternative to this approach, which provides a center of excellence to make the decentralized data management model affordable for most businesses.

Hub-and-spoke model: An alternative to a data network

Clearly, neither a centralized nor a decentralized approach can provide flexibility and consistency at the same time. These goals are contradictory. However, there is a model that can present the best of both worlds if implemented with the right tools and processes.

The Hub-and-spoke model is an alternative to data network architecture with some critical differences. Namely, the hub-and-spoke model represents a central data command or perfection center (“hub”). This team has an information platform, tools, and process standards, and business domain teams (“speakers”) have information products for their domains. This approach addresses the “everything goes” phenomenon of a decentralized model, while empowering topic experts (SMEs) or data managers to create information products that meet their needs.

Critical connection: Information model

Supporting a decentralized, centralized, and conversational model for creating information products requires teams to speak a common information language, not SQL. Needed a logical a method of determining information relationships and work logic that are separate and different from the physical description of the data. The semantic data model is an ideal candidate to serve as the Rosetta Stone for individual data domain groups because it can be used to generate data. digital twins business by adapting physical data to the right conditions for the business. Domain professionals can digitally encode their business knowledge for others to query, communicate, and improve.

For this approach to work on a scale, the application of a common semantic layer platform that supports data model sharing, appropriate dimensions, collaboration, and ownership is critical. With a semantic layer, a central data group (hub) can identify common models and appropriate dimensions (i.e., time, product, customer), while domain experts (words) can define their business process models (i.e., “invoice”, “shipping,” “leading” gen ”). With the ability to share model assets, business users can combine their models with models from other domains to create new mixes to answer deeper questions.

It succeeds because it plays with the strengths of centralized and business domain teams: the centralized team has a technical platform and manages and publishes shared models, while business groups create domain-related information products using a sequential set. without the need to understand business definitions and business models of other domains.

How to get there

The transition to a central and voice model does not have to be disruptive to deliver information products. There are two ways to succeed depending on your existing model for delivering analytics.

If you have a current analytical organization centralized, the central team and business teams should work together to identify key data areas, define data management, and include an analytical engineer in each. The analytical engineer can come from a central team or a work team. Using a semantic layer platform, an internal analytics engineer can work within a business domain team to create data models and data products for that domain. The internal analytics engineer works with a central data team to set tools and process standards when defining general models.

If you have a current organization decentralized, you can create a central data group to create tools and standards for the process. In addition to managing the semantic layer platform and its shared objects and models, the central data team can manage information pipelines and information platforms shared by domain groups.

Building for measurement

The optimal organizational model for analytics will depend on the size and maturity of your organization. However, setting up for scale is never quick. No matter how small, investing in a decentralized model for creating information products will pay dividends now and in the future. By promoting the management and ownership of data by domain experts using a common set of tools and semantic definitions, your entire organization will have the authority to create a wide range of information products.

David P. Mariani He is a CTO and co-founder AtScale, Inc.