Problem Statement: Brittle Integrations and Decentralized Data Cost You Money
Transportation companies have a lot in common. In an industry that can be slow to adopt new technology and quick to mergers and acquisitions (M&A), companies today are often stuck with evaluating the performance of a technology stack that was built through acquisition, and largely without an enterprise architecture (or plan), therefore under-performing and expensive to maintain.
So what is a brittle architecture and why is a point-to-point integration bad? Gartner defines point-to-point (P2P) as “... the simplest form of integration, but it can introduce technical debt with a complex spaghetti architecture that is hard to manage and change. Application leaders responsible for integration need to understand where and where not to use point-to-point integration.” While it’s simplicity makes it attractive to engineers, point-to-point integration is considered an anti-pattern by most enterprise architects. As this pattern repeats, an engineering team’s development velocity declines as the effort to maintain or update features grows due to the complexity associated with multiple point-to-point integrations. Mulesoft. A leading provider of p2p integration software has written a wonderful article that outlines the traps of P2P integration.
So what does this mean for you? If you’ve already done the work and evaluated your technology architecture, it’s time to start planning for and building out your “Star Schema.”
What is the Star Schema and why is it your new best friend?
The Star Schema is another name for the hub-spoke model that we in the transportation industry have known for a very long time. It represents a central data store (I like to call it a data puddle) that aggregates and normalizes all of the data objects from the various applications on the spokes. It looks something like this:
Your enterprise software architecture is made up of various software components or applications that each have their own definitions of data entities (engineers call them objects). Think of a Transportation Management System (TMS) that moves “loads” and a Warehouse Management System (WMS) that moves “inventory.” Both technically represent the same thing but each application has defined them differently. For this we need to build a “data babelfish” or universal translator that converts the data entities from each application into standardized objects that represent enterprise definitions for storage and reflection to other applications.
A real-time data pipeline is used to relay or reflect incoming data back out to other applications that have shared concerns. Imagine a “load” that has just crossed a geo-fence and is about to be delivered as “inventory” to a warehouse. Both the TMS and the WMS are “concerned” and have different jobs to do when the data entity (load/inventory) state changes. This leads to a very important requirement of the Star Schema Architecture; each spoke thinks it’s the hub.
Each spoke thinks it’s a hub
To pull off an effective Star Schema Architecture, the applications sitting on each spoke need to think they are the hub or center-of-the-universe. We build a facade (fancy for translation layer) that sends and receives data as if the source application is full control, when it actually is participating in a much larger system. We do this because software companies, especially transportation software companies, are dis-incentivized to write their applications to be integration friendly. We’ll address these incentives in a later blog post. Suffice it to say, that if they don’t build their applications to share data, customers will be forced to pay more and less likely to jettison their software. We’re here to fix that.
What does the Star Schema do for you?
The Star Schema Architecture is a powerful design. In addition to connecting and normalizing the data flow between applications there are a number of extremely important benefits it can deliver.
Centralizing data makes it yours (an asset)
Your company is thriving today because it is competitive and does things differently than the competitors. Because of this you should be thinking about your data on your terms instead of those of the software company that only deals with one facet of your business. Properly designed and maintained data is also an asset. If you have 10 years of centralized, cleaned and maintained data you have a treasure trove of data that can be mined for automation, process improvement, decision support and predictive analytics. Your data scientists will shout GOLD!!! and so will your investors.
Real-time enterprise visibility achieved
Nothing is worse than waiting overnight for data to be processed before you can understand a particular situation. Now imagine if the applications sitting on the spokes of the Star Schema had to wait 8 hours for an update. Your business would come to a screeching halt. Not only is real-time data flow important to ensure that all applications are always up to date, a natural byproduct of this design is always current data in a central location. This makes it very easy to build Business Intelligence (BI) dashboards that can accurately illustrate the state of the entire enterprise. Do you want to know where your assets are? All of them across all business units, the Star Schema is here to serve. Just remember, BI dashboards are cheap, central data is not.
Easily add and jettison software technology
Imagine a world where you could throw out that old TMS that is impossible to use but you're stuck with because it contains all of your data and investment for the last 20 years. In our new Star Schema world we get to apply The Strangulation Pattern to your legacy technology (that old TMS) and slowly hack away the useless workflows until you can shut it off.
But the real value is in the ability to use a Buy-vs-Build process to identify which software applications best serve your users to complete their daily tasks (called workflows). We use the competitive market as a provider of non-differentiating technology. Things that every company does and does not differentiate them competitively, like accounting software. If the market does not offer a workflow that serves your business needs, it’s likely that you have a differentiating process, a process that is different than any of your competitors, and therefore you should invest in creating software that addresses that custom workflow exactly as you need it. This is where you spend your engineering dollars. Invest in building differentiating technologies and buy the rest.
Summing it up
The Star Schema Architecture is an attractive solution for companies looking to “catch up” with technology disruptors because it enables them to centralize data, see transactions in real time, and add and jettison software workflows as the company or the market evolves. The Star Schema is optimized for cloud deployment, builds immense value as data accumulates, and is the key to escape the prison that is legacy software.
Would you like to learn more?
Register below to set up a free consultation and learn more about how Metafora can help you with your technology strategy:
