Integration Strategy – Integration Principles

A well-structured integration strategy must be grounded in core, non-negotiable principles that have been collectively agreed upon across the enterprise. While certain aspects of these principles may vary between organizations, their foundational elements must remain intact. The objectives, purpose, and capabilities they are designed to enable should not be compromised by other internal influences or priorities.

Through years of experience in this field, alongside my colleagues, we have identified several principles we consider essential. While other enterprises may adopt additional ones, a common baseline of integration principles is typically consistent, and can be summarized as follows:

  • API based integrations
  • Service-Oriented Architectures
  • Loose Coupling and High Cohesion
  • Interoperability
  • Hybrid Integration Platform (HIP)

These five integration principles collectively shape the approach to integrating enterprise solutions. Depending on specific needs—often related to security—additional principles may be incorporated. However, the above list represents a comprehensive framework of mission-critical elements that underpin a successful integration strategy.

API Based Integrations

APIs have become a ubiquitous topic in today’s industry, often associated primarily with REST APIs. However, this narrow view overlooks the broader concept, its historical roots, and its long-standing presence in the industry. API-based integration is a development approach in which a product’s core functionalities are made available through well-defined APIs from the very beginning of the design process. The key takeaway here is that APIs are considered “from the start” rather than as an afterthought.

This principle became clear to me through a valuable collaboration with top specialists at FedEx, during a partnership program I was fortunate to be a part of several years ago. Their excellence in this field was particularly evident and changed forever my mind and my way to see how successful enterprise care about integration of their SW assets. The way they referred to this approach was “API-First,” which is also the terminology we have adopted within our enterprise since then, thanks to the outstanding results of this experience we were lucky enough to have.

The key emphasis is on designing APIs that prioritize the needs of the broader ecosystem they support. This means APIs should not be treated as an afterthought, but rather as the foundational layer on which the entire product is built, facilitating seamless integration and collaboration with external systems and services. APIs must be the primary interface to focus on from the very start, becoming a core component of your design strategy.

In my many years of experience with application integrations, I’ve encountered countless situations where APIs were developed only “if requested,” “if necessary,” or “in case someone asks for them.” This approach is a guaranteed path to failure, unless the system has been specifically designed to accommodate future integration needs based on substantial prior experience from top developers or architects.

When an API is designed after a service has been delivered or (even worse) after a service got traction in your business, you have very high chances to deliver a piece of junk, to deliver sub optimal interfaces. Here’s a refined version of that statement: This is because your interfaces will inevitably have to adapt to inefficient code, resulting in a suboptimal and compromised solution.

There are several rationales behind this. Let’s try to elaborate some (far from being a fully comprehensive list):

  • Enabling Service based design, and hybrid integrations: API based design serves as a foundational ingredient in crafting a robust and efficient service-based environment. We live in a world where solutions evolve fast and incrementally. This approach is an essential aspect of modern architectural paradigms such as SOA (but I like more the term Service Based Design), Microservices and ecosystem composability.
  • Enabling reusability and speed to Value: By exposing capabilities, data, functions through APIs from the outset, the API based approach encourages the reusability of developed components across different parts of the enterprise. Write once, use many times, avoid duplications, simplification of the efforts. The objective is to speed up the development process of the consumer services, increasing speed-to-market and achieve faster the wanted business value.
  • Contract based relationships: Applying API based design implies that a well-defined contract between service providers and consumers is in place. A contract compliance is a must in this approach, once you create a dependency, the dependency must be honoured and therefore clear rules must be in place. The beauty of this is that it minimizes integration hiccups, malfunctioning, enhances communication, and creates a culture of collaboration and trust between different parts of the ecosystem within your enterprises and among organizations that collaborate.
  • Making systems integration predictable: We live in a world of complex technological environments with hundreds, sometimes thousands part evolve independently. In such environments, a proper API design is essential for seamless systems integration. A proper API Based design addresses the challenges posed by various technologies, stacks and practices and promotes the creation and evolution of APIs that ease integration. By doing so, it naturally promotes interoperability.
  • Simplicity, comprehensibility, and usability as an obsession: A crucial aspect of API design within the API-First paradigm must be an obsession for simplicity, comprehensibility, and usability. APIs designed with this in your mind are intuitive and user-friendly. If they are intuitive, easy to be used and user friendly, the become easy to be used for developers to be adopted and work with. If they conquer the hearth of your developers, they will naturally become a dominant patter leading to a more efficient development process.

Implications in applying this principle are many, some of them can be summarized in these points.

  • Start right from the beginning, on a clear definition: The API based design starts from the need of the others. You need to have a clear definition of what is needed when you expose something to others via APIs. This is the foundation to the rest; this paves the road to success. If you design well your interfaces, the definition of what is your scope the way to interact with it you make sure you give the right direction in the way the API is constructed. Having good quality APIs with a proper definition ensures that APIs are at the core of the development process. Developers will love using your APIs since good APIs make their life easier. Do otherwise and you will notice guaranteed low adoption and experience failure.
  • Outside-In API design: APIs should be designed from an “outside-in” perspective, prioritizing the requirements of the broader ecosystem within the enterprise. This approach ensures that APIs creation and subsequent evolution are driven by the needs of the different stakeholders in the enterprise, enhancing their relevance and usability. No one wants to use a clunky and sub optimal API that only partially achieve their goals.
  • Comprehensive service exposure: Every capability offered by solutions should be exposed and made accessible in the form of APIs. This practice facilitates the creation of a composable mesh of services within the ecosystem. This is the opposite of the other approach where APIs are developed only “if needed”. This enables flexible and efficient integration between different components and certainty that, “if a capability exists, a corresponding API to interact with it exists as well”.

Service Based Architectures

Every software component in an enterprise’s ecosystem must be designed with a service-based architecture in mind. I believe everyone knows SOA and a lot of people know how many casualties it caused throughout the last decades. There was a moment of time when SOA was a mantra, a moment when JAVA application servers were sold for millions of dollars from big companies. It was like a religion and people religiously applied it. All of this was driven by good intentions, but you know better than me that the path to the hell is paved by good intentions.

Together with my team and the practice we deliver, we like to have a milder interpretation of the SOA principal. We call it Service Based Architectures. Does it change by a lot? No. It just changes enough to differentiate from a practice we don’t believe it is still actual, the differences are not obviously only linked to the name.

What we mean by Service Based Architectures (SBA) is that everything should be designed as a component of a bigger ecosystem when and if it makes sense. If a component has any hope to be reusable, if the project has enough traction in the organization and the team behind it is valid, the component must be created with the scope to be reusable outside its original scope of usage.

By doing this, enterprises enable a scenario of strong modularity. Modularity by reflex enables reusability and composability. This approach must be set in stone in every practice in the enterprise. Developments are never isolated things and never end in themselves.

Does it look similar to the previous principle for API based integrations? It might look similar but they see the integration and the composability from different standpoints, APIs is just a way to connect components, service based design is mainly a disciplined execution model.

There are so many rationales behind this which is difficult to capture all of them. Some noteworthy ones are:

  • SBA increases speed to market compared to a practice to develop solutions monolithically. Each component is easier to be verified compared to the entire monolith and functionalities can be shared by design.
  • SBA fosters organizational alignment. Having specialised and dedicated teams allocated on specific components make the development practice easier.

Implications

  • This is not a new concept in the industry, many architectural styles approach this model such as SOA, microservices, event-driven, and hybrid-based architecture. Each of these approaches’ present pros and cons. What really fits your needs depend on you. The advice here is to not be too extreme or to ignore this completely. Two sides of the same medal requiring proper analysis and proper experience.
  • Most of the Service based architectures rely on clear what is called eventual consistency to ensure the data processed by separate deployment units is at some unspecified point in time all synchronized into a consistent state. This makes the development a bit more complex if not done by experienced people.
  • The transition from a monolithic (layered) architecture to a service-based architecture is not an easy job (and, sometimes, inconvenient). Microservice architectures have many benefits, and it is recommended for greenfield projects. The SOA style is suitable where a strong level of orchestration between business services and enterprise services is required. Hybrid-based architecture could be considered a middle ground between service-oriented architecture and microservices.
  • Service-based architecture styles, while offering possibly superior performance, scalability, and availability compared to monolithic architecture styles, also come with notable trade-offs.
  • A Layered monolithic approach is, in my opinion, still an acceptable pattern but should be limited to prototype developments or situations where the rapid deployment of a new service is a critical requirement. Bearing in mind the SBA model also when such approach is consciously being used can save sometimes from a complete refactoring of your entire codebase.

Loose Coupling and Cohesion

This is a principle and an approach to systems design that I will never get tired to stress out its importance. When you design your systems and the components forming those systems you need to have the objective to design a loosely coupled yet well encapsulated set of solutions.
Each component and, more generally, each of the implemented solutions, must be designed in a way that make them capable of tolerating system failures or malfunctioning and, ultimately, resume once the required dependencies are back to a normal status.
I think it is useless to be said yet important to be recognized that louse coupling applies to distributed systems. Distributed systems are complex environments that require expertise and skills to be managed. Implementing a monolithic monster growing to the limit of technology is a walk in a park sometimes. Implementing a system able to scale, evolve and with the right coupling requires discipline and knowledge.
This methodology is backed up by different rationales and considerations, all of them equally important, I believe they can be summarized in the following:

  • Loosely coupled refers to a design attribute in systems, characterized by minimizing interdependencies between modules or components. This approach focuses on creating interfaces that reduce the likelihood of changes in one module negatively impacting others. The primary objective is to enhance system flexibility, making it easier to add, replace, or modify individual modules without affecting overall functionality.
  • This approach becomes foundational for architectures that aim to grow in volumes, numbers, and complexity. Any other approach becomes challenging is this model is not adopted.
  • Loose Coupling is by far one of the paradigms that benefits the most from a RESTful API development and the other core principles such as API Based and Service Based designs.


There are several implications to be taken in consideration when this model is adopted as well such as:

  • If systems are designed in a tightly coupled system (considering as an extreme of it, monolithic solutions), the performance a system can offer is defined by its slowest component. This applies also to reliability, in a tight coupling design, the failure of a side component can easily jeopardize the rest of the capabilities. On the contrary, if implemented components are decoupled, systems can be designed to tolerate failure and scale independently from each other.
  • To achieve a properly designed and implemented loose coupled architecture, two sub-paradigms are required, bringing a ton of advantages and positive side effects which are single-responsibility design and separation of concerns patterns.
  • Transactional integrity is an extremely important problem to be faced in this paradigm since loosely coupled design pairs with distributed systems, potentially difficult animals to be tamed.

Leave a comment