The Great Migration: Peculiar Complexities of Monolith to Microservices Migration

This article reviews the peculiar complexities that threaten the success of the monolith to microservices migration projects and offers a solution.

There are peculiar complexities that threaten the success of the monolith to microservices migration projects by making them difficult and costly to undertake.

Unfortunately, you cannot hide or ignore these complexities.

Even if, somehow, you manage to complete a migration project by evading the complexities, the party wouldn't last long.

You will end up getting haunted by complex ghosts for years to come.

Let me explain why I am saying this, but before I do that, let us set the context right so that we are on the same page.

Let’s assume you want to migrate a monolithic ERP system to a microservices architecture.

You aspire to utilize the cloud-native infrastructure.

You hope to bring in latest infrastructure into play.

You want to go serverless.

You want to add more bells and whistles to microservices.

You believe in the promise that your costs of operations will come down once you have created these new services.

The new application will be easy to maintain and scale.

Fair expectations.

Nothing wrong with these.

But it is not the curse of the shiny things where your problems will start.

Not that the promise of microservices was oversold, it is just that you did not prepare for the real complexities that you were going to face to fulfil the promise.

Enough of the scare.

Let’s see how you would typically approach a monolith to microservices migration of an ERP system and, then, we will see what exactly the complexities are.

The Typical Approach

Typically, if you have got it right, you will undertake the migration using what is called a strangler pattern.

You will start by analysing the ERP system modules.

You will then design a logical breakdown of the modules into probable services.

Then, you will identify the first batch of modules that can be moved out. The obvious choice will be the ones with the least dependencies.

You will rewrite the features of these modules as microservices and, then, let microservices shadow the modules for some time.

Once the microservices are stabilized, strangulate/disconnect the original modules.

You will continue this process for other modules until all of them have been moved to microservices.

Easy-peasy-lemon-squeezy! Right?

Not quite.

There Are Complexities and More

All along you did not realize that sneaking behind this innocent approach of strangulation, there are monsters of complexity ready to blow your head off with nightmares and unending pain.

Here is what you need to know about these monsters:

The feature transfer from old architecture to new distributed architecture is not as straightforward as you think. It is a paradigm shift monster. You will have to rewrite the old features using different event-driven patterns while making sure in the process you are not losing the essence of the functionality. Actually, you can't lose any functionality.

If you design service boundaries wrong, which most likely you will, it will lead to very messy microservices, and they will bring unnerving pain for years to come. To save yourself from this future agony, you will have to spend a lot of time, effort, and attention on domain design. But isn’t the domain already designed in the ERP? Yes and No. Did you see this one coming?

You will repeatedly be writing lots and lots of plumbing code for the new microservices architecture. Regression issues will start blowing your costs off the roof.

You will be swarmed with plumbing code for the new and old infrastructure making it challenging to move out from old infrastructure bindings.

If you make wrong choices in architecture and domain design early on, you might repent and pay for it later. The ghosts of your wrong choices will haunt you, maybe forever.

In the process of handling these complexities, you might run out of time to rewrite features using scalable and extensible design patterns. You will be forced to copy old patterns and code from the legacy application. This will make you wonder if the entire process was worth it.

The crux of the matter is this: when you had told us that you will lead the way to microservices, knowingly or unknowingly you were promising us that you will provide a better architecture, better code, and better design.

So, not fulfilling that promise makes you look bad. Very bad, isn’t it?

Yes, there is this promised land of microservices. But you must make sure that you are ready to make the journey.

How the Complexities Playout During Development

Nevertheless, you decide to jump into it, the blazing guns of glory. What happens next?

Let us go a little deeper to see how your Mad Max movie plays out in the real world of project deadlines. First off, you are bumped into spending a lot of time in domain design because if you did not do that, you know you will not last long.

But the project sponsor loses patience when you tell them after a month that you have not written any useful line of code except for the design. You know what will happen next. You will be iterating the newly designed features a lot. You will be repeatedly designing, redesigning, and testing the new event-driven avatars of the old features. This will eat up your time.

“Are we there yet? Are we? When will it be done?” such questions will loom over your head every day. Then every few hours. And then they will get louder. You will be designing the architecture of the new application before writing any code. You better be. It cannot be an afterthought.

See, you are not writing an application from scratch. You are turning an already working healthy application into microservices, whose only sin in this world seems to be that it is a monolith.

You cannot leave the “architecture design” for later consideration when the application has scaled to a large number of users, blah blah blah. Nope. You deserted that option long back. You will be writing a lot of plumbing code for the new event-driven features.

Soon you will run into standardization issues, naming problems, and more. You will pull your hair off when names start conflicting such as:

Remember _customername_ can be _custname_, _cusname_, _custnm_ or _cusnm_?

And, that is just domain names. Imagine this happening to your plumbing code. That’s the California wildfire. Your fascination for cloud-native infrastructure will make you write even more plumbing code and things will start getting weirder.

So, here is the thing, while the above-mentioned work is doable, it seems like you will have to throw the budget, time, and quality constraints out of the window.

Then the million-dollar question is:

Is it practical to undertake a monolith to microservices migration at all?

Yes, it is, and that is the reason I have written this article. This is not a doomsday article. The below section is what you can do to make it happen.

Proposed Solution

The problem with microservices migration is that there is a sheer lack of tooling that can help developers to solve these complexities elegantly. It is like writing an ERP software using Assembly code. So, what can you do?

This is where I would like to suggest an approach any team can follow:

You should put together a distributed event-driven architecture design with the necessary infrastructure at the very outset. Make sure you have this design ready before you make any commitment to the migration.

There should be some tooling where you can quickly iterate and test the event-driven avatar of the new features. And be able to redo the features correctly all over again without much hassle and delay.

Work out design patterns for loose coupling of components for the event-driven architecture. What is the point of an event-driven architecture if you end up making tightly coupled components?

And, because you will need these patterns again and again, you need to find a way to templatize them. Even if somewhere, you must bypass these patterns (due to project pressures of course), you should be able to refactor the code later using the templates. Set coding standards. Especially for the plumbing code, or even better, figure a way to auto-generate the plumbing code.

Let me give you a hint, if your architecture design is set, then you know that your plumbing code will follow a certain repetitive pattern. Therefore, around 80% to 90% of plumbing codes can be auto-generated. Introduce some common services and tech early on like, single sign-on, API gateway, and containerization.

If you can do this, then what is left is the work that you really wanted to do in the project. That is following the strangler pattern and migrating to microservices.

Summary

I hope that I was able to help you identify the complexities, and I hope this will help you execute your microservices migration projects carefully. Hopefully, you will be better prepared and avoid many pitfalls.

The solutions that I have mentioned above can be cooked up using exciting development tools and implementing tight controls in projects. Though it might be cumbersome, you will be able to complete projects.

ZippyOPS Provide consulting, implementation, and management services on DevOps, DevSecOps, Cloud, Automated Ops, Microservices, Infrastructure, DataOPS, and Security

Services offered by us: https://www.zippyops.com/services

Our Products: https://www.zippyops.com/products

Our Solutions: https://www.zippyops.com/solutions

For Demo, videos check out YouTube Playlist: 

https://www.youtube.com/watch?v=4FYvPooN_Tg&list=PLCJ3JpanNyCfXlHahZhYgJH9-rV6ouPro

If this seems interesting, please email us at [email protected] for a call.

Relevant Blogs:

Defining schedule in pipeline Enabling 

webhook in pipeline 

Role based access control 

Defining paramertes in a pipeline