6

At the moment we have one huge API which is used by our backoffice, our frontend, and also our public API.

This causes me a lot of headaches because when building new endpoints I find a lot of application specific logic in the code which I don't necessarily want to include in my endpoint. For example, the code to create a user might contain code to send a welcome email, but because that's not needed for the backoffice endpoint I will then need to add a new endpoint without that logic.

I was thinking about a large refactor to break our code base in to a number of smaller highly specific service APIs, then building a set of small application APIs on top of those.

So for example, an application endpoint to create a new user might do something like this after the refactor:

customerService.createCustomer(); paymentService.chargeCard(); emailService.sendWelcomeEmail();

The application and service APIs will be entirely separate code bases (perhaps a separate code base per service), they may also be built using different languages. They will only interact through REST API calls. They will be on the same local network, so latency shouldn't be a huge issue.

Is this a bad idea? I've never seen/worked on a codebase which has separated the two before, so perhaps there is a better architecture to achieve the flexibility and maintainability I'm looking for?

Advise, links, or comments would all be appreciated.

Improve this question
1
  • Creating several dedicated & reusable services is indeed a great idea. When I see your workflow, I think you should try to use event driven architecture. So, when a user is created, your application just have to send an event with the contact information of the new user. And another dedicated application with only wait for this kind of events and send the mail to the new customer. If you have several interaction of this kind, it could worth the effort of introducing an event bus. If not your approach, is already valid. Commented Jan 22, 2018 at 15:06

3 Answers 3

Reset to default
2
+25

Your idea of making multiple, well-defined services is sound and really it is the best way to approach this. Going with purely micro-services approach however trendy it might seem, proves to be an overkill most often than not. This is why I'd just redesign the existing API/services properly and follow solid and sound SOA design principles below. Good Resources could be found on both serviceorientation.com and soapatterns.org I've always used them as reference in my career.

Consider what types of services you need

Service types and Layers (image from serviceorientation.com)

  • Entity services are generally your Client, Payment services - e.g. services centered around an entity in your domain. They should be business-agnostic, and be able to be reused in all scenarios. They could be called sometimes by clients directly if sufficient for their needs. They could be called by Task services.
  • Utility services contain logic you're likely to reuse in other services, but are generally not called by the clients directly. Rather, they'd be called by Task and Entity services. An example might be a Transliteration service.
  • Task services combine and reuse Entity and Utility services into meaningful tasks. Most often they are not that agnostic and they do implement some specific business logic. They have meaningful business operations and they are what clients mostly call.

Principles to follow when redesigning

I strongly recommend going over this cheat sheet and making sure everything there is covered when you do your redesign. It's great help.

In general, you should make sure that:

  1. Each service has a common context and follows the separation of concerns principle. E.g. Clients service is only for clients related operations, etc.

  2. Each of the Entity and Utility services is business-agnostic and basic enough. So it can be reused in multiple scenarios and context without being changed. Contract must be simple - CRUD and only common operations that make sense in most usage scenarios.

  3. Services follow a common data model - make sure all the data structures you use are used uniformly in all services in order to prevent need for integration efforts in the future and promote combination of services for clients to exploit. If you need to receive a customer that another service returns, this should be happening without the need for transformation

OK, but where to put the non-agnostic logic?

Now, you have multiple options for abstracting business logic whenever you have a need for complex business functionality. It depends on your scenario what you're going to chose:

  • Leave logic to all clients. Let them combine your simplified services
  • If there is business logic that is commonly implemented in multiple of your applications and has the potential to be reused heavily you can implement a composite service that reuses multiple existing underlying services and exposing the logic.

Service Composability. Concerns on multiple API calls communication overhead.

Well, this is an age-old question - should you make multiple API calls when they will probably create some communication overhead? The answer is - it depends on how complex your scenario is, how much reuse you expect and how flexible you want to be. Also is speed critical? To what extent? In Service Oriented Architecture though, this is a very common approach - to reuse your existing services and combine them in new configurations as needed. Yes, it does add some overhead, but I've seen implementations in very complex environments, for example Telecoms, where thanks to the use of ESB solutions, message queues, etc the overhead is negligible compared to the benefits. Here is a common architecture approach (image from serviceorientation.com):

Composing Web Services to Solve a Bigger Problem

The mandatory legacy refactoring heads-up

More often than not, changing the established contract for multiple existing client systems is a messy business and could very well lead to lots of refactoring and need for looking for needle-in-a-stack functionality that's somewhere deep in the (possibly) legacy code. Business logic might be dispersed everywhere. So make sure you're ready and have the controls, time and will to lead this battle.

Hope this helps

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

1

Is this a bad idea?

No, but this is a big overall question to be able to provide very specific advice.

I'd like to separate this into 3 areas:

  • Approach
  • Design
  • Technology

Working backwards, the Technology is the final and most-specific part, and totally depends on what your current environment is (platforms, skills), and (hopefully) will be reasonable self-evident to you once the other things are in progress.

The Design that you outlined above seems like a good end-state - having multiple, specific, focused APIs, each with their own responsibility. Again, the details of the design will depend on the skills of you and your organization, and the existing platforms that you have. E.g. if you are already using TIBCO (for example) and have a lot invested (licenses, platforms, tools, people) then leveraging some of their published patterns/designs/templates makes sense; but (probably) not if you don't already have TIBCO exposure.

In the abstract, the REST API services seems like a good starting point - there are a lot of tools and platforms at all levels of the system for security, deployment, monitoring, scalability, etc. If you are NGINX users, they have a lot of (platform-independent) thoughts on how to do this also NGINX blog, including some smart thinking on scalability and performance. If you are more adventurous, and have an smart, eager team, a look at Event-driven architecture - see this

Approach (or Process) is the key thing here. Ultimately, this is a refactoring, though your description of "a large refactor" does scare me a little - put that way, it sounds like you are talking about a big-bang change and calling it refactoring. Perhaps it is just language, but what's in my mind would be "an evolution of the 'one huge API' into multiple, specific, focused APIs (by refactoring the architecture)". One place to start is Martin Fowler, while this book is about refactoring software, the principles and approach are the same, just at a higher-level. Indeed, he talks about just this here

IBM talk about refactoring to microservices and make it sound easy to do in one step, but it never is (outside the lab).

You have an existing API, serving multiple internal and external clients. I will suggest that you'll want to keep this interface solid for these clients - separate your refactoring of the implementation from the additional concerns of liaising with and coordinating external systems/groups. My high-level starting approach would be:

  • identify a small (3-7) number of related methods on the API
    • ideally if a significant, limited-scope change is needed anyway with these methods, that is good - business value with the code change
  • design/specify a new stand-alone API specifically for these methods
    • at first, clone the existing model/naming/style
  • code a new service just for these
    • with proper automated CI/CD testing and deployment practices
    • with associated monitoring
  • modify the existing API to have calls to these methods re-direct to call the new service
    • perhaps have a run-time switch to change between the old implementation and the new implementation
  • remove the old implementation from codebase
  • capture issues, assumptions and problems along the way
    • the first pass will involve a lot of learning about what works and doesn't.
  • then repeat the process over & over, incorporating improvements each time.

At some point in the future, when appropriate due to other business-driven needs, the API published to the back-end, front-end and/or public clients can change, but that is a whole different project.

As you can see, if the API is huge (1,000 methods => 140 releases) this is a many-months process, and having a reasonably frequent release schedule is important. And there may be no value improving code that works reliably and never changes, so a (potentially) large portion of the existing API may remain, just wrapped by a new API.

Other considerations:

  • public API? Maybe a new version (significant changes) will be needed sooner than the internal APIs
    • focus on the methods/services used by it
  • what parts/services change the most (have the most enhancement requests approved)
    • these are the bits most likely to change, and could benefit most from a better process/architecture
  • what are future plans for change and where would the API be impacted
    • e.g. change to user management, change to payment processors, change to fulfilment systems
    • e.g. new business plans (new products/services)
    • consider affected methods in the API
  • Also see:

Probably the biggest 4 pieces of advice that I can give is:

  • think refactoring: small changes that don't affect function
  • think agile: small increments that are valuable, testable, achievable
  • think continuous: have a vision for where you will (eventually) get to, then work the process continuously
    • script & automate the processes from code, documentation, testing, deployment, monitoring...
    • improving it every time!
  • you have an application/API that works - keep it working!
    • That is always the first priority (you just need to work to carve-out time/budget for maintenance)
Improve this answer

2 Comments

Hey, thanks so much for all the information. I'll be taking a look through those links you provided later. As a quick follow up though, do you have any specific thoughts regarding my dual-layered API approach? For example, we run Angular on our frontend, and making three different service API calls from the frontend just to sign up a customer (see my example) seems like bad design to me which is why I thought about adding the second layer. Is there a more typical way of solving this problem perhaps?
You really end up with different layers, with different responsibilities and granularities. At a lower level, you will have services/APIs that have pretty focused and specific responsibilities, which means that their API/interface is pretty fine-grained. Easier to avoid cohesion and maintain these services/APIs/interfaces, but more complicated for your clients. So, for a more client/user/business-focused API, this is analogous to an orchestration layer - see thenextweb.com/dd/2013/12/17/… for a quick example.
1

Not a bad idea at all.

Also what are your looking is microservices arch. and with that the question comes is how you break your system into well defined services.

We use Domain Driven Design Arch. to break our system into microservices and lagom framework , which allows every service to be in diff. code base and event driven arch. between microservices.

Now lets look at your problem at low level: you said a service contains code like creating a user and sending a email and one with just creating a user and there might be other code as well.

First we need to understand how many type of code you are writing:

  1. Domain Object Logic (eg: User Object) -- what parameters are valid and all -- this should be independent of service endpoint and should be encapsulated in one Class like user class and we say it an Aggregate in Domain Driven Design terms
  2. Business Reactions -- like on user creation send a email -- using event driven arch. these type of logics are separated into process managers or sagas which could most cases work conditionally like a for user created externally send a mail and for user created internally send a email , by having extra data in the event

Also the current way you are doing it , how are you handling transaction across services???

Improve this answer

2 Comments

We have a monolith Java backend currently. I was thinking of breaking this into smaller code bases and making transactions across services with REST. I don't really know a good way to go about setting up microservices. I'm sure this probably isn't the best way, but so far I haven't really found out the "correct" way to do what I'm trying to do.
transactions across services with REST --- what do you mean by this, you probably need a saga or process manager . Also you could use domain driven design to first understand how your system should have been broken up, then for impl you can use lagom

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.