• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

Spring Microservices in Action: Microservices, Containers and Database Design

 
Ranch Hand
Posts: 75
Mac Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Welcome - I look forward to checking out your book.  I'm interested in reading how the overall architecture of Microservices is discussed in the book - in particular how database design should change, and how containers come into play with microservices in terms of isolation.  I'm also interested in the effects that Microservices have on a Continuous Delivery pipeline.  Looking forward to a book that takes things beyond the buzzwords, and puts Microservices to practice in practical environments!

Thanks
David Sachdev
 
Author
Posts: 93
5
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi David,

Thanks for posting.  The topic of database design is really one of the hardest topics to talk about for people who come from a relational database background because we are used to designing databases around complete applications.  (I know because I started my career as an Oracle DBA and developer 20 years ago :-( ).  Anyways, the key things to keep in mind is that with a microservice, is that a microservice completely owns it own data.  That means that all data access for the domain should be down through the service.  That can be difficult where for data optimization reasons it was often very easy to to perform really complex real-time queries or  across different data domains.  

What I have found is that you usually are going to have to do more calls to a service to get data then by joining.  If you need to optimize and reduce database lookups then you can put a cache in place to hold data near to your service and use events via message bus (like Kafka, or RabbitMQ) to signal when data is not valid anymore and must be refreshed.  This is true for OLTP-based applications.  For reporting, you can propagate copies of the data owned by individual services and renormalize the data into a more performant structure by having the microservice that is making changes publish that a change has occurred.  This is usually where you see things like the "data lake" pattern emerge.

Quick summary:

1.  A service should completely own its data and the data it owns should be relatively small (4 to 5 tables).  No one but the service should be able to update the data accept through the microservice interface.  I have seen people enforce this in multiple ways, including:  convention (catch it during code review), separate access to tables behind individual service passwords (only the service has access to the tables), to the extreme of each service having their own database.

2.  For performance do not be afraid to use caching to keep data that is heavily used from one service near to another service that is consuming it.  Its more expensive, but avoid having multiple services use the same cache.  A down cache could impact multiple services.

3.  Use a message bus to publish changes to data.  This way even if a microservice does not own the data in question it can respond to changes in the state of the data and communicate back to the originating state.

4.  Don't make your microservices too fine grained turned them into distributed DAOs whose sole job is to manage data.  If you start finding a large proliferation of services that is a code smell that you have decomposed your data mode.

5.  My advice with your model is start coarse-grained.  Dont try to make too many microservices upfront.  Its easier to refactor a "fat" microservice then it is to consolidate a bunch of small services.

I need to know a little bit more about what you are looking for around Continuous Delivery.  Chapter 10 in the book deals with the subject.  If you have some specific questions.  

Please let me know.

     Thanks,
        John
 
David Sachdev
Ranch Hand
Posts: 75
Mac Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

John Carnell wrote:Hi David,

What I have found is that you usually are going to have to do more calls to a service to get data then by joining.  If you need to optimize and reduce database lookups then you can put a cache in place to hold data near to your service and use events via message bus (like Kafka, or RabbitMQ) to signal when data is not valid anymore and must be refreshed.  This is true for OLTP-based applications.  For reporting, you can propagate copies of the data owned by individual services and renormalize the data into a more performant structure by having the microservice that is making changes publish that a change has occurred.  This is usually where you see things like the "data lake" pattern emerge.



I have heard a lot about "data lakes" recently in various podcasts including the AWS TechChat, but I haven't really tried to wrap my head around the pattern and concepts yet - now that you have brought it into the context of MicroServices, I will have to go back and pay a bit more attention.  Given the changes in the way data is managed in microservices, do you see any big changes in the way that ORM is done on the horizon?  Just today I heard reference to MyiBatis, and someone pushing to change our JPA layer over, and I realized it has been a while since I've pondered if JPA is still the best path forward on the DB front.  Any thoughts?

 
reply
    Bookmark Topic Watch Topic
  • New Topic