Understanding Microservices

Microservices represent a modern approach to software design. In this paradigm, numerous self-contained services interact with each other, each possessing its own dedicated processes and resources. This approach deviates from the traditional client-server model. Typically, client-server applications are composed of one or more clients, a monolithic backend encompassing all domain data and logic, and an API that enables clients to interact with the backend and utilize its functionality.

In the realm of microservices, the aforementioned monolithic backend is deconstructed into a collection of distributed services. This architecture offers several advantages: improved segregation of responsibilities, streamlined maintenance, enhanced flexibility in selecting technologies for individual services, and greater scalability and fault tolerance. However, complex distributed systems introduce their own set of challenges. They are more susceptible to race conditions and debugging becomes more intricate because problems are not localized to a single service but spread across multiple ones. Without adhering to best practices during development, managing such a system can become overwhelming. Careful attention must be paid to service payload contracts, as modifications in one service can impact all its clients and, consequently, the entire backend service suite.

While these considerations are crucial, let’s assume you’ve already addressed them. Now, you’re eager to learn how to construct a microservices backend independently. Let’s delve into that.

Establishing a Microservices Architecture

Numerous approaches exist for setting up microservices. This guide will concentrate on a broker architecture.

Exploring Broker Architectures

A broker architecture facilitates communication among services. In this setup, services connect to a central messaging server, the broker. Services transmit messages to the broker, which then determines the appropriate service(s) to receive those messages. This eliminates the need for services to retain information about each other. Instead, they rely on the broker for messaging, enabling them to remain isolated and focused solely on their specific domain. Brokers can also temporarily store messages when intended recipients are unavailable, allowing senders and receivers to operate asynchronously. While advantageous, this approach has drawbacks. The broker can become a bottleneck as all communication flows through it, and it can also represent a single point of failure. However, these issues can be mitigated. Implementing multiple broker instances in parallel enhances system fault tolerance. Another solution is utilizing other architectures. Alternative architectures differ from this guide’s implementation by either not employing a broker, using a different broker architecture, or adopting a different messaging protocol like HTTP.

Inter-Service Communication

This guide employs ZeroMQ for managing communication between services and the broker.

ZeroMQ provides a protocol abstraction layer for handling multipart asynchronous messages across various transports. The advantages of using ZeroMQ for service-broker communication fall outside this guide’s scope. For a deeper understanding, refer to the following Quora article. To explore alternative inter-service communication methods, consider examining Broker vs. Brokerless article.

Constructing the Microservices Suite

This article will guide you through creating your microservices suite. Our system comprises a broker and a service. A small client script will be used to test calls to the service suite; this client code can be easily repurposed.

Let’s commence building.

Getting Started

Ensure you have the necessary components to run the broker and the service. Download and install Node.js, ZeroMQ, and Git. OSX users can find Homebrew packages, while most Linux distributions offer packages as well. Windows users can utilize the provided download links.

Broker Operation

After installing dependencies, let’s initiate the broker. This guide uses a Node.js broker implementation from the ZMQ Service Oriented Suite. The code and documentation are available on GitHub. Clone the Broker bootstrap to your machine. This repository provides a bootstrap for using the broker library, offering the flexibility to modify default configurations.

Execute the following Git command:

1

$ git clone git@github.com:dadah/zmq-broker-bootstrap.git

Navigate to the created directory:

1

$ cd zmq-broker-bootstrap

Install package dependencies:

1

$ npm install

The broker is ready. Run:

1

$ bin/zss-broker run

Configuration files for various environments reside in the config/ directory. The default development configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
 "broker": {
   "backend": "tcp://127.0.0.1:7776",
   "frontend": "tcp://127.0.0.1:7777"
 },
 "log": {
   "consolePlugin": {
     "level": "debug"
   }
 }
}

The backend parameter specifies the ip:port address for the broker’s backend and frontend. The backend address handles requests and replies from services, while the frontend interacts with service clients. The log.consolePlugin.level controls logging verbosity. Options include trace, debug, info, warn, and error, dictating the amount of logging information generated by the broker.

Service Operation

With the broker running, let’s create your first Ruby microservice. Open a new console window and navigate to your desired service directory. This guide utilizes the Ruby client and service of ZMQ SOA Suite. A bootstrap “Hello world” service is available to expedite the process.

Clone the bootstrap repository:

1

$ git clone git@github.com:dadah/zmq-service-suite-ruby-bootstrap.git

Navigate to the new directory:

1

$ cd zmq-service-suite-ruby-bootstrap

Install dependencies:

1

$ bundle install

Start the service:

1

$ bin/zss-service run

Your first service is running.

The broker’s console window should display:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

2015-12-15 16:45:05 | INFO | BROKER - Async Broker is waiting for messages...

2015-12-15 16:45:14 | DEBUG | BACKEND - received from: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: 76f50741-913a-43b9-94b0-36d8f7bd75b1
2015-12-15 16:45:14 | DEBUG | BACKEND - routing from: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: 76f50741-913a-43b9-94b0-36d8f7bd75b1 to SMI.UP request...
2015-12-15 16:45:14 | INFO | SMI - SMI register for sid: HELLO-WORD instance: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b!
2015-12-15 16:45:14 | DEBUG | BACKEND - reply to: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: 76f50741-913a-43b9-94b0-36d8f7bd75b1 with status: 200
2015-12-15 16:45:15 | DEBUG | BACKEND - received from: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: 3b3a0416-73fa-4fd2-9306-dad18bc0502a
2015-12-15 16:45:15 | DEBUG | BACKEND - routing from: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: 3b3a0416-73fa-4fd2-9306-dad18bc0502a to SMI.HEARTBEAT request...
2015-12-15 16:45:15 | DEBUG | BACKEND - reply to: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: 3b3a0416-73fa-4fd2-9306-dad18bc0502a with status: 200
2015-12-15 16:45:16 | DEBUG | BACKEND - received from: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: b3044c24-c823-4394-8204-1e872f30e909
2015-12-15 16:45:16 | DEBUG | BACKEND - routing from: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: b3044c24-c823-4394-8204-1e872f30e909 to SMI.HEARTBEAT request...
2015-12-15 16:45:16 | DEBUG | BACKEND - reply to: hello-word#aaa65374-8585-410a-a41d-c8a5b024553b rid: b3044c24-c823-4394-8204-1e872f30e909 with status: 200

This indicates that the broker recognizes the new service and is receiving heartbeat messages, confirming the service instance’s availability.

Service Consumption

How do we use the running service?

The bootstrap repository includes a dummy client for testing the “Hello World” service. In a new console window or tab, navigate to your service directory and execute:

1

$ bin/zss-client

You should observe:

1
2
3

15-49-15 16:49:54 | INFO | ZSS::CLIENT - Request 90a88081-3485-45b6-91b3-b0609d64592a sent to HELLO-WORD:*#HELLO/WORLD with 1.0s timeout
15-49-15 16:49:54 | INFO | ZSS::CLIENT - Received response to 90a88081-3485-45b6-91b3-b0609d64592a with status 200
"Hello World"

The service’s console window should show:

1
2
3
4

Started hello-word daemon...
15-45-15 16:45:14 | INFO | ZSS::SERVICE - Starting SID: 'HELLO-WORD' ID: 'hello-word#aaa65374-8585-410a-a41d-c8a5b024553b' Env: 'development' Broker: 'tcp://127.0.0.1:7776'
15-49-15 16:49:54 | INFO | ZSS::SERVICE - Handle request for HELLO-WORD:*#HELLO/WORLD
15-49-15 16:49:54 | INFO | ZSS::SERVICE - Reply with status: 200

You’ve successfully launched and consumed your “Hello World” microservice. Now, let’s move beyond the basics and build custom services.

Service Development

Stop the “Hello World” service (Ctrl+C in its console window). We’ll transform it into a “Person” service.

Code Structure

Let’s examine the project’s code structure:

• bin: Contains scripts for launching your service.
• config: Stores configuration files.
  • boot.rb: Add service dependencies. Existing dependencies are listed; add more as needed.
  • application.yml: Stores application settings (explored later).
  • config/initializers: Add initializer scripts (e.g., ActiveRecord, Redis) executed on service startup.
• db/migrate: Store database migrations (ActiveRecord, Sequel). This directory can be removed if not used.
• lib: Houses your main application code.
  • settings.rb: Loads application.yml, making configurations accessible throughout the service.
  • service_register.rb: Register services and service routes (explained later).
  • hello_world_service.rb: Defines the “Hello World” service’s endpoints.
  • lib/daos: Store ActiveModel objects (ActiveRecord) or other data access objects (e.g., Sequel models).
  • lib/dtos: Store data transfer objects (DTOs) representing data sent to clients.
  • lib/repositories: Store repositories responsible for data access (using DAOs) and returning DTOs.
  • lib/repositories/mappers: Store mappers for converting between DAOs and DTOs.

The application.yml file in the config directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

defaults: &defaults
  broker:
    backend: tcp://127.0.0.1:7776
    frontend: tcp://127.0.0.1:7777
  logging:
    console:
      level: info

development:
  <<: *defaults

test:
  <<: *defaults

production:
  <<: *defaults

This sets the broker’s backend and frontend addresses, and the logging level.

Don’t worry if this seems overwhelming; it will become clearer as we proceed.

Implementing the “Person” Service

Let’s configure the database connection. Open config/initializers/active_record.rb and uncomment the line within. In application.yml, add the following to your development configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

defaults: &defaults
  broker:
    backend: tcp://127.0.0.1:7776
    frontend: tcp://127.0.0.1:7777
  logging:
    console:
      level: info
  database:
    adapter: postgresql
    database: zss-tutorial-development

Create the database. For a default PostgreSQL database, run:

1

$ rake db:create

For other databases, add the appropriate gem to the Gemfile and run bundle install.

Next, create a migration file named 000_creates_persons.rb in db/migrate:

1

$ touch db/migrate/000_creates_persons_table.rb

Define the migration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

class CreatesPersons < ActiveRecord::Migration

  def change
    create_table :persons do |t|
      t.name
      t.timestamps
    end
  end
  
end

Run the migration:

1
2
3
4
5
6

$ rake db:migrate
== 0 CreatesPersons: migrating ================================================
-- create_table(:persons)
DEPRECATION WARNING: `#timestamp` was called without specifying an option for `null`. In Rails 5, this behavior will change to `null: false`. You should manually specify `null: true` to prevent the behavior of your existing migrations from changing. (called from block in change at /Users/francisco/Code/microservices-tutorial/db/migrate/000_creates_persons.rb:6)
   -> 0.0012s
== 0 CreatesPersons: migrated (0.0013s) =======================================

Create a model for the table in lib/daos/person.rb:

1

$ touch lib/daos/person.rb

Edit the model:

1
2
3
4

module DAO
  class Person < ActiveRecord::Base
  end
end

Create a DTO model for “Person” in lib/dtos/person.rb:

1

$ touch lib/dtos/person.rb

Define the DTO:

1
2
3
4
5

module DTO
  class Person < Base
    attr_reader :id, :name
  end
end

Create a Mapper to convert between the DAO and DTO in lib/repositories/mappers/person.rb:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

module Mapper
  class Person < Mapper::Base

    def self.to_dao dto_instance
      DAO::Person.new id: dto_instance.id, name: dto_instance.name
    end

    def self.to_dto dao_instance
      DTO::Person.new id: dao_instance.id, name: dao_instance.name
    end

  end
end

Mapper::Base requires implementing self.to_dao and self.to_dto or overriding self.map to handle DAO-DTO conversions.

Now, create the repository in lib/repositories/person.rb:

1

$ touch lib/dtos/person.rb

Define the repository:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

module Repository
  class Person < Repository::Base

    def get
      DAO::Person.all.map do |person|
        Mapper::Person.map(person)
      end
    end

  end
end

This repository’s get method retrieves all persons from the database and maps them into a collection of person DTOs.

Create the service and its endpoint in lib/person_service.rb:

1

$ touch lib/person_service.rb

Define the service:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

class PersonService < BaseService

  attr_reader :person_repo

  def initialize
    @person_repo = Repository::Person.new
  end

  def get payload, headers
    persons = person_repo.get()
    if persons.empty?
      raise ZSS::Error.new(404, "No people here")
    else
      persons.map &:serialize
    end
  end

end

The “Person” service initializes the repository. Public instance methods have optional payload and headers (both Hashie::Mash instances) for storing data sent to the endpoint. Replies mimic HTTP responses with status codes (e.g., 200 for success, 500 for server errors, 400 for parameter errors). The get endpoint returns a 404 status code if no people are found, similar to HTTP. DTOs are serialized into JSON objects by default.

Register the service by modifying lib/service_register.rb. Replace all instances of “HelloWorld” with “Person”:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

module ZSS
  class ServiceRegister

    def self.get_service
      config = Hashie::Mash.new(
        backend: Settings.broker.backend
      )

      service = ZSS::Service.new(:person, config)

      personInstance = PersonService.new

      service.add_route(personInstance, :get)

      return service
    end

  end
end

The string in add_route is omitted if the service verb matches the method name (e.g., GET verb maps to get method).

The ServiceRegister class defines self.get_service for initializing, connecting, and binding the service:

1
2
3
4
5

config = Hashie::Mash.new(
  backend: Settings.broker.backend
)

service = ZSS::Service.new(:person, config)

1

personInstance = PersonService.new

1

service.add_route(personInstance, :get)

1

return service

Create an executable script for the “Person” service. Modify bin/zss-service, replacing “hello-word” with “person”. Run the service:

1
2
3
4
5
6

$ bin/zss-service run
Starting person:
	PID: ./log
	LOGS: ./log
Started person daemon...
15-29-15 19:29:54 | INFO | ZSS::SERVICE - Starting SID: 'PERSON' ID: 'person#d3ca7e1f-e229-4502-ac2d-0c01d8c285f8' Env: 'development' Broker: 'tcp://127.0.0.1:7776'

Test the service by editing bin/zss-client. Change sid to “person” and the call from hello_world() to get(). Run the client:

1
2
3
4
5

$ bin/zss-client
/Users/francisco/.rvm/gems/ruby-2.1.2/gems/zss-0.3.4/lib/zss/client.rb:41:in `new': No people here (ZSS::Error)
	from /Users/francisco/.rvm/gems/ruby-2.1.2/gems/zss-0.3.4/lib/zss/client.rb:41:in `call'
	from /Users/francisco/.rvm/gems/ruby-2.1.2/gems/zss-0.3.4/lib/zss/client.rb:55:in `method_missing'
	from bin/zss-client:12:in `<main>'

A ZSS::Error occurs because the database is empty.

Handle the error by modifying zss-client:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

begin
  client = ZSS::Client.new(sid, config)
  p client.get()
rescue ZSS::Client => e
  if e.code == 404
    p e.message
  else
    raise e
  end
end

Run the client again:

1
2

$ bin/zss-client
"No people here"

Add people to the database:

1

$ rake service:console

1
2
3
4
5
6
7
8

$ rake service:console
[1] pry(main)> DAO::Person.create name: 'John'
=> #<DAO::Person:0x007fe51bbe9d00 id: 1, name: "John", created_at: 2015-12-16 13:22:37 UTC, updated_at: 2015-12-16 13:22:37 UTC>
[2] pry(main)> DAO::Person.create name: 'Mary'
=> #<DAO::Person:0x007fe51c1dafe8 id: 2, name: "Mary", created_at: 2015-12-16 13:22:42 UTC, updated_at: 2015-12-16 13:22:42 UTC>
[3] pry(main)> DAO::Person.create name: 'Francis'
=> #<DAO::Person:0x007fe51bc11698 id: 3, name: "Francis", created_at: 2015-12-16 13:22:53 UTC, updated_at: 2015-12-16 13:22:53 UTC>
[4] pry(main)> exit

Run the client:

1
2

$ bin/zss-client
[{"id"=>1, "name"=>"John"}, {"id"=>2, "name"=>"Mary"}, {"id"=>3, "name"=>"Francis"}]

You have a working “Person” service.

Concluding Thoughts

While the code might seem verbose, following the repository pattern (depicted below) is beneficial.

Contributions to the SOA service suite are encouraged.

This guide provided a starting point for building microservices. A complete version is available on GitHub.