Skip to Content

Blog Archives

Dev jobs: A microservices architecture journey

Recently at Tangosource I was assigned to work on an internal project named Dev Jobs (see dev jobs on iOS and dev jobs on Android), where I had the chance to work next to great teammates and implement a microservice architecture with Ruby on Rails, nodeJS, JWT, Ionic, Docker, and several testing tools, including RSpec, Mocha, Frisby, and Protractor. In this first post, I’ll explain the basics of the microservice architecture.

What is a Microservice Architecture ?

Martin Fowler gives a great definition.

[1]“The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.”

The Problem

Let’s have some context. We are building a server side application that must support different client side platforms such as:

  • Desktop browsers
  • Mobile browsers
  • Native mobile applications

 

Also, we need to consume an external API service. In our case, Linkedin through Oauth2 for authentication. And the ability for users to chat between each other.

On an architectural matter we still had to decide between monolithic or microservice. So let’s dig a little more on this two.

Monolithic vs MicroServices

Monolithic

If we take this approach, we need to keep the following in mind:

  • Build one application to rule them all.
  • Use one language ( Ruby, PHP, JavaScript, etc. ) we could not take any extra advantages from other programming languages.
  • The architecture needs to be flexible enough to support API through AJAX and render HTML.
  • The development team needs to feel comfortable with the language we picked.
  • Security. We will have a public API to serve mobiles/clients.
  • Best tools for RTCP. So we can have a nice looking chat
  • Testing. build testable code

 

I have seen great monolithic architectures that can support all of this. One of my favorites is RoR engines based or nodeJS services based applications.

MicroServices

In the other hand, if we go for MicroServices we must take in mind the following:

  • Delegate responsibilities in small applications.
  • We can use different languages per service. Which is the best choice per responsibility?
  • How are we going to communicate each service ? Normally with tcp/ssl.
  • How are we going to persist Data through services?
  • Security: Protect the services.
  • Testing: Test all microservices.

 

On a personal note, as a developer working on a microservice platform, I can focus only on the technical details wrapping a specific service. This does not mean I should not understand the whole set of services as a unit.

“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.”

— Melvyn Conway, 1967

Having this in mind, we had to start by making technical decisions, such as how to protect our API, how to manage RTCP, by nature HTTP is stateless which means is sessionless so for our case we had to maintain a session through tokens ( JWT ), at first we though maybe nodeJS for RTCP since nodeJS is strong for DIRT ( Data Intensive Real Time )  apps. But we had not only had to take care of the tech devs but also for dev resources, currently at TangoSource we have strong Ruby on Rails devs so we had to work all together and take advantage of the expertise of each team member, our best bet was to use RoR for business logic, and take advantage of nodeJS. So the combination of RoR and nodeJS makes the solution solid and strong.

Solution

For this particular project, we decided to go for the MicroService and have some fun! The first thing that we needed to figure out was defining the responsibilities per service. So we end up with 4 microservices.

  • Ionic
    • More than a service, it’s the client side of the application. Ionic by itself is a front end SDK for hybrid mobile apps using AngularJS.
  • IDP
    • Responsible for (a) providing identifiers for users looking to interact with a system, (b) asserting to such a system that such an identifier presented by a user is known to the provider (c) possibly providing other information about the user that is known to the provider.
    • This may be achieved via an authentication module, which verifies a security token, and that can be accepted as an alternative to repeatedly and explicitly authenticating a user within a security area.
  • SP
    • In charge of persisting all user’s data, including interaction with the system. This service will contain all the business logic
  • Chat
    • In charge of establishing real-time communication among users within the system.

 

We decided to protect the services with JWT

Architecture

devjobsarchitecture

Each square represents a service, and the lines are how they communicate between each other.

Ionic : This might be the most critical service of them all, but not the most important. We have 3 main components:

  • Ionic server
    • An HTTP server in charge of sending all the assets ( HTML, CSS, JS ) to the next component.
  • Ionic Cordova
    • A Mobile SDK, which provides the user’s interaction with the app, and consumes the rest of the microservices as resources. This component runs on top of 3 different type of devices.
      • Android
      • Ios
      • Browser
  • Ionic App
    • Ionic admin panel, this panel allow us to configure GCM and APNS exposing a protected API.

 

IDP : This service is in charge of giving identity to a component/user. This service will expire a valid token with a component/user’s basic information on the JWT’s payload. Similar to memoization, this service will keep track of sessions using Redis on a basic TLS connection. Thanks to the RFC 7519 standard, this service can scale in other microservice based architectures and be reused.

SP : This is an API/Https service protected with JWT. This micro service is in charge of persisting all user’s interaction/information using MySQL with a standard TLS connection. It also manages the business logic.  

Chat : This microservice will provide real-time communication between users. This RTCP (real time control protocol) is being protected with JWT on the handshake process, and persists the communication activity on MongoDB, a NoSQL database.

Conclusion

Taking these kind of decisions is not simple. It requires a lot of experience in order to compare libraries/languages, you must at least know how they work, and it’s even better if you have used them. Remember that you will not build this by yourself, you are part of a team. It’s smart to sit with your team and discuss each individual’s strengths and weaknesses in order to make the best tech decisions.

DO NOT mix responsibilities of the micro services, instead identify your services. Add all the needed documentation to help the team such as /*Comments */,  READMEs, and Wikis.
A microservice architecture is not always the best approach, and could be painful if you are not careful.

3 8 Continue Reading →

A Tmux crash course: tips and tweaks.

~ Intro

If you are one of those devs who uses the terminal a lot and ends up with way too many tabs open, or practices pair programming, then this post is for you. During the last months, I’ve started using Tmux a lot. Since I’ve found it to be very useful, I thought I would write a post where I share a few recommendations and pro-tips. I’ll show you what Tmux is and how to use it in combination with Vim to make a more effective and elegant use of the Terminal.

So, this is what we’ll cover:

  1. Tmux basics.
  2. The best of Tmux
    1. Windows
    2. Panes
    3. Sessions
    4. Fast text navigation and copying
    5. And a very neat pair programming feature
  3. Tweaks to improve Vim integration.
    1. Colorscheme background
    2. Static cursor shape
    3. Indentation at pasting
  4. A few extras to enhance the Tmux experience.
    1. Tmuxinator for session automation.
    2. Changing your color of your Tmux bar.

 

An important thing to bear in mind, this is the tool stack I had installed while writing this post, I tested what I say here with these versions:

  • Tmux 1.9a
  • Vim 7.4
  • iTerm 2.1
  • Mac OS (Mavericks and Yosemite)

 

Let’s start!

 

~ The Basics

What is Tmux?

Tmux is a tool that allows running multiple terminal sessions through a single terminal window. It allows you to have terminal sessions running in the background and attach and detach from them as needed, which is very useful. Later on, we will see how to make the most out of that feature.

Here’s a screenshot of a Tmux session:

What you see in the image:

– Left: Vim

– Right: a system’s shell

– Bottom-left: the Tmux session name (“pomodoro-app”)

– Bottom-middle: the current Tmux windows (“app log”, “editor” and “shell”)

– Bottom-right: the current date

How to install Tmux?

In Mac OS:
  1. Install Homebrew

Here’s more info.

  1. Install Tmux:

In Ubuntu:

Run this on the terminal:

The Tmux prefix

In order to isolate its own keyboard shortcuts from other shortcuts, Tmux provides a shortcut prefix. When you want to trigger a Tmux shortcut you will press the Tmux prefix and then the Tmux shortcut key. The prefix that Tmux uses by default is Ctrl-b (“Ctrl” key in combination with the “b” key). For instance, let’s say you want to trigger the shortcut that lists the current Tmux sessions, which is the *s* key. Here is what you will need to do:

  1. Press Ctrl-b keys (Tmux prefix)
  2. Release Ctrl-b keys
  3. Press the s key

 

A few recommendations:

  1. If you have not already mapped the *ctrl* key to the *caps-lock* key and vice-versa I suggest you do it.

 

Calling ctrl from the caps-lock key is very practical. This is because when coding you need to call ctrl very frequently. Moreover, it is a lot easier/quicker given the caps-lock key aligns with the default position of your fingers in the keyboard.

 

  1.  I recommend changing the Tmux prefix to Ctrl-a . Once the  *Ctrl* key has been set to the *caps-lock* key, it gets a lot easier/quicker to call Ctrl-a instead of Ctrl-b, because the new prefix keys are very close to each other on the keyboard.

 

Here is what you need to add in your ~/.tmux.conf file to change the prefix to Ctrl-a:

 The config file

*~/.tmux.conf* is a file that Tmux reads every time a new session opens. It’s where the customizations for Tmux need to be placed. Suggestion: in the case that you need (and chances are you will) to apply a new change made to the without opening a new session, you can add the following line to the ~/.tmux.conf file:

This way, once you have added a new change to the *~/.tmux.conf* file, just press ctrl-b R to reload your configuration without having to open a new Tmux session.

 

~ The best of Tmux

Quick note: the screenshot shown here may differ slightly from what you see by default when you install Tmux. This is because I modified the status bar. If you want to do the same follow the steps on the “Pimp your Tmux bar” section of this post.

Panes

I like the idea of dividing the screen vertically, so that on one side of the screen I have Vim and on the other side I have the output of my code. I could even have another console if I wanted to. Here’s how Tmux makes it happen:

What you see in the image:

– Left side: Vim (at the top: a Ruby class file, at the bottom: a test file for the Ruby class).

– Right side: a bash session.

A vertical *pane* is easy to create. Once you have launched a new Tmux session just press “Ctrl-b %” and a new vertical pane will appear. Also, if what you need is a horizontal division then press “Ctrl-b “”. Navigating through Tmux panes is easy, just press the Tmux prefix and then any of the direction arrows depending on which pane you want to go.

 Windows

In Tmux, a window is a container for one or more panes. Tmux windows allow you to arrange multiple panes inside windows depending on what you need. For instance, in my case I usually have one window called “server” for running the app’s server (where I can see the log), another window called “editor” (where I do the coding). One pane for Vim and another for running the code tests. And another window called “shell”, which is a bash shell where I can run system commands. Tmux windows are useful since they allow users to allocate panes inside of them, to see more about each pane by going to the window that contains it. This is an efficient use of the available screen space.

The list of existent windows in a Tmux session displays at the bottom of the screen. Here is an example of how Tmux displays (by default) the list of windows created. In this case, there are three windows: “server”, “editor” and “shell”):

image3

 

In order to create a new window you need to press “Ctrl-b c”. To navigate through windows press Ctrl-b followed by the index number of the window you want to go. The index number displays next to the name.

Sessions

A Tmux session can contain multiple windows. Sessions are a neat feature; I can create a Tmux session that is exclusive to a particular project. To create a new session just run the following command on your terminal:

If I need to work on a different project I will just create a new session for that. Although the focus will be on the new session, the original session will remain alive. This allows me to get back to it later, and continue where I left off. To create a new session press Ctrl-b : and then enter the following command:

Tmux sessions remain alive until you restart your machine or you explicitly kill the session. As long as you don’t restart your machine, you can jump from a project’s session to another as you need it.

 

Navigation through Tmux sessions

To get a list of the existing sessions, press Ctrl-b s. Here is an example of what Tmux will show you:

image4

Each session listed has an ID number, starting from zero. In order to go to that session type the session’s ID number in your keyboard. In the case that you are not in Tmux but you have one or more sessions running just use:

This command will take you back to your Tmux sessions.

Fast text navigation and copying

I always disliked the fact that to copy content from iTerm2 quickly you need to use the keyboard plus the mouse. I think that there should be a quicker way to do it without having to use the mouse. Fortunately, Tmux allows for that, since it is run from the command line, where the use of the mouse is not allowed.

Navigate text

Tmux allows for text navigation in a way that is very similar to Vim. You know, where the k key goes one line up, w moves one word forwards, and so on. Yet you can increase Tmux’s similarity with Vim by telling it to use the *vi* mode. Here is what you need to add in your *~/.tmux.conf* file to accomplish this:

Send copied text to System’s clipboard

By default, when you copy text from Tmux, the text is only available to be pasted inside that same Tmux session. In order to make that text available to be pasted anywhere, you have to tell Tmux to copy to system’s clipboard. Here’s how to do it:

  1. Install retach-to-user-namespace, this is very easy with brew, just run the following command:

  1. Update ~/.tmux.conf file

Select and copy text

Now that the *vi* mode is set and *rettach-to-user-namespace* installed, let’s see how to copy text from a Tmux session. Let’s say you’ve run the *ifconfig* command because you wanted to copy your ip address. Now, follow these steps to copy that text:

  1. Enter copy mode: ctrl-b [. You’ll see a short highlighted text appear at the top right of the screen as in the following image (“*[0/0]*”).

 

image5

  1. Start moving across the text as you would do in Vim: with j, k, l, h, etc..
  2. Once you get to the text you want to copy press the spacebar and start selecting text (exactly as you would do it in Vim).
  3. Once text is selected press the enter key

 

Now that you have copied your IP address, paste it wherever you want.

Make text copying even more Vim-like

You can use *v* key to select text and *y* to copy it, just add the following to your *~/.tmux.conf* file:

Effective pair programming

You can share the address of a Tmux session with someone else, and that person can connect to the session via *SSH*. Since the connection runs over *SSH*, it will be very lightweight. For the user connecting to the remote Tmux session, it will feel as if the session is running locally, providing the internet connection is fast enough.

Tmux with Tmate

Tmate is a tool that makes it very easy to create a Tmux session and share it with someone else over the internet. To share a new Tmux session using Tmate these are the steps you have to follow:

  1. Install Homebrew:

  1. Install Tmate

  1. Launch a new session with Tmate:

  1. Copy the SSH URL given by Tmate on the Tmux session. An example is showed in the following image (message at the bottom: “*[tmate] Remote session: ssh …*”):

 

image6

  1. Ask the other person to access via *SSH* using the URL you just copied.

 

Now that you know how to make good use of Tmux’s pair programming feature, you can further improve the interactivity of your session by doing a voice call via your preferred provider.

 

~ Tweaks for Vim integration

Colorscheme background

When I first opened Vim through Tmux I found the colors weren’t being correctly applied. The background color was being displayed only where characters appeared. Here is an example:

image7

This issue is due to Vim’s need of setting a different term parameter when ran through Tmux. To set the right term parameter just add the following lines to your *~/.vimrc* file:

After updating the ~/.vimrc file, the color scheme is displayed correctly:

image8

Static cursor shape

By default when Vim is run through Tmux, the cursor shape is always the same regardless of the current Vim mode (insert, visual, etc). This makes it hard to identify the current Vim mode. To fix this, you need Tmux to tell iTerm to update the cursor shape. You can do it by adding the following to your ~/.vimrc file:

*Thanks to Andy Fowler who originally shared this tip.*

Indentation at pasting

Sometimes, when pasting text into Vim the indentation of the text is changed, which is a problem when you paste a large amount of text. This issue can be prevented by executing :set nopaste before pasting. However, there is a better way to do this. By adding the following to your ~/.vimrc file, Vim will automatically prevent auto-indenting the text when pasting:

*Thanks to Marcin Kulik who originally shared this tip.*

 

~ Nice extras

Tmuxinator (automate sessions for your projects)

Let’s say you start coding for application A and you always create a Tmux session with three windows for that: “servers”, “editor” (for the project’s code) and “shell” (to run system commands). At certain point of the day you need to temporarily switch to application B, and you will create a Tmux session with the same windows arrangement you had for application A, with a few slight differences (such as the directory and some commands). With Tmuxinator, you can declare the configuration for each Tmux session and then create them with a single command! Pretty sweet.

Tmuxinator is a gem that allows you to automate the creation of Tmux sessions. It is done specifying the details of the sessions in the configuration files and then creating the sessions with a command.

Let’s see how to install Tmuxinator and how to add the configuration to start a project session. Install the Tmuxinator gem by running the following command:

Now that Tmuxinator is installed you can run the tmuxinator or mux commands from the system’s shell. Let’s create the configuration file for your first application as described above (three windows: “servers”, “editor” and “shell”), tell Tmuxinator to create and open the config file for this project:

At this point the file ~/.tmuxinator/project_a.yml should have been automatically opened. In order to accomplish what is needed for project A you need to update the content of project_a.yml to:

Once you have added the configuration to the Yaml file for project A, just run the following command to start the Tmux session:

Or, if you prefer, use the Tmuxinator alias:

There you have it. Now, in order to start coding for project A just run the Tmuxinator command.

*Find the official documentation in the Tmuxinator repo.

Pimp your Tmux bar

By default, the Tmux bar looks like the following (green bar at the bottom of the image):

image9

You can change its appearance if you want. In my case I like something cleaner as in:

image10

In order to accomplish it, I use the following settings in my ~/.tmux.conf file:

~ Conclusion

In summary, we revised basic functionality and the most useful features Tmux offers. We also reviewed a few tweaks and extras. What are your impressions about Tmux so far? Are there any other features/tweaks that you have found useful? Let us know in the comments section below.

Thanks for reading!

TN.

12 28 Continue Reading →

What? Rails can’t hear you? Use Flash!

First I’d like to mention that this post is not for beginners, that means that I’m assuming that you know about programing with Ruby on Rails (for this post I’m using Rails 3.1) and some of Javascript. If you don’t know Rails or JS at all, you may need to do some research about the terms and instructions I say here. With that said, lets start!

Have you ever needed to record and save user’s audio by using computer’s microphone using Rails? If so, I bet you realized that there is almost no information about how to do this using Rails.

The Story

I was working in a project where the customer needed to implement a feature to allow users to record his voice by using his computer’s microphone. I thought “this can’t be that hard, there is HTML5 now,” but I had no idea what I was talking about.

Researching: HTML 5 or Flash?

I started researching about HTML5 and it didn’t take me that much to see that HTML5 is an “still in progress work“ when it comes to record live audio. The browser that currently has the best implementation for HTML5 audio tag is Chrome, and yet it needs the user to configure some of its flags for this tag to work properly, so I discarded HTML5 as an option.

I continued researching and soon I started reading in many posts that the best approach to accomplish what I was trying to do was by using Flash in the client side. I know, I don’t like Flash either, but lets face it, most browsers support Adobe plugins and this is better than having the application working only in Chrome, so I decided to give it a shot, and took the Flash & Rails path.

Once I decided to go with Flash, the next impediment I had was that I’m not a Flash programmer, but that was solved when I bumped into the jRecorder plugin (I want to thank to Sajith Amma because he certainly saved me hours learning Flash). jRecorder is a plugin that uses a Flash object to get access to user’s microphone and records the microphone input. This plugin has also an API built in javascript to initialize and interact with the Flash object.

Ok, enough background, let’s jump into the interesting part:

Coding

Download the jRecorder plugin, and unzip the file. You’ll see 7 files; the ones we need are: jRecorder.js, jRecorder.swf, and jquery.min.js

In your Rails application:

Explaining the code

This is the DOM’s object where the flash object will be inserted.

The js code is actually the constructor of the js object that will interact with the Flash object (we can say it is a sort of an API). You can read the documentation of the plugin here, however there are 2 parts that I would like to explain since the example that our good friend Sajith implements is in PHP and you may get confused:

In ‘host’ we are defining the route where we want to receive the recorded audio, in this case, for my application <%= audio_path %> points to a controller named audio in the action upload_file that we’ll use to save the audio file. You can create a different route, just make sure it uses the POST method because we are sending information to the server, and filename=audio.wav is a parameter that contains ONLY the name (that means NOT the recorded audio). The result of “<%= audio_path %>?filename=audio.wav” would be something like ‘/audios/upload_file?filename=audio.wav’.

The other interesting option that you need to translate in the example from PHP to Rails is ‘swf_path’. This option is where you set the path of where jRecorder.swf object is located, and all you have to do is to use Rails helper ‘asset_path’ and pass to it the name of the flash object.

At this point we are done with the client side.

Jumping into the backend

Go to the controller that responds to the route that you specified in ‘host’ parameter in the js code, in my case it’s the action upload_file in audios_controller.rb. If you already read the jRecorder documentation that means that you’ve seen this:

(PHP code)

This code is very self-explanatory and you can infer that all this code is doing is receiving the recorded audio, opening a file, writing the recorded audio, and closing the file. Now, what is the equivalent code in Rails? Here it is:

The interesting line here is ‘request.raw_post’. That line contains the actual audio recorded by the user that was sent by our view using the plugin. That line is the equivalent of ‘file_get_contents(‘php://input’)’. Also check that we are opening the file with ‘b’, which is very important because the file we are reading/writing is a binary file (otherwise you’ll face encoding issues).

That’s it! You should now have the file stored in the root path of your application!

Improve it

Depending on your application, there are many things that you can improve, like converting the file from .wav to .mp3 or .aac formats using Zencoder, uploading the file to S3 buckets, implement background jobs to save the audio, etc.

Got questions?

Let me know in the comment section below.

0 0 Continue Reading →