Skip to Content

Blog Archives

Creating a continuous integration server in a hackathon, and how to make SSH connections using ruby

Rails Rumble wrapped up recently and with that, the end of an exciting 48 hour experience where developers can challenge themselves, prove all their programming skills and create an awesome product from an idea.

This is the third hackathon I’ve participated in since I began working as a professional programmer, and I wanted push myself and create my own version of a continuous integration server.

While creating a continuous integration server we went through a lot of challenges, from user experience design to interesting architectural decisions, but one of the most challenging and interesting situations I went through was running the build jobs from a repository in a virtualized environment. At the very beginning we realized that the safest way to run the tests for a given repository was to isolate the test environment from the web server environment due to because of security risks, so we decided to setup the architecture in the following way:

 

CI2

 

 

 

The idea was to connect to a remote (virtualized) server over the SSH protocol and run the script with a provisioned environment (ruby, rvm, rubygems, postgresql, sqlite, mysql, etc.). We spent some time researching how to connect via SSH using ruby and found a library called Net::SSH which allows you to create SSH connections easily and execute a command. We did some tests and it worked but unfortunately it was very hard to navigate through folders and request a bash environment just like a normal SSH connection from the UNIX terminal, so after a long researching, testing, and reverse engineering many open source projects that use Net::SSH we decided to create abstraction layers for each of its components (use-cases).

 

CI3

 

 

By giving single responsibility to each of the classes we were able to easily build the programming interfaces on top of the CI module (see SOLID).

The simplest case scenario, you can connect to a server just by instancing the objects from the top level class of the CI module as following:

Pretty easy, right? Let’s take a look inside the module:

This class is only responsible for setting up the connection parameters that Ci::SSH will handle as a connection string, so we have encapsulated the Ci::SSH work in a lower level of the Ci namespace. You can actually use it outside the Ci::Environment class but you have to customize it as seen in thedef initialize method above. Now let’s take a look at how the Ci::SSH works.

By defining these two classes the usage of Net::SSH becomes pretty straight forward:

By the end of the hackathon, all this made communication possible between the continuous integration environment and the UI. We connected the shell output to a websocket using the pusherservice, so we could push notifications from the server in real time to the user; you can see it live by visiting the actual project from RailsRumble Simple CI.

Questions?

Let me know via comments on this post or via email antonio@tangosource.com.

0 0 Continue Reading →

What? Rails can’t hear you? Use Flash!

First I’d like to mention that this post is not for beginners, that means that I’m assuming that you know about programing with Ruby on Rails (for this post I’m using Rails 3.1) and some of Javascript. If you don’t know Rails or JS at all, you may need to do some research about the terms and instructions I say here. With that said, lets start!

Have you ever needed to record and save user’s audio by using computer’s microphone using Rails? If so, I bet you realized that there is almost no information about how to do this using Rails.

The Story

I was working in a project where the customer needed to implement a feature to allow users to record his voice by using his computer’s microphone. I thought “this can’t be that hard, there is HTML5 now,” but I had no idea what I was talking about.

Researching: HTML 5 or Flash?

I started researching about HTML5 and it didn’t take me that much to see that HTML5 is an “still in progress work“ when it comes to record live audio. The browser that currently has the best implementation for HTML5 audio tag is Chrome, and yet it needs the user to configure some of its flags for this tag to work properly, so I discarded HTML5 as an option.

I continued researching and soon I started reading in many posts that the best approach to accomplish what I was trying to do was by using Flash in the client side. I know, I don’t like Flash either, but lets face it, most browsers support Adobe plugins and this is better than having the application working only in Chrome, so I decided to give it a shot, and took the Flash & Rails path.

Once I decided to go with Flash, the next impediment I had was that I’m not a Flash programmer, but that was solved when I bumped into the jRecorder plugin (I want to thank to Sajith Amma because he certainly saved me hours learning Flash). jRecorder is a plugin that uses a Flash object to get access to user’s microphone and records the microphone input. This plugin has also an API built in javascript to initialize and interact with the Flash object.

Ok, enough background, let’s jump into the interesting part:

Coding

Download the jRecorder plugin, and unzip the file. You’ll see 7 files; the ones we need are: jRecorder.js, jRecorder.swf, and jquery.min.js

In your Rails application:

Explaining the code

This is the DOM’s object where the flash object will be inserted.

The js code is actually the constructor of the js object that will interact with the Flash object (we can say it is a sort of an API). You can read the documentation of the plugin here, however there are 2 parts that I would like to explain since the example that our good friend Sajith implements is in PHP and you may get confused:

In ‘host’ we are defining the route where we want to receive the recorded audio, in this case, for my application <%= audio_path %> points to a controller named audio in the action upload_file that we’ll use to save the audio file. You can create a different route, just make sure it uses the POST method because we are sending information to the server, and filename=audio.wav is a parameter that contains ONLY the name (that means NOT the recorded audio). The result of “<%= audio_path %>?filename=audio.wav” would be something like ‘/audios/upload_file?filename=audio.wav’.

The other interesting option that you need to translate in the example from PHP to Rails is ‘swf_path’. This option is where you set the path of where jRecorder.swf object is located, and all you have to do is to use Rails helper ‘asset_path’ and pass to it the name of the flash object.

At this point we are done with the client side.

Jumping into the backend

Go to the controller that responds to the route that you specified in ‘host’ parameter in the js code, in my case it’s the action upload_file in audios_controller.rb. If you already read the jRecorder documentation that means that you’ve seen this:

(PHP code)

This code is very self-explanatory and you can infer that all this code is doing is receiving the recorded audio, opening a file, writing the recorded audio, and closing the file. Now, what is the equivalent code in Rails? Here it is:

The interesting line here is ‘request.raw_post’. That line contains the actual audio recorded by the user that was sent by our view using the plugin. That line is the equivalent of ‘file_get_contents(‘php://input’)’. Also check that we are opening the file with ‘b’, which is very important because the file we are reading/writing is a binary file (otherwise you’ll face encoding issues).

That’s it! You should now have the file stored in the root path of your application!

Improve it

Depending on your application, there are many things that you can improve, like converting the file from .wav to .mp3 or .aac formats using Zencoder, uploading the file to S3 buckets, implement background jobs to save the audio, etc.

Got questions?

Let me know in the comment section below.

0 0 Continue Reading →