Thursday, December 26, 2013

Test Driving the FizzBuzz Kata In Clojure

I've been enjoying my journey down the path of functional languages. I started with Clojure, than looked at Erlang, and now I am back at Clojure again. This post does not try to explain why you should look at functional languages, I assume you passed that since you're reading this blog post.

The books will teach you the basic concepts. Those ideas vanish fast unless I practice them with a project or some kind of challenge. Programming quizzes or coding puzzles are a good way to exercise a new language. I usually start with the simple ones, and I work my way towards the more complex projects. The FizzBuzz kata is on the easier side, I was even asked to code it on a white board not too long ago.

 

Here is how I solved the FizzBuzz kata in Clojure test driving it with its own test-is tool. I describe every changes I made, feel free to follow along by typing in the way I did it.

I'll assume you have Clojure and Leiningen installed on your computer. Getting these tools is simple thanks to Homebrew on OSX. Please do a Google search if you don't have them already installed and you're using an OS other than OSX. I am using Clojure 1.5.1 version at the time of this writing.

I generated a skeleton project with Leiningen:lein new app fizzbuzz. The directory structure had a src and a test directory. lein test runs the tests and lein run executes the program.

I put all my code in the src/fizzbuzz/core.clj and in the test/fizzbuzz/core_test.clj files. I write code in Vim, and I use a vertical split to look at the test and program files in the same window. I usually have the test on the left, and the code under test on the right hand side.

Here is the first failing test I wrote:

(ns fizzbuzz.core-test
  (:require [clojure.test :refer :all]
            [fizzbuzz.core :refer :all]))

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1)))))
I provided the skeleton for the fizz-buzz-printer function this way:
(ns fizzbuzz.core
  (:gen-class))

(defn fizz-buzz-printer [limit])

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  ;; work around dangerous default behaviour in Clojure
  (alter-var-root #'*read-eval* (constantly false))
  (println "Hello, World!"))
When I ran the test with lein test, this is the error I had:

FAIL in (fizz-buzz-printer-test) (core_test.clj:7)
with 1
expected: (= "1\n" (fizz-buzz-printer 1))
  actual:
 (not (= 1
 nil))

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
Tests failed.

The simplest thing that could possibly work is this:

(ns fizzbuzz.core
  (:gen-class))

(defn fizz-buzz-printer [limit]
  "1\n")

...
Please note that I omitted the -main function as it's not changed yet. I ran the tests, and it all passed:

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.

Great! Now I am on to the next test:

...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2)))))
The following code change made the test pass:
...

(defn fizz-buzz-printer [limit]
  (if (= 2 limit)
    "1\n2\n"
    "1\n"))

...
A conditional won't serve me long to print out numbers. I refactored the function this way:
...

(defn fizz-buzz-printer
  ([output current limit]
    (let [product (str output current "\n")]
      (if (< current limit)
        (fizz-buzz-printer product (+ 1 current) limit)
        product)))
  ([limit]
    (fizz-buzz-printer "" 1 limit)))

...
This might need a little explanation. I chose recursion instead of using a loop. The fizz-buzz-printer function is overloaded: it accepts 1 (the limit) or 3 arguments (the output, current value and limit). The formatted string is captured in the product value. If the current value in the cycle is less than the limit I call the same function but the current value is incremented by one, and if not, I know I reached the limit and just return the formatted string - the product.

With this code I should be able to print the numbers on the screen followed by a line break. I wrote a quick - temporary - test to verify that (see the last assert):

...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "with 4"
    (is (= "1\n2\n3\n4\n" (fizz-buzz-printer 4)))))

I was ready for the fun part of this kata! For every number divisible by 3 I had to print out the word: "Fizz".
First I wrote the failing test:

...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "from 1-3"
    (is (= "1\n2\nFizz\n" (fizz-buzz-printer 3)))))
The error was obvious:

FAIL in (fizz-buzz-printer-test) (core_test.clj:11)
from 1-3
expected: (= "1\n2\nFizz\n" (fizz-buzz-printer 3))
actual: (not (= "1\n2\nFizz\n" "1\n2\n3\n"))

Ran 1 tests containing 3 assertions.
1 failures, 0 errors.
Tests failed.

I was expecting the word Fizz, but I was getting the number 3. That logic was missing. Here is how I added that:

...

(defn fizz-buzz-printer
  ([output current limit]
    (let [product (if (= 3 current)
                    (str output "Fizz\n")
                    (str output current "\n"))]
      (if (< current limit)
        (fizz-buzz-printer product (+ 1 current) limit)
        product)))
  ([limit]
    (fizz-buzz-printer "" 1 limit)))

...
I felt generating the formatted string - the product value - put way too much logic in the let function. I extracted out that logic into its own function called format-output:
...

(defn- format-output [output current]
  (if (= 3 current)
    (str output "Fizz\n")
    (str output current "\n")))

(defn fizz-buzz-printer
  ([output current limit]
    (let [product (format-output output current)]
      (if (< current limit)
        (fizz-buzz-printer product (+ 1 current) limit)
        product)))
  ([limit]
    (fizz-buzz-printer "" 1 limit)))

...
The function format-output is private, it's only visible within its own namespace thanks to declaring it with defn-.
The increased complexity in the let function triggered the "extract function" refactoring. I felt the format-output function should be responsible for deciding if it has to print the number or the words "Fizz", "Buzz" or "FizzBuzz".

The next test was simple, I wanted to make sure the logic worked from zero to four. I did not have to modify the code for it:

...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "from 1-3"
    (is (= "1\n2\nFizz\n" (fizz-buzz-printer 3))))
  (testing "from 1-4"
    (is (= "1\n2\nFizz\n4\n" (fizz-buzz-printer 4)))))
I reached a point with my tests where I had to make my code a bit easier to test. I did not want to write tests that compares more than 3 numbers from the fizz-buzz-printer. I felt 3 is a large enough set to verify the boundaries. I added a new test for this:
...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "from 1-3"
    (is (= "1\n2\nFizz\n" (fizz-buzz-printer 3))))
  (testing "from 1-4"
    (is (= "1\n2\nFizz\n4\n" (fizz-buzz-printer 4))))
  (testing "from 3-4"
    (is (= "Fizz\n4\n" (fizz-buzz-printer 3 4)))))
Since I did not have the function override for 2 arguments, the test failed:

ERROR in (fizz-buzz-printer-test) (AFn.java:437)
from 3-4
expected: (= "Fizz\n4\n" (fizz-buzz-printer 3 4))
  actual: clojure.lang.ArityException: Wrong number of args (2) passed to: core$fizz-buzz-printer

I added the function override which looked almost identical to the function with one argument:

...

(defn fizz-buzz-printer
  ([output current limit]
    (let [product (format-output output current)]
      (if (< current limit)
        (fizz-buzz-printer product (+ 1 current) limit)
        product)))
  ([start limit]
   (fizz-buzz-printer "" start limit)) ; added function with 2 arguments
  ([limit]
    (fizz-buzz-printer "" 1 limit)))

...
All the tests passed. However, this code had duplication: the unary function (method with one argument) initialized the output just like the binary function. I decided to remove this duplication by changing the unary function to call the binary function that calls the ternary one.
...

(defn fizz-buzz-printer
  ([output current limit]
    (let [product (format-output output current)]
      (if (< current limit)
        (fizz-buzz-printer product (+ 1 current) limit)
        product)))
  ([start limit]
   (fizz-buzz-printer "" start limit))
  ([limit]
    (fizz-buzz-printer 1 limit))) ; does not initializes the output

...
I was now ready to zoom into the ranges where the program had to produce the different words instead of numbers. The next test did just that:
...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "from 1-3"
    (is (= "1\n2\nFizz\n" (fizz-buzz-printer 3))))
  (testing "from 1-4"
    (is (= "1\n2\nFizz\n4\n" (fizz-buzz-printer 4))))
  (testing "from 3-4"
    (is (= "Fizz\n4\n" (fizz-buzz-printer 3 4))))
  (testing "from 4-5"
    (is (= "4\nBuzz\n" (fizz-buzz-printer 4 5)))))
I did the simplest thing that could possibly work: I used a nested conditional:
...

(defn- format-output [output current]
  (if (= 3 current)
    (str output "Fizz\n")
    (if (= 5 current) ; introduced this nested conditional
      (str output "Buzz\n")
      (str output current "\n"))))

...
Everything passed again, but this code was ugly. Here is how I used cond to refactor it:
...

(defn- format-output [output current]
  (cond
    (= 3 current) (str output "Fizz\n")
    (= 5 current) (str output "Buzz\n")
    :else (str output current "\n")))

...
My next test covered the range from 4-6, where the word "Fizz" should be printed out for the number 6.
...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "from 1-3"
    (is (= "1\n2\nFizz\n" (fizz-buzz-printer 3))))
  (testing "from 1-4"
    (is (= "1\n2\nFizz\n4\n" (fizz-buzz-printer 4))))
  (testing "from 3-4"
    (is (= "Fizz\n4\n" (fizz-buzz-printer 3 4))))
  (testing "from 4-5"
    (is (= "4\nBuzz\n" (fizz-buzz-printer 4 5))))
  (testing "from 4-6"
    (is (= "4\nBuzz\nFizz\n" (fizz-buzz-printer 4 6)))))
This test failed again as I was only checking for the number three and not for numbers divisible by three without a remainder. I changed the format-output function to use the modulus operator:
...

(defn- format-output [output current]
  (cond
    (= 0 (mod current 3)) (str output "Fizz\n")
    (= 5 current) (str output "Buzz\n")
    :else (str output current "\n")))

...
All the tests now passed again.

I suspected the same error at number 10, therefore my next test was around it (between 9 and 11):

...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "from 1-3"
    (is (= "1\n2\nFizz\n" (fizz-buzz-printer 3))))
  (testing "from 1-4"
    (is (= "1\n2\nFizz\n4\n" (fizz-buzz-printer 4))))
  (testing "from 3-4"
    (is (= "Fizz\n4\n" (fizz-buzz-printer 3 4))))
  (testing "from 4-5"
    (is (= "4\nBuzz\n" (fizz-buzz-printer 4 5))))
  (testing "from 4-6"
    (is (= "4\nBuzz\nFizz\n" (fizz-buzz-printer 4 6))))
  (testing "from 9-11"
    (is (= "Fizz\nBuzz\n11\n" (fizz-buzz-printer 9 11)))))
The test failed, I received 10 instead of "Buzz", I adjusted the second conditional to use the modulus function as well:
...

(defn- format-output [output current]
  (cond
    (= 0 (mod current 3)) (str output "Fizz\n")
    (= 0 (mod current 5)) (str output "Buzz\n")
    :else (str output current "\n")))

...
This was great! Now I had to write code for one more scenario: for the numbers divisible by 5 and 3 I had to print the word "FizzBuzz" instead of "Fizz" or "Buzz". My last test did just that:
...

(deftest fizz-buzz-printer-test
  (testing "with 1"
    (is (= "1\n" (fizz-buzz-printer 1))))
  (testing "with 2"
    (is (= "1\n2\n" (fizz-buzz-printer 2))))
  (testing "from 1-3"
    (is (= "1\n2\nFizz\n" (fizz-buzz-printer 3))))
  (testing "from 1-4"
    (is (= "1\n2\nFizz\n4\n" (fizz-buzz-printer 4))))
  (testing "from 3-4"
    (is (= "Fizz\n4\n" (fizz-buzz-printer 3 4))))
  (testing "from 4-5"
    (is (= "4\nBuzz\n" (fizz-buzz-printer 4 5))))
  (testing "from 4-6"
    (is (= "4\nBuzz\nFizz\n" (fizz-buzz-printer 4 6))))
  (testing "from 9-11"
    (is (= "Fizz\nBuzz\n11\n" (fizz-buzz-printer 9 11))))
  (testing "from 14-16"
    (is (= "14\nFizzBuzz\n16\n" (fizz-buzz-printer 14 16)))))
The error was what I expected: the program only printed out the word "Fizz" instead of "FizzBuzz". Adding the modulus check for 15 made this last test to pass:
...

(defn- format-output [output current]
  (cond
    (= 0 (mod current 15)) (str output "FizzBuzz\n")
    (= 0 (mod current 3)) (str output "Fizz\n")
    (= 0 (mod current 5)) (str output "Buzz\n")
    :else (str output current "\n")))

...
All my tests passed!

Ran 1 tests containing 9 assertions.
0 failures, 0 errors.

I also wanted to see the program in action by running it with leain run. Here is how I changed the main function:

...

(defn -main
  [& args]
  ;; work around dangerous default behaviour in Clojure
  (alter-var-root #'*read-eval* (constantly false))
  (print (fizz-buzz-printer 100)))

And that's it! Please take a look at the final solution in this Gist.
Hope you had as much fun reading this as I had preparing it.

Happy New Year everyone!

Wednesday, November 20, 2013

Reporting in Rails - with Rails Admin and ...

Understanding the data is very important for a business. "How many active customers do I have right now?" "How many did we have last week?" "What are they buying?" "What they are not buying?" "Why?" I could go on and on with these questions that businesses would like to get answered.

I was shadowing one of our customer service folks back in March when someone from sales stopped by and started talking about how great it would be if they could easily export all the candidates for a job and where they applied from. I knew what they were asking for is in the database, I just had to find a way to extract that information.

Our nightly job takes a snapshot of the production database and restores it in our QA environment. This data is good enough for our data mining purposes. The data analyzing tool I was looking for did not have to be very efficient as we can easily restart the QA servers if a long running query would bring it down.

The tool phpPgAdmin works great, but I wanted to find something more lightweight with Rails and nothing turned up at that time. A few years ago I took a look at the Rails Admin project, I figured that might work for us. I generated a skeleton Rails app, migrated all the DataMapper models to ActiveRecord ones and introduced it to the team shortly after.

They loved it!

The product team was able to see the database tables with the data stored in them without an abstracted reporting layer on top of it. Exporting data as a CSV file was also simple, users used Excel to analyze it further.

The story could end here, but it does not. As the users of Rails Admin wanted to use the tool for intricate queries, we bumped into roadblocks: when the association is more complex between tables, Rails Admin won't work out of the box. I remembered the days when I used phpMyAdmin and how that tool provided a SQL editor tool where I was able to do anything with the data through DML.

We needed something similar and I came up with this tool:

The users have to put the SQL script in the query textarea, click on the "Run it" button and the result in CSV is pushed back to the client. Since most of the data analysis happens in Excel anyway, I did not bother building a paginated view for the result. I added basic error checking that reports Postgres errors back to the client. I don't restrict this tool for read-only database access as only our product team is using it. If they figure out how to delete the users table through this SQL tool, we can restore the production snapshot in a few minutes.

This is what the logic looks like for the SQL controller:

require 'csv'

class SqlController < ApplicationController
  before_filter :authenticate_user!

  def editor
    @query = nil
  end

  def run
    @query = params[:query]

    begin
      results = ActiveRecord::Base.connection.execute(@query)
    rescue ActiveRecord::StatementInvalid => ex
      flash.now[:error] = ex.to_s
      render 'editor' and return
    end

    if results.count == 0
      flash.now[:notice] = "No records were found."
      render 'editor' and return
    end

    send_data prepare_csv(results),
              type: 'text/csv; charset=iso-8859-1; header=present',
              disposition: 'attachment;filename=exported_data.csv'
  end

  private

  def prepare_csv(result, options = {})
    CSV.generate(options) do |csv|
      keys = result.first.keys
      csv << keys
      result.each do |row|
        csv << row.values
      end
    end
  end

end

We came up with a couple of SQL queries and stored them in a Google Doc. The process is as simple as coping and pasting the query from that Google Doc, changing the where clause for their needs and running the script. I highlighted the data that can be changed in the queries (like date ranges or record IDs) and showed them how it can be adjusted before the query runs.

The product team just can not be happier! We got to the point where this SQL query editor is going to be the primary data retrieval tool. Rails Admin can do the same with simple and straightforward queries but selecting the tables and fields through check boxes is more time consuming than copying a SQL script into the textarea and clicking the button.

Would we ever give this tool to our customers? Of course not! It will always remain a well secured internal tool. But this will be an easy way to figure out what kind of additional reports most of our users want and we will build those into our product as we go. Until then, they can just contact us, and we'll send them the data they need in a few minutes. Win!

Tuesday, September 10, 2013

The Broken Light Switch

Ever since I read the book "The Design of Everyday Things" I notice awkward objects that have not bothered me before.
What is wrong with this light switch?

I know the wall plate is broken. I'll replace it. It costs about 50 cents and I'll take care of it albeit I don't own this property

OK, I tell you. The problem with this light switch is that the toggle on the left side, closer to the door, should turn on the main light in our master bathroom. It doesn't. That one turns on the light in the shower cabinet and the other on the right side turns on the main light which is totally the opposite the way I'd expect it to work. Whenever I enter the bathroom I have to think about which toggle to use: the one on the left or the other on the right hand side?

I asked my 5 year old son this afternoon to grab something from the bathroom. I turned back from my desk and watched what he was going to do. He walked into the bathroom, switched on the toggle closer to the door which turned on the shower light. He noticed that's not what he wanted, quickly turned it off and switched on the other one on the right hand side. This is the point where I paused for a few seconds. A 5 year old's mind is basically working with instincts, he hasn't had much chance to reprogram his brain. And yet he still used the toggle closer to the door first thinking that is the one for the main light.

I wish I could use his help to do user acceptance or API testing. If something is not obvious for a 5 year old then it should be further simplified. Like how Steve Jobs wanted to ask a 4th grader to write a manual for the Macintosh back in 1983.

Monday, August 26, 2013

Me and My (2013 or v2) Nexus 7

The TL;DR Version

Pros of the Nexus 7 (2013):

  • High resolution - 324 ppi which is great for reading books
  • Shape - it's narrower than the iPad mini, I can easily hold it with one hand
  • Price - less expensive than other tablets on the market
  • No additional software - comes with pure OS installed, no need to uninstall crapware
Cons:
  • Plastic back - I just like Apple's more expensive casing
  • Rough headphones jack - I had trouble using it the first time
  • Unpolished OS - starting from the square buttons to "no command" errors during the OS update

The Nexus 7

I had about 30 minutes drive to my former employer last year. I used this time listening to audio books as I talked about it in my talk at Railsconf 2013. After moving to Chicago my commute changed from driving to riding the train. This was a great opportunity to make progress with my ever growing list of books to read.

I have a subscription to Safari Books Online. It's rather pricey, but you get access to a large number of software engineering titles including unreleased "rough cuts". It also has an offline bookbag feature as well, where I can save 3 books on my device and read it without Internet connection.

I seriously considered buying an iPad mini as that size was more manageable to me than the full size - 10" - iPad. The only reason I did not buy it before was the lack of retina on the iPad mini. And then Google's Nexus 7 (version 2 or 2013) came out one Wednesday. While the iPad mini offered only 162 pixels per inch (ppi), the Nexus 7 offered twice as much: 324 ppi. As I needed this device for reading the higher resolution was the selling point for me. I ordered the device through Amazon and after a few days the USPS guy handed it over to me when I bumped into him outside the house.

Apple cares a lot about the packaging and your first experience with the product, Asus does not. The device has a plastic back cover and its charger is an unattractive black plastic cube. I turned it on, set up the Wifi password and within a few minutes I was on my way. The first app I installed was - of course - Safari Books Online app which worked neatly.

My first initial impression was that the screen was beautiful with the retina display but the device felt cheap and the Android 4.3 isn't as polished as iOS. The lack of rounded corners on the screen keyboard buttons was a surprise to me.
I tried to plug in my headphones into the jack and it did not go in all the way. A pressed a bit, gently, still no luck. I googled this "feature" and bumped into a video on YouTube where a guy had the exact same problem and he recommended using more force. I did that and it worked. I had a few Apple products but I never had this kind of experience with their product. They all just worked.

A few days later Android asked for an OS update. I went with it and when the device rebooted I received a "no command", meaningless error message. Sure the Nexus 7 costs less than the iPad mini but I was a bit worried what else I will find later.

After all the initial quirks, this device is great for reading books and I use it every time I travel to and from the city.

Monday, July 8, 2013

Trust and Discipline are the Core Ingredients of Remote Work

I've just wrapped up seven weeks of remote work from Europe where I worked from several cities from Hungary, Austria and Germany.
Just think about for a second how fortunate we - software engineers - are compared to dentists, assembly line workers or plumbers: all we need is a computer with decent Internet connection.

Unfortunately, airfare is expensive to Europe for an entire family. Whenever we go we try to stay as long as we can to justify the cost with a longer stay. I've had trips with three-four weeks, but this seven week stretch was the longest.

I am pretty sure my team was a bit nervous about my trip before I left, but they trusted me. They knew that I'll get the job done even when I am not on the same continent and they also knew that I'll try to make this as seamless as possible. I did. I started working late in the afternoon and wrapped up work at 11 pm or midnight which overlapped with the work hours of my team in Chicago.

Here is what worked:

  • Phone calls through Gmail - free calls launched from my browser
  • Skype - we used these rarely but worked fairly well
  • Join.me was a good tool to quickly share our screens, however the delay was annoying sometimes
  • Tmux was excellent for sharing the console when I worked with our system admin on server issues
  • Chat

What did not work so well:

  • Conference calls - sometimes the quality of sound was bad. Really bad. There was one time when I had to listen only with muted mic and "talk" through chat
  • Calling cell phones from Gmail had audio quality problems

I am not saying it was easy to log in and focus on work after a great morning and lunch with my family on the Old Continent. But just like running, self discipline helped. And when I noticed my attention had waned, I used pomodoro sessions to keep me focused for 25 minutes at a time.

Tuesday, June 11, 2013

Simple and Elegant Rails Code with Functional Style - RailsConf 2013 Talk

I was very excited to talk about my beliefs on simple and elegant software at Rails Conf 2013 earlier this May. Confreaks recorded all the talks and lightning talks.

Here is mine. Enjoy!

Monday, April 8, 2013

A Tale of Two Rails Rewrites

The TL;DR Version

Watch the video "So You Think You Need a Rewrite" by Chad Fowler and Rich Kilmer. Kill your old app with a Strangler App as you always, constantly release new code.

Preface

I worked for a software company that tried to replace its aging software with a new one. Incremental releases were not possible, it was an attempt to do a "Big Bang" rewrite. The customers received the preview 5 years after development started and due to the lack of smooth upgrade nobody was willing to take the brand new software. The economy turned sour, people were laid off and tens of millions of dollars later - after I left the company - the project was canned.

JRubyConf in 2010

I attended JRubyConf in Columbus, Ohio in 2010 and watched the talk "So You Think You Need a Rewrite" by Chad Fowler and Rich Kilmer. I had read Chad's book "The Passionate Programmer" before I heard his talk there. Little I knew about rewriting Rails applications at that time, but the main message of their talk sticked with me for a long time: "Never, ever do a big bang rewrite!"

The First

Both of the startups I worked for recently had two things in common: the codebase was fairly old (3-5 years) and neither of them had automated tests when I started on the project.

I knew rewriting the entire app was not possible. It would have taken too long, it would have been too expensive, and we wouldn't have been able to release anything to production until the project was done. What we attempted was an "in-place", partial rewrite. Instead of taking the risk of rewriting the entire app we decided to do it in piecemeal: rewriting the app one product at the time.

We kept the database and the Active Record ORM layer intact and rewrote everything above it just for that product. This meant about 15% of the entire app. The new app lived side by side to the old Rails app, the routing engine took the users to the old or to the new product depending on which one they were set up with.

This approach was less risky, but we could not deliver the project until the entire product was rewritten in the app even if it meant just 15% of the codebase. It was still a minified all-or-nothing approach where the code finally made it into production, but deadlines were missed a couple of times.

The Second

We have a 2 years old Rails app with about 17k LOC at my current company - Hireology. It has served its purpose and we are now at a point where we are ready to "refactor" the app. The question is how? We have to face several constraints:

  • We only have two engineers
  • We have to develop new features and enhancements
  • We have to deploy our code several times a week
  • We constantly have to increase the code quality
I read about the "Strangler App" idiom by Martin Fowler years ago. I really loved this concept: you start with a new app, as it grows it has more and more of the business logic and functionality of the old app and it takes over everything in the end, slowly but surely strangling the old app.

Progressive Insurance built their Claims application this way. They wanted to get off of the mainframe and rewrite the app with ASP.NET. The first thing they did was replacing the outdated black and green terminal UI with similarly looking web page talking to the mainframe. This change went smoothly for the employees who used this application since they barely noticed they were using a web page instead of a terminal screen. The same key bindings worked and it looked just like the terminal app.

Then every two or four weeks a tiny new feature went into production. First just a select box with country codes came from the app server, everything else was fed by the mainframe. Then a form was replaced by .NET web forms. And this went on for a year or more and eventually the same web UI was talking entirely to app servers and SQL Server databases cutting off its reliance on the mainframe, slowly strangling it to "death".

We will try something similar. Since all our clients are asking for our APIs, exposing our business logic through them makes a lot of sense. The old app runs on an older version of JRuby and this brand new Rails API app will be using Rails 4 with Ruby 2.0. It will be properly tested, when a new version of Rails comes out this app should be running on it shortly after.

We start with a tiny feature: there is one page in the app where we have to load all the different job profiles a client has. The old app receives the request, its controller wraps the params and sends it over to this new API app. The API app is sitting on top of the exact same database as the old Rails app. It collects the open, closed and pending jobs, serializes the collections to JSON. Passes it back to the old app which loads up the models from the received JSON data. It renders the view with data it just received from the API app without talking to the database again.

This feature is small, the users will never notice we started on this project and we will be able to release it in a week or two. If we keep doing this, moving the business logic into our new Rails API app, little by little we will shrink the legacy Rails app to just a dumb presentation engine. I am pretty sure that will be further reduced by building a rich HTML app - using a client side MVC or MVVM framework like Ember.js or Spine.

I was asked how long this will take. Maybe a year. Maybe less, maybe more. It's hard to say without having any kind of baseline data.
We will constantly move logic to our new API app where the code will have supporting automated tests. This API application - if it's structured properly - can feed not only our own app, but other third-party applications as well if they want to integrate our business logic into their own apps.

In the end, everybody wins.

Tuesday, February 12, 2013

Crossroads

The TL;DR Version

I like working for startups even if it's a gamble and I can lose my job fast. I enjoy the freedom it provides: I work from anywhere I want, I use the tools I like and I am working with passionate people.
I am moving to Chicago in the summer.

The Story

I wrote about frequent job changes and I was hoping my current job would improve the track record. It did not.

There were a couple of warning signs triggering my job search right after my CodeMash talk. I knew the market for Ruby jobs is very limited in Northeast Ohio and I remembered what it was like in Chicago. My wife was OK with moving to a bigger city so I started my search in the greater Chicago area early January.

First of all I wrote a resume. I don't believe in resumes, but I was targeting a geographic area where I have very few contacts. I pinged friends, tried to find out what's available through them. I also felt that time is running out and in an inexplicable moment, I uploaded my resume to Workbridge Associates' web site. Five minutes later my phone rang. We chatted for a few minutes and later that day I started receiving company profiles looking for Rails/Ruby developers. It was great, I found some amazing companies that I would have never found on my own.

I was on the phone quite a bit in the next two weeks. I talked to the headhunter firm and with the companies looking for developers. I must say having a blog and a Github repo helped me a lot. The hiring managers, developers were able to read up on how I think, what problems I solve and what kind of person I am. I was also happy to see that no company wanted to "grill me" on algorithms and different binary trees, they gave me homework assignments instead, which I think is a much better way of figuring out what the developer knows, how he/she thinks.

By the beginning of February my job search narrowed down to four different companies:

I took the Megabus to Chicago on the 3rd of February and by midnight I was in my hotel room anxiously waiting for the next morning to talk with these great companies.

Table XI gave me a homework assignment which I completed ahead of their deadline. I did not build a CLI interface to feed files into the program, I proved out it's behavior with Cucumber features. I also missed a scenario where you could order multiple times from the same dish.
Table XI invests a lot in their culture. I met with them three times while I was in Chicago. We went through my homework assignment, they had me code on the board and I also had to make changes to a Rails app I have not seen before. I believe they are very selective of who they hire, but again, their team seemed great. Oh, and they have their own chef who prepares lunch four times a week.

I met with the team of Hireology next. A printed paper was taped to their door saying "Hireology welcomes Attila Domokos!". I was flattered.

The company's CEO took me out for lunch. We chatted about our families, what I thought of their product while we ate our delicious soup and sandwich at a restaurant that did not look much from the outside.
After lunch the CEO, the head of product development and I sat down for an afternoon session where they talked about their business, about their growth potential and why they believe in their product. I showed them how I write software, what it means to run tests with and without Rails. But I think they were most intrigued by the executed Gherkin scenarios through Cucumber.
I got together with the CTO of their sister company Urban Bound to find out how I code with others.
At the end of the day we went down to the first floor and had a couple of beers at the pub. By the way, in Chicago almost every office building has a pub or a wine bar on the ground floor. You don't even have to leave the building!

The next day I visited Aggrego which is a small startup under a big company. They are working on the next generation of online news media for both news creation and consumption. Their team is extended with a couple of Thoughtworks engineers. I paired with Brian Kung in the morning, we added a few specs to cover his helper that loaded JS or static images for banner ads depending on the type of browser that made the request. I spent most of my time with Dave Willkomm in the afternoon, we test drove an EC2 server facade object they used in their deployment scripts.

The last day I took a quick cab ride to Giveforward, which is a startup in the Wicker Park neighborhood of Chicago. Their fundraising software helps with medical and other related expenses. Their application is written in PHP and they are ready to port it to Ruby on Rails. I met with two engineers and a designer/UX person. After their product demo and going through some their code I did a quick demo of how I build software. We did a quick brainstorming session on URL structures and on database entity relationships. I had a good conversation with them and their app would be interesting to work on.

After all this, which company did I chose?

Well, not so fast! I was about to get on the bus and travel back to Cleveland when I started receiving news from my coworkers about layoffs happening at my current employer. I knew it was coming, I just did not know it would happen so fast.

Before I answer the question I have to tell you: I liked all the companies I visited. None of them was perfect, there were things I liked in one and were missing in others.

However, I felt a special connection with the team at Hireology. I was amazed by how much they cared about their product, the amount of work they put into it and how firmly they believed in their success. I was also applying for the lead engineer position allowing me to define their development process, work out the details of the relationship between product development and engineering. Building an engineering team is something I found very attractive as well.

By the third and last day of my trip, after exchanging a couple of emails with Adam Robinson, the CEO of Hireology I knew I wanted to work with them. Two hours after I heard the layoffs that I was - unfortunately - part of and when I was riding the Megabus back to Cleveland, I received a fantastic offer that I could not refuse. I'll join Hireology today as their CTO, where I will not only oversee their software development, but I'll get to work on other areas that affect the success of the company.

I'll start my work remote, but I'll make frequent, one week long trips to Chicago so I can work with others in person at Hireology. I'll be back in Europe for seven weeks in May and June and we are moving to the North suburbs of Chicago in July.
I love Chicago, I fell in love with that city the first time I visited in 1998. Seeing the sparkling startup life there just proves it to me that my wife and I made a great decision.

So my dear friends in Northeast Ohio, let's spend time together while I am around. I'll try to attend user group meetings, coding events, work from coffee shops or from your office. I may be leaving soon, but I am sure I'll be back. Maybe for CodeMash next year?!

Thursday, February 7, 2013

User Devised - Disconnecting Your User Entity From Devise

We have several Rails apps with duplicated user information. If you're John Smith in one of the apps, we will create a new user with the same information in the others. That's crazy, single sign-on should be the obvious solution.

In order to get there we had to disconnect the authentication mechanism from our User entity. Once we have the same separation in our apps, pulling the authentication out into a service or API should be very easy.
Also, I like to keep the User model application specific. I would much rather have fields that are relevant to our app needs in our User model and not mix devise's fields into that entity.
Here is what I had to do to disconnect devise from our User model.

First of all, I had to generate a member table with the following migration:

class AddMembers < ActiveRecord::Migration
  def change
    create_table(:members) do |t|
      ## Database authenticatable
      t.integer :user_id
      t.string :email,              :null => false, :default => ""
      t.string :encrypted_password, :null => false, :default => ""
      t.string :first_name,         :default => ""
      t.string :last_name,          :default => ""
      t.string :middle_name,        :default => ""

      ## Recoverable
      t.string   :reset_password_token
      t.datetime :reset_password_sent_at

      ## Rememberable
      t.datetime :remember_created_at

      ## Trackable
      t.integer  :sign_in_count, :default => 0
      t.datetime :current_sign_in_at
      t.datetime :last_sign_in_at
      t.string   :current_sign_in_ip
      t.string   :last_sign_in_ip

      ## Confirmable
      t.string   :confirmation_token
      t.datetime :confirmed_at
      t.datetime :confirmation_sent_at
      # Only if using reconfirmable
      t.string   :unconfirmed_email 

      ## Lockable
      # Only if lock strategy is :failed_attempts
      t.integer  :failed_attempts, :default => 0 
      t.string   :unlock_token
      # Only if unlock strategy is :email or :both
      t.datetime :locked_at

      ## Token authenticatable
      t.string :authentication_token

      # Uncomment below if timestamps were not included
      # in your original model.
      t.timestamps
    end

    add_index :members, :email,                :unique => true
    add_index :members, :reset_password_token, :unique => true
    add_index :members, :confirmation_token,   :unique => true
    add_index :members, :unlock_token,         :unique => true
    add_index :members, :authentication_token, :unique => true
  end

  def self.down
    # By default, we don't want to make any assumption
    # about how to roll back a migration when your
    # model already existed. Please edit below which fields
    # you would like to remove in this migration.
    raise ActiveRecord::IrreversibleMigration
  end
end

Instead of putting all the Member related code into the User model, I created a concern that can be mixed into the User model. This module can be reused in all of the Rails apps to connect the Member entity to the User.

require 'active_support/concern'
require 'active_support/core_ext/module'

module ModelExtensions
  module UserDevised
    extend ActiveSupport::Concern

    included do
      has_one :member
      validates_associated :member

      after_save :save_member, :if => lambda {|u| u.member }
      delegate :last_sign_in_at, :password, :password=,
               :password_confirmation, :password_confirmation=,
               :to => :member

      before_validation do
        self.member.first_name = self.first_name
        self.member.middle_name = self.middle_name
        self.member.last_name = self.last_name
        self.member.email = self.email_address
      end
    end

    def initialize(*params)
      super(*params)
      self.build_member(*params) if member.nil?
    end

    def save_member
      self.member.save!
    end
  end
end

I'd like to mention a couple of things about the module you see above. I am using one-to-one relationship between User and Member. Whenever I save a User, I save the Member as well by using the after_save callback. Admins can overwrite the users' password, I delegate those fields from User to Member, so the Member entity remains hidden behind the User. When a new User is initialized, I create a Member entity as well as you see it in the initialize method.

Devise strategies and overrides are defined in the Member model.

class Member < ActiveRecord::Base
  belongs_to :user

  # Include default devise modules. Others available are:
  # :token_authenticatable, :confirmable,
  # :lockable, :timeoutable and :omniauthable

  devise :database_authenticatable, :lockable, :timeoutable,
         :recoverable, :trackable, :validatable, :omniauthable
         # non-registerable :registerable, :rememberable

  validates :first_name, :last_name, :presence => true

  def active_for_authentication?
    # Comment out the below debug statement to view
    # the properties of the returned self model values.

    super && user.active?
  end

  def confirmed_account?
    (self.last_sign_in_at.nil? == false &&
        self.reset_password_token.nil?)
  end

end

I remember how hard it was to find information on the separation of the application specific User and the devise Member entity when I was working on this. I hope someone will find this code helpful, it would have helped me a lot.

Saturday, January 5, 2013

Simple & Beautiful Rails Code with Functional Style

In early October I submitted this abstract to CodeMash:

"Do you have to look at Rails models with 1400 lines of code? Or 80 line method loaded with iterators, conditionals and instance variables? Not only you, even the code author does not understand what's going on in there.

I'll show you how you can craft simple and beautiful Rails application by adopting functional programming inspired ideas. Say goodbye to the mess you have by constructing tiny classes and functions that you can use to build up a complex system."

My talk got accepted and I gave it in early January, 2013. In case you could not attend it, or if you want to review the slides again, here is an abridged version of it.