Friday, April 15, 2011

Running Rails Rspec Tests - Without Rails

I opened up my twitter client this afternoon and I saw "54 Messages, 28 Mentions". I tell you honestly, the first thought I had was: my twitter account had been hacked. Then I started to comb through the messages and I found out what happened. It all started with a tweet from Joe Fiorini.


We both worked together on a large Rails application. The application was a little light on tests, so I asked the other developers why they are not writing more specs? The answer was all too familiar: "it just takes forever to run them". Yup, Rails had to load up, schema needed to be verified, the entire universe had to be included and 30 seconds later our specs were executed.

We started creating POROs - Plain Old Ruby Objects - as pure services and put their RSpec tests into APP_ROOT/spec/units directory. Our goal was to keep the execution time under or around 2 seconds. Sure, it's easy when you don't have to load Rails controllers or active record models. But what happens when you have to?
This post will explain that.

The controller I used for this example is simple:
class TracksController < ApplicationController
  def index
    signed_in_user
  end

  def new
    @track = Track.new
  end

  def create
    feed = params[:track]["feed"]
    @track = TrackParserService.parse(feed)

    unless @track.valid?
      render :action => 'new'
      return
    end

    @track.save_with_user!(signed_in_user)

    render :action => 'index'
  end

  def destroy
    Track.find(params[:id]).destroy

    @user = User.first
    render :action => 'index'
  end

  private

  def signed_in_user
    # No authentication yet
    @user ||= User.first
  end
end
The first controller action I wanted to test was "index".

I created the directory structure APP_ROOT/spec/units/controllers and saved my file in this directory under the name tracks_controller_spec.rb.

I started out with this code:
APP_ROOT = File.expand_path(File.join(File.dirname(__FILE__), "..", "..", ".."))
$: << File.join(APP_ROOT, "app/controllers")

require 'tracks_controller'

describe TracksController do
  
end
You could move the first two lines into a spec_helper, I wanted to keep it here for clarity.

I received the following error:
`const_missing': uninitialized constant Object::ApplicationController (NameError)

No worries: TracksController inherits from ApplicationController, it's part of my app, I just had to require it.
require 'application_controller'
And the error:
`const_missing': uninitialized constant Object::ActionController (NameError)

This was the point where I had to require Rails.

Instead of doing that, I just defined the class myself so the controller was aware of it. I also needed to declare the class method "protect_from_forgery", but I left the implementation blank. Please note that the class declaration is above the require statements.
Here is the entire spec after my changes:
APP_ROOT = File.expand_path(File.join(File.dirname(__FILE__), "..", "..", ".."))
$: << File.join(APP_ROOT, "app/controllers")

# A test double for ActionController::Base
module ActionController
  class Base
    def self.protect_from_forgery; end
  end
end

require 'application_controller'
require 'tracks_controller'

describe TracksController do
  
end
Running the spec:

Finished in 0.00003 seconds
0 examples, 0 failures

The first test just ensures that the User active record model will load the first user if the @user instance is nil.
describe TracksController do
  let(:controller) { TracksController.new }

  specify "index action returns the signed_in_user" do
    # setup
    user = stub
    User.stub(:first).and_return user

    # execute action under test
    returned_user = controller.index

    # verify
    returned_user.should == user
    controller.instance_variable_get(:@user).should == user
  end
end
The test is straightforward. User model is returning a stub - I don't really care what that returned object is, I just check if they're the same object. In the verification part I made sure that the instance variable was set properly. Great that your can check an un-exposed field on a object with a little bit of metaprogramming?

I executed the spec and received the following error:

Failures:

  1) TracksController index action returns the signed_in_user
    Failure/Error: User.stub(:first).and_return user     NameError:       uninitialized constant RSpec::Core::ExampleGroup::Nested_1::User

Well, I need to require the User model to fix this error. Or do I? I am not using any functionality of the User class - whatever I am using is stubbed out. I just defined the class without any implementation.

This line was added to the spec right above the describe block.
class User; end
I execute the test and it's all green.

TracksController
  index action returns the signed in user

Finished in 0.00079 seconds 1 example, 0 failures 1.26s user 0.28s system 99% cpu 1.546 total

1.5 seconds is not all that bad to run a controller action test.

Let me describe how I tested the "create" action.
Take a look at the controller code above and review what it does. The @track instance is constructed by the TrackParserService class' parse method. Then active record validates it and if the model is invalid the controller's "new" action is rendered.

Here is the spec for that:
context "when the model is not valid" do
  it "renders action => 'new'" do
    # define a method for params - TracksController is unaware of it
    controller.class.send(:define_method, :params) do
      {:track => "feed"}
    end

    track = stub(:valid? => false)
    TrackParserService.stub(:parse).and_return(track)

    render_hash = {}
    # hang on to the input hash the render method is invoked with
    # I'll use it to very that the render argument is correct
    controller.class.send(:define_method, :render) do |hash_argument|
      render_hash = hash_argument
    end

    controller.create

    # verify the render was called with the right hash
    render_hash.should == { :action => 'new' }
  end
end
I used Ruby's metaprogramming again to set up the params hash. It really doesn't matter what's in it, since I stub out the TrackParserService. The method "render" comes from Rails, I had to define that as well. Please note that I record what the render method was invoked with, this way I can verify that the input hash was correct.
I also had to define - with no implementation - the Track and TrackParserService classes.

When I executed the specs, all of them passed:

TracksController
  index action returns the signed in user
  new action returns an instance of Track
  when the model is not valid
    renders action => 'new'

Finished in 0.00203 seconds
3 examples, 0 failures
bundle exec rspec spec/units/controllers/tracks_controller_spec.rb -fd 1.32s user 0.29s system 99% cpu 1.614 total

You can review the entire example in this gist.

This code is rough. I just used it to show you how we try to keep our test execution fast. I acknowledge that I am doing some very dangerous stubbing here. However, I have the higher level cucumber tests to protect me against unexpected errors.

I can't tell you what it means to run all of my 150+ specs within 2 seconds. I think it's a little bit of an extra work, but it's well worth the effort!

Sunday, April 10, 2011

Rapid Feedback

I am not a WPF expert. In fact, I don't know it well enough. But let me tell you what it was like working on a WPF project last year.


It was 10 o'clock in the morning and I sat in my cubicle. I had to pull the latest changes from the source control server. I had to run a couple of batch files to compile all the code that took about 3 minutes. In the mean time I fired up Visual Studio 2010 and ran my latest unit tests to make sure everything was in good shape.

I started up the WCF services, 86 of them, and a little later I was ready to run the UI app. It took another 30 to 40 seconds to load and get to the login screen. I logged in and selected from the menu where I wanted to get to. That page was a list of items, I had to select one of them just to get to the detail page. Finally, I was there!

Let me sum it up:
* 180 seconds to compile the app
*   50 seconds to fire up all WPF services
*   20 seconds to start the WPF UI App
*   60 seconds to log in and go to the page I had to modify
TOTAL: 310 seconds

My task was adding a new TextBox to this page. Simple. I opened up the XAML file which was an ugly xml file with weird namespaces and special attributes all over. I grabbed a TextBox XAML code from somewhere, pasted it in, made sure all the namespaces were fine and I was ready to run it.

I had to shut down the UI app, compile the UI project, start it up again, log in, select the menu option to get to the list page and choose one item to see the detail.

Here is how long this took:
* 30 seconds to compile the UI app
* 20 seconds to start the WPF UI App
* 60 seconds to log in and go to the page I had to modify
TOTAL: 110 seconds

And it turned out that I did not set up the Grid for this TextBox properly, so I had to do some minor tweaks to the XAML page. I did that, killed the UI app, complied the code, ran the WPF UI app, logged in, went to the page and 110 - or one hundred and ten - seconds later I verified that all look good.

But this was the fast part. Once I had all the UI set up properly, I had to get under the hood and modify the domain object. The change was "simple": just add a text field to the database, modify the domain object, set up the NHIbernate mapping, change the Data Transfer Object, add this field to it and set up its mapping if I had to.
Now to make sure all this worked I had to shut down the UI app, the WPF services. Compile the code, regenerate the NHibernate mappings, fire up the WPF services, run the UI, log in, select the page and pick an item to get to its detail. Simple, right?

Here is the break down:
* 30 seconds to compile the Data Access Code
* 30 seconds to regenerate the NHibernate mapping xml
* 50 seconds to fire up all WCF services
* 20 seconds to start the WPF UI App
* 60 seconds to log in and make sure that all looks good
TOTAL: 190 seconds

Wait! 3 minutes just to see if everything is working properly?

Give me a break.

What company with a tight budget and ever approaching deadlines could afford spending 3 minutes just to see if a simple change is functioning properly or not? Who would dare to touch the existing code to clean it up a bit?

One of the great things I like about working with Ruby and Rails is the rapid feedback. No, I am not talking about how long it takes to execute my - Rails disconnected RSpec - tests. (I'll try to write about that in an upcoming post.) I just change the code, hit the browser's refresh button and about 5 seconds later I have the page loaded, the session preserved and I have the answer.

I am talking about 5 seconds and not a couple of minutes.