Where Should I Host My Family Videos?

Wow, it's been a while since I last posted anything on my blog! A lot has happened: I spent an entire month with my family in Hungary. I even took a short trip to Austria as well. I enjoyed many Black Forest cakes with great cappuccinos.
Returning to the US has its ups and downs: on one hand we are back in our comfortable home, on the other we miss all our relatives living on the Old Continent.

We've been posting short - 30 seconds long - videos on our family web site that I host on my home server. We have a LAMP setup with Gallery2 on it. It has served us well, however, our cable provider noticed that there is significant data download through our 8080 port. I received their first warning letter right after we got back. It was pretty obvious that I had to look for some kind of a cloud hosting solution.

Some people recommended using Amazon S3 but I wanted to stick with Google. Posting photos to a Picasa web album is very simple but it has one shortcoming: you can't upload a video in flv format. You can upload them to YouTube and then somehow link that into the album but I just was not ready to go with that hybrid solution.

We chose Blogger.com to host our photos and videos. We can easily upload photos there plus adding a little more text is always welcomed by our family. But what about the videos? Well, Google Sites is there for the rescue.

I use the unbelievably awesome ffmpeg to convert my ~60 MB 30 seconds clip into a 1 MB flash movie.

For you, dear reader, here is the script I use:

ffmpeg -i [your_avi_movie.AVI] -ar 11025 -ab 32 -f flv -s 320x240 [your_output_movie.flv]

This might seem cryptic when you look at it first, but google ffmpeg and you should find all the info you need. For example the "-f" switch is the format, "-s" is the size.

Once the movie is converted, I just upload it to my Google Site. It was not straightforward where and how to upload it, so here are the steps I took:

1) Go to your Google site (you should have one with a Google account)
2) Go to More Actions => Manage Site
3) Under "Site content" select "Attachments"
4) You should see the "Upload" button to upload your movies

Having the movie there is all fine, but I needed a Flash Player to properly play them in the web page. After a quick search I settled with OS Flv. I uploaded the player swf file to my site and I had everything ready to post my videos.

Here is the short HTML code I used to link in the flv movies through the OS Flv player into our blog page.

<object height="240" id="flvPlayer" width="320">
  <param name="movie" value="[link_to_os_flv_player.swf]">
  <param name="FlashVars" value="&movie=[link_to_your_movie_file.flv]">
  <embed src="[link_to_os_flv_player.swf]" flashvars="&movie=[link_to_your_movie_file.flv]" width="320" height="240" type="application/x-shockwave-flash"></embed>
</object>

And here is a quick movie clip I made in July, 76 MB compressed to around 1 MB. Enjoy!

Dynamic Properties In Ruby

I've been working on this Rails 3 app where users can define what fields they want to have for their track records. The problem I was facing: how can I dynamically add properties to an object?

I have this class:

class Track
  attr_accessor :value

  def initialize(value)
    @value = value
  end
end

The value field is populated by the application, it'll always be valid JSON data.

After the track object is initialized, I'd like to be able to call "distance" and "running" properties on my track object. The following RSpec code describes what I need:

describe Track do
  it "should add distance and running as read-only properties" do
    track = Track.new('{"distance":2,"what":"running in the park"}')    
    track.distance.should == 2
    track.what.should == 'running in the park'
  end
end

The question is: how can I do that?

First I thought about hooking into the method_missing method in the Track object. It worked, but I was unhappy with the solution. It seemed clumsy and it's not going to provide the best performance either. I exactly know what my methods are going to be called since it'll be set from the value field.

After googling the topic I found the solution: define_method.

I had to parse the JSON data which was easy with the json gem.

require 'rubygems'
require 'json'

data = '{"distance":2,"what":"running"}'
parsed_data = JSON.parse(data)
puts parsed_data["distance"] # => 2

Once I knew how I'll parse the JSON string, adding the define_method calls to the initialize method was easy.
You can find the final solution here:

require 'rubygems'
require 'json'
require 'spec'

class Track
  attr_accessor :value

  def initialize(value)
    @value = value

    parsed_values = JSON.parse(value)
    fields = parsed_values.keys.inject([]) do |result, element|
      result << element.to_sym
    end

    fields.each do |field|
      self.class.send(:define_method, field) do
        parsed_values[field.to_s]    
      end
    end
  end
end

describe Track do
  it "should add distance and running as read-only properties" do
    track = Track.new('{"distance":2,"what":"running in the park"}')
    track.distance.should == 2
    track.what.should == 'running in the park'
  end
end

There are a couple of things worth mentioning here.
See how elegantly the symbol array is populated from the hash keys on line 12 with the array's "inject" method.
All the magic with dynamic methods happens on line 15. Please note how define_method message is sent to the object's class reference. All it does is returns the value from the parsed JSON data.

I found this article very helpful. I went through the examples and now I know what's going on behind the scene when I use "has_many :addresses" is my Rails models.

Hackibou

It all started a couple of weeks ago when I suggested to @pragmaticpat that we should get together for a little BDD with RSpec in Ruby. I have always wanted to work on the Roman Numerals ruby quiz that I found in the book "Best Ruby Quiz", I thought this could be a pretty good chance of trying it. Set up the place (Caribou - hence the name Hackibou), date and time and we were ready to do it. A couple of days before the date I tweeted about it. Word got out and two great guys (@joelhelbling and @johnwesp) decided to join us.

We had three hours to work on this quiz. I got a latte, asked a lady to give up the table she used, set up a blank project with @pragmaticpat and we started coding. We did two 45 minutes sessions in the afternoon. We discussed what we did and switched pairs after the first session. We intentionally kept the code we wrote and tried to build on top of that in the second session. Unfortunately we did not finish the quiz. Joel took it the farthest, he worked on it at the end of the day. You can see his code on github.

All in all, we all had a great time.

At the end of our session Joel did a quick demo on jspec with JavaScript. It seems really cool, I have to give it try soon.

I hope Hackibou won't be a one time event. I am planning on organizing one in 2-3 weeks. I hope to see you there!

Put on Your Hat and Dance to Sinatra

It's been almost two weeks since we had our kick-off/planning meeting. We ended up with screen shots - very vague, drawn on a white board - and some stories in Pivotal Tracker. I immediately started to look for css templates. I am not a designer, but I do like simple and clean design. Ended up picking one from http://www.freecsstemplates.org/.

My goal was not to jump into code and create models, database tables and such. I'd like to have a rough HTML prototype that can help us figuring out what we really need. I emphasize HTML, because I want to be able to change it easily. I don't want to have a migration script created when we need to add one more field to a form.

I started out with an index and the user signup page. When I added the confirmation page I quickly realized that I'll have serious duplication unless I do something about it. My header and footer html content was repeated in three HTML files. I know this is only a prototype helping us discovering how our web app should work. But still. Even my prototype code should be DRY-ed up. I badly needed a template solution.

I could have created a skeleton Rails app, but that just seemed too heavy for this purpose.

I remember Matt Otto's (@matthewotto) tweet from late March:

I saw the tweet, I did not really pay attention to it then. But after I created my thrid page and noticed the duplication, I knew I had to find something. I can't recall why I looked at Sinatra a couple of days later, but I was blown away. It is exactly what I needed! I put a simple app together with it and I was convinced: I found my template engine for the prototype!

Here is the only Ruby file I had in my sample app:

# myapp.rb
require 'rubygems'
require 'sinatra'

get '/' do
  'Hello, World!'
end

get '/hello/:name' do
  # matches "GET /hello/foo" and "GET /hello/bar"
  # params[:name] is 'foo' or 'bar'
  "Hello #{params[:name]}!"
end

get '/test' do
  @data = 'Hello from test!'
  erb :index
end

The first block is executed when you make a request - assuming you run the app locally - to http://localhost:4567. There is no template rendering, you'll only see the string "Hello, World!" in the browser.

Ok, this works, but how could I use this for my templates? 

Digging into the Sinatra documentation I found the solution. I had to create the following directory structure:

-|

-|-views

        |- index.erb

        |- layout.erb

-|-myapp.rb

layout.erb is my template:

<html>
<head>
<title>Something Test</title>
</head>
<body>
<div id="content">
<%= yield %>
</div>
<div id="footer">
<hr/>
</div>
</body>
</html>

And index.erb provides the page content:

<div>
This is the content: <%= @data %>
</div>

Please note that I set the @data variable in the "get '/test'" block.

And that's it. I tell Sinatra to grab the index.erb file and since I have layout.erb it knows it has to render that as a template. There is a whole lot more to Sinatra than this of course. I found the "Sinatra Book" really helpful. I'd recommend taking a look at it if you want to give Sinatra a try.

I used HTML in ERB first, but then Matt Otto tweeted that he went with (HAML) so I started playing with that.

HAML looked cryptic, but once you get the hang of it, it's really simple.

In the end, I started using HAML and SASS for my prototypes, I'll blog about them in my follow-up blog post.

Planning Meeting

We started to work on our new application over the weekend. No specifics at this time, let's just call it "Project X". It'll be a web app, most likely developed in Rails using mySQL back-end. But it doesn't really matter just yet.

We got together Saturday around 10 AM. After establishing the agenda we started to work on the details.
Here is what our agenda was:

1) Check competitors
2) Look at HTML Templates
3) Define system user types
4) Work out the workflows for our different user types
5) Create wire frames
6) Create mock-ups
7) Enter story cards in Pivotal tracker

The real work started with item 3 on our list. It is like the 10,000 feet view of the app. No specifics, discussing what happens when. Our technique was similar to a UML activity diagram, focusing on what triggers the application to move from one state to the other.

Once we had a general idea of the different workflows that different user types had, we started to "zoom into" the areas by creating wire frames. We wanted to focus on what pages we needed and how you could get from one page to the other.

We estimated around 12 – 15 pages we need to get the application off the ground. Our mock up task focused on what elements or fields we'd have on those pages. What describes a user? Name, email, company name, etc. Do we need categories to help searches? Sure! So we added that making it easy for the different user types to connect.

One of us was working on the white board, while the other entered stories into the excellent Pivotal Tracker.
We finished around 3:30 PM. I was tired, but we did accomplish what we wanted. Create stories and start to move forward with the project.

What's next? We'll pick a free web template and just build up HTML prototypes. Our idea is vague right now and we want to make changes freely and easily. Having a DB back-end would slow us down when we want to switch things around. We'll learn about the application by doing this and asking ourselves: As User Type "A" when I go to my dashboard, should I see historical items to do rating?
Development will start when we are OK with our HTML pages.

Do you follow the same process with your customers?

Testing Excel with Cucumber

Testing an Excel application with Ruby's unit testing tool is doable, however, the tests are not easily readable. Why not using Given/When/Then structured Cucumber tests instead? Cucumber is an active open source project enjoying great popularity in the Rails universe. I won’t be describing the tool in this blog post, if you’re new to Cucumber, please visit the official website or read the excellent Cucumber book to learn more about it.

Here is the task I need to write tests for: we have a basic Excel spreadsheet with a handful of cells in it.



The row cells with yellow background are the column headers. Column cells starting from A2 are captions. "Category1" has three child elements and "Category2" has only two. The category rows are calculated by the sum of their children. Value Total is the sum of Value1 and Value2 for each row. The totals in the bottom are calculated by adding up Category1 and Category2 values.

First I'd like to verify that the column headers and captions are in place. I started writing my feature this way:

Feature: Modify values in the Excel sheet
In order to show my power
  As a user
  I want to interact with Excel

Scenario: Display column headers and captions
  Given I have 2 categories
    And I have 3 child elements under the first category
  When I open the Excel workbook
  Then I should see "Category" in the "A1" cell
    And I should see "Value Total" in the "B1" cell
    And I should see "Value1" in the "C1" cell
    And I should see "Value2" in the "D1" cell
    And I should see "Category1" in the "A2" cell
    And I should see "Child1" in the "A3" cell
    And I should see "Child2" in the "A4" cell

I created the following folder structure:
/features
  |
  |-- step_definitions
  |      |-- excel_handler.rb
  |      |-- excel_steps.rb
  |-- support
  |      |-- env.rb
  | - excel.feature

The feature from above was saved in the excel.feature file.

I am not particularly concerned with the "Given" part of the scenario. The data can be loaded into Excel either from a CSV, XML or a remote data store. I'll ignore this part to keep my examples clear and concise.

My previous blog post described how I can interact with Excel through the great WIN32OLE object. I created the ExcelHandler class which does that:

class ExcelHandler
  include Singleton

  # set this to your Excel file path
  @@excel_file_path = 'C:\Temp\TestWorkbook.xlsx'
  def open_excel
    begin
      @excel = WIN32OLE.connect('excel.application')
      @wb = @excel.ActiveWorkbook
    rescue
      @excel = WIN32OLE::new('excel.application')
      @excel.visible =true
      @wb = @excel.Workbooks.Open(@@excel_file_path )
    end
  end
end

I might not have the most elegant code in the open_excel method, but this allows me to attach to a running instance of the Excel workbook which is a big thing for me. In case the workbook has not been launched yet, I take care of it here. Launching and closing Excel takes time and resources, I'd like to reuse running instances of Excel whenever I can.

I take advantage of the singleton pattern in this class. Starting up Excel is an expensive operation, I want to make sure that one and only instance is handling the workbook. My tests are single threaded, I think I should be safe there.

The env.rb file has the require statements:

require 'spec/expectations'
require 'win32ole'
require 'singleton'

All the magic happens in the excel_steps.rb file:

When /^I open the Excel workbook$/ do
  ExcelHandler.instance.open_excel
  @worksheet = ExcelHandler.instance.worksheet
  @worksheet.extend CellValueGetter
end

Then /^I should see "([^\"]*)" in the "([^\"]*)" cell$/ do |value, cell|
  @worksheet.get_cell_value(cell).strip.should == value
end

module CellValueGetter
  def get_cell_value(cell)
    get_cell(cell).value
  end

  def set_cell_value(cell, value)
    get_cell(cell).value = value
  end

  def get_cell(cell)
    cell_values = cell.split('')
    cell_values.length.should == 2
    cells(cell_values[1].to_i, cell_values[0])
  end
end

Look at how I add methods to the @worksheet object. You gotta love Ruby for that!
The methods in this module are responsible for getting and setting a value based on the provided cell.
I left out the Given parts that I ignore anyway. You can look at the entire source code after you pulled it from my github account.
When I execute "cucumber features" in the command prompt I get this:

1 scenario (1 passed)
10 steps (10 passed)
0m0.027s

Hey, the first scenario is passing!!

All right, let's verify in the second scenario that the data was loaded correctly:

Scenario: Display loaded values
  Given I have 2 categories
    And I have 3 child elements under the first category
  When I open the Excel workbook
  Then I should see 111 in the "C3" cell
    And I should see 353 in the "C2" cell
    And I should see 458 in the "B3" cell
    And I should see 1523 in the "B2" cell

I had to add one step definition to the excel_step.rb file:

Then /^I should see (\d+) in the "([^\"]*)" cell$/ do |value, cell|
  @worksheet.get_cell_value(cell).should == value.to_i
end

When I execute "cucumber features" in the command prompt I see this:

1 scenario (1 passed)
7 steps (7 passed)
0m0.024s


I know I am not using the "Given" part of the scenarios, however, I do repeat code there. I used the background feature of Cucumber and DRY-ed up my scenarios a little bit.

Background:
Given I have 2 categories
And I have 3 child elements under the first category

I use scenario outline in my third scenario. I set the "Value1" cell for the "Child1" row to 211. Take a look at the result in Excel:



I also try to set the same cell to 51, I got these numbers then:



I am verifying numbers in red in the last scenario:

Scenario Outline: Change values
  Given the default values were loaded
  When I open the Excel workbook
    And I put <child1_value> in the "C3" cell
  Then I should see <category_value1> in the "C2" cell
    And I should see <child1_sum> in the "B3" cell
    And I should see <category_total> in the "B2" cell
    And I should see <value1_total> in the "C9" cell

Examples:
| child1_value | category_value1 | child1_sum | category_total | value1_total |
| 211 | 453 | 558 | 1623 | 1281 |
| 51 | 293 | 398 | 1463 | 1121 |

I added the following step to the excel_steps.rb file:

When /^I put (\d+) in the "([^\"]*)" cell$/ do |value, cell|
  @worksheet.set_cell_value(cell, value)
end

Let's see what we get now:

2 scenarios (2 passed)
18 steps (18 passed)
0m0.038s

All the scenarios passed individually; let's see how we do when we execute them all at once:

4 scenarios (4 passed)
37 steps (37 passed)
0m3.718s

Everything is rock solid so far. Am I testing the right file? I change the first scenario's "Then I should see "Category" in the "A1" cell" line to this: "Then I should see "CategoryABC" in the "A1" cell".
When I execute "cucumber features" I get the following output:

4 scenarios (1 failed, 3 passed)
37 steps (1 failed, 6 skipped, 30 passed)
0m3.710s

I see that the scenario I just changed is now failing:

Then I should see "CategoryABC" in the "A1" cell # features/step_definitions/excel_steps.rb:23
  expected: "CategoryABC",
    got: "Category" (using ==)
  Diff:
  @@ -1,2 +1,2 @@
  -CategoryABC
  +Category
  (Spec::Expectations::ExpectationNotMetError)
  ./features/step_definitions/excel_steps.rb:24:in `/^I should see "([^\"]*)" in the "([^\"]*)" cell$/'
features\excel.feature:13:in `Then I should see "CategoryABC" in the "A1" cell'

I change it back to the original one and everything is OK again.

One more thing I need to do: when the feature is complete, I'd like to close Excel.

I use the cucumber's at_exit hook to accomplish that:

at_exit do
  ExcelHandler.instance.close_excel
end

When I execute the cucumber feature I see Excel popping up and closing down.
What if I wanted to run the feature in headless mode? It's simple, I just have to change the @excel.visible = true value to false in the excel_handler.rb file.

I think cucumber is a great way to automate testing on an Excel spreadsheet. The cucumber features can serve both as requirement documents and test scripts, hence providing executable documentation. They are easy to read and understand for the entire team.

You can get the demo's source code from my github account.

I developed the code on Windows 7 using ruby version 1.9.1. You also need cucumber and the rspec gems installed to run the code.

Happy coding!

Automated Testing of an OBA App with Ruby

Hello!

You could call a me a late adopter, or a busy dad, but what matters is that it's finally here. I am writing my first blog post!!
I was thinking what the best topic could be for The First one? How I got into programming? Or what I worked on yesterday? Maybe a little bit of both.

I've been doing .NET development as a day job but I am hacking out Ruby/Rails code in the evenings.
I am test infected, I can't live without tests and it really bothers me when I see code written without supporting tests around it. I've seen far too many projects starting out with everything running smooth: features are being developed with the speed of light, everybody is happy. And then, maybe 4-6 months into the project development speed slows down, defect rates go up and the code is just plain fragile. Nobody dares to touch it, not knowing what could go wrong. Changing anything is close to impossible.

I admire the testing enthusiasm I see in the Ruby community. The way they practice TDD, BDD, the way they won't write a single line of code without a failing test first. And while Ruby is really flexible, it can be very dangerous. I'd say not having sufficient test coverage in a dynamic language is like you sitting in car running towards a cliff.

I work on an OBA (Office Business Application) project at work. We have decent unit test coverage, however, not having automated functional testing is a risk. Sure we have QA staff, they can just go through the test scripts manually over and over. I think what could automated should be automated.
I bumped into the win 32 OLE automation object in ruby (http://ruby-doc.org/core/classes/WIN32OLE.html) a couple of months ago but I never had the time to fully investigate it. I was able to open Excel, read and write a value in the cell, save the worksheet and close it. This was all cool, but not enough.

require 'win32ole'
require 'test/unit'
ExcelTest < Test::Unit::TestCase

 def setup
  @excel = WIN32OLE::new("Excel.Application")
  @excel.visible = true
  @workbook = @excel.Workbooks.Open("C:\\path_to_your_excel_file\some_file.xls")
  @sheet1 = @workbook.worksheets(1)
  @sheet2 = @workbook.worksheets(2)
 end

 def test_should_verify_captions
  assert_equal('Category 1', @sheet1.cells(2, "A").value)
  assert_equal('Child 1', @sheet1.cells(3, "A").value)
  assert_equal('Child 2', @sheet1.cells(4, "A").value)
  assert_equal('Child 3', @sheet1.cells(5, "A").value)
  assert_equal('Child 4', @sheet1.cells(6, "A").value)
 end
end

I am trying to use Ruby unit testing. It's easy and simple for now. In the setup method I create an Excel object with the tested workbook loaded into it. I create two instance variables for the worksheets to keep my code simple.

Then in the first test, I verify caption cells. Nothing magical here, I just make sure that in the 2nd row's "A" column the cell should have a value of "Category1".

To be continued...