Friday, May 1, 2015

(Software Engineering) Meeting Best Practices

The TL;DR Version:

Software engineering teams should have two types of engineering meetings:
  1. A forward-looking one that explores new technologies
  2. A self-checking one that discusses current issues and challenges

Have them biweekly, one type of each every single week. Use active listening techniques to encourage equal and engaged participation. Have them in the morning, afternoons should be reserved for writing code and getting work done.

(Software Engineering) Meeting Best Practices

It was a Monday in February of 2010, around 11 am when I received a ping in Campfire to discuss the place we would go to have lunch and have our weekly Tech Talk meeting. We tried to combine lunch with our engineering conversations on Mondays, as that was the day when everybody was in the office. We decided to go to a restaurant, which did not work for us very well. We just couldn't have an engaging conversation when someone was fighting with the fries and the other person heard only half of what the speaker was saying thanks to the loud music at the restaurant.

A year later, when I worked for another company, we had no technical conversations at all. I initiated brown bag lunches combined with watching technical talks, but that was pretty much it. We never really had a recurring event to discuss code or best practices.

I was the first engineer at my current employer. We could have started whatever made the most sense for our team, however, we did not really need a technical meeting until our team grew. We could just stay longer on the call after our morning standup and discuss topics right there. We did not need to schedule and break for another meeting.
In retrospect, I had waited too long to start any kind of engineering meetings. It was one of our senior engineers who started Lightning Talks, and that meeting turned into our "forward-looking", tools, languages, frameworks exploration meeting.

At the beginning of this year, I also started "self-checking", architecture meeting. Engineers can (and are encouraged to) sign up with topics they want to discuss. We usually have one larger topic that someone presents, and we have one or two minor discussions around smaller subjects if we can fit them into one hour. Our team is remote, the presenter shares his/her screen during the Zoom session to show the slides.

We set an order of the participants at the beginning of the meeting, and we go around the room following that order. This way everybody has a chance to speak up, ask questions, and voice an opinion. If someone does not have questions or comments, that person still has to state he/she does not have anything to add. I found this technique working really well for us, as the discussion is not hijacked by opinionated and very vocal team members.

We have our "self-checking" meeting on Tuesday one week, and our "forward-looking" meeting on Thursday the next week. This way there is a large enough gap between these two meeting.

One day my schedule was scattered with meetings and 30-60 minutes break in between them. I had a chunk of 20 to 45 sessions to get any work done. That works for sending an email or killing small chores, but it's just not enough time to get in the zone, understand a problem, and solve it. I need about 3-4 hours of uninterrupted working time to get "wired-in" and be effective.
Therefore, I asked our product and leadership team to support my idea: let's condense all meetings for engineers in the morning, and leave their afternoons meeting-free. This way the engineers will get more done, and the business will benefit from it.

I hope you will find these techniques helping the engineering team to be more effective.

Tuesday, April 14, 2015

The Long and Winding Road to US Cizitenship

Today I became a US citizen.

A naturalized one. Which means I am eligible to vote, run for office, serve on a jury. However, I'll never be a US President. But hey, I could be the Governor of California.

It was a cold, snowy day in early January of 1998, when I set foot on US soil for the first time. I landed in Minneapolis, where my brother lived at that time. The previous week I was a university student with an easy-going outlook on the future, the next week I was an exchange student from Hungary, working at a greenhouse in the north suburbs of Minneapolis.

I traveled a lot that year, thinking I might never be back to this country. I flew back home in March 1999, continued my studies at the university, which I paused for a year to improve my English and to see the world.

However, I visited Seattle and Vancouver, BC in the fall of 1999, just six months after I left the US. I remember landing in Seattle, standing in the middle of the airport terminal thinking "I am home".

I decided to pursue a Ph.D. degree after graduation. In retrospect, I was just buying time, trying to find a way to come back and live in the US.

My brother arranged a job interview with his former manager, who had a web design shop in Minneapolis. This small business owner thought if I am half as smart as my brother is, he was going to get the better end of the bargain. I got hired on the spot, but I had to find an exchange student visa to make my employment legal. I found one, and in 2001 May I moved to the United Sates for good.

The first year went by fairly fast, but my exchange student visa had an expiration date. I switched to a work visa which allowed me to stay and work in the country for up to 6 years.

I went from one company to the other, did my traveling journeyman phase of my professional career. We bought a house in May 2005, and three weeks after moving in I received a phone call from a headhunter, who was trying to find software engineers for a large, Fortune 500 company in the Cleveland, Ohio area. I decided to go through the hiring process, and the next thing I knew was selling our house after the mere 3 months of purchasing it, and we moved to Ohio.

I was put on a fast track with my permanent residency application, which was sponsored by my employer. I received my green card after 18 months. I was happy when I opened the mailbox finding the letter from USCIS notifying me about adjusting my status to a permanent resident.

I had to wait 5 years before I became eligible for US citizenship. In fact, I could have become one in 2012, but moving from Cleveland to Chicago was a big enough challenge for us at that time.

Last year, when we came back from Europe and we entered the country, we had to go through US immigration. Our children are US citizens, we had green cards, but we still had to wait in line with "the visitors" to enter into the country we called home. That was the moment we decided to do something about it. We filed our paperwork, we prepared for our civic test, went through the interview, and today we recited the oath to become part of this Nation.

Sunday, March 8, 2015

Education

On the days when I go to the office, I have an about 25 minutes walk from the train station. I had listened to music for a long time, but a few weeks ago I switched over to podcasts. I have heard great things about "This Developer's Life", and after listening to the first couple of episodes, I was hooked. About a week ago I listened to the session on "Education". I have two children, this topic is a very important one for me. I care a great deal about how my children are educated, what they like to learn, what they are interested in.

The podcast discusses the topic of "do you need formal education to be a good software engineer"? I am not going to give out the conclusion here, please listen to that episode if you have time. However, as I was listening to it, my mind started cruising.

I worked for one of the largest insurance companies in the US as a software engineer. I had a coworker there who was roughly 15 years older than me. He was burnt out, the 9-5 kinda' guy. We were expecting our first child at that time and I received one of life's big lessons from him. He said: "Attila, when your child goes to high school, you should discourage them from learning software engineering. All those jobs will be outsourced, they will be better off being a plumber, than a software engineer." I don't blame him for saying this. Indian contractors were imported to do QA and other tasks for us. They worked from dawn 'till dusk without a break, causing resentment among my fellow software engineers.

But I disagreed with him. I will do everything I possibly can to encourage my children to be software engineers. My dream is that they will love data and math. "Data is the new oil," - said the European Consumer Commissioner according to the book, Predictive Analytics. This profession has a bright future, the possibilities are endless in their lives. I read this somewhere: "No humans will be needed to collect tolls on the highway, but software engineers will have to write code to keep the system running."

Source: The Activist Post

Look at the job market after the great recession at the end of 2008. Corporations' profit is through the roof, Wall Street is flying high while the number of active workers remained at or close to what it was at the height of the downturn. Companies achieved this with increased level of automation. They had to tighten their belts, but money was pumped into automation. Who did they need to drive that growth? Software engineers. And now, when the market is doing better, corporations learned to live lean, being very efficient with fewer people through automation.

Hardware is cheap and getting cheaper. There is only one component they need to fuel the growth based on automation - Software Engineers. This is a great time to be one of them, tell your children to start coding!

Thursday, January 15, 2015

The Case For and Against Cucumber

The TL;DR version

Cucumber has 3 benefits:

  1. Feature Discovery
  2. Automated Acceptance Testing
  3. (Executable) Documentation
In order to use Cucumber successfully within your organization, you need to take advantage of at least 2 of these benefits.

The Case For and Against Cucumber

Last week I gave a talk on Cucumber at CodeMash. I was glad to see the roughly 40 people who came to hear me despite being scheduled as one of the last sessions of the conference.

I ended my talk with this very personal story. I had worked in the Microsoft .NET space for 8 years, but I wanted to do something else. I was fascinated by the Ruby community, the innovation, the sharing I had seen among its members. I lived in Cleveland, OH, and there were only a handful of companies working with Ruby at that time.

My ticket to the Ruby World was my familiarity with Cucumber. My good friend - Joe Fiorini - approached me if I'd be interested in joining their company as a QA Engineer, helping them with QA automation with Cucumber. I was eager to say yes and joined them shortly after.

I wrote the first couple of features, showed them how to write Gherkin. Our test suite happily grew during the first few months of my employment. However, as more and more engineers joined, the case against Cucumber increased. Some of the engineers said they are not against acceptance testing, but those acceptance tests should be written in RSpec and not in Cucumber. Cucumber seemed to them an unnecessary extra layer they did not need.

I felt sad and disappointed. Why my fellow engineers were not seeing the value of Cucumber? What did I do wrong? Should I have spent more time explaining the values of executable documentation? I felt helpless. I asked Jeff "Cheezy" Morgan - who knows a lot more about the values and application of Cucumber at various organizations - to have breakfast with me and one of the engineers.

We met with Cheezy a few weeks later. I told him: "Cheezy, I think Cucumber is a fantastic tool, it expresses business logic like nothing else. Our company should use it. Please, be the judge here, what are we doing wrong?" Cheezy had one question: "Who is reading your Gherkin?" I said: "Who? It's us, the engineers, and maybe our QA folks." He said: "You should not use Cucumber, you would be better off with just RSpec. Cucumber is a tool for discovering requirements." "Huh?!"

I went back to work feeling a bit disappointed. I used Cucumber for acceptance testing, I did not want to hear about any other tools to do that.

It took me a few months to realize that Cheezy was right. I blindly used Cucumber for its expressiveness, and not for its value as a feature discovery tool.

Fast forward a few years to today and I wonder, why Cucumber or Gherkin is useful to us at Hireology. The answer is clear now: the entire Product, QA and Engineering team values and leverages Cucumber for feature discovery. Product will try writing a couple of scenarios when they brainstorm on a new feature. Those scenarios will be fine-tuned, extended with new ones during our 3 Amigos Meeting (a meeting to flush out feature requirements with Product, QA and Engineering). We just happen to automate those specifications during the development process.

I love how we start thinking about edge-cases well before the development begins with the help of Cucumber and Gherkin. What if the external system is not responding? Where will the user be redirected after a successful form submission? The benefit of doing this kind of planning is a more accurate estimation. Estimating engineering effort of a feature is hard, but if you know what you need to build, then at least you can take a decent stab at it, it won't be a complete swag.

We successfully use Cucumber for (1.) feature discovery and for (2.) automated acceptance testing. Now on to its third benefit: documentation.

Our Cucumber (Gherkin) scenarios are living together with our code base. Looking at them is still hard and not available for everyone at our company. I'd like to make all our features accessible to everybody, from our CEO to all our sales folks. "How does feature X work?" "I don't know, go through the feature document by clicking on this hyperlink."

Have you tried reading up on RSpec's mocking and stubbing functionality. In case you have, I am sure you have visited the Relish app. Take a look at the page that describes a basic functionality of RSpec mocking. Does it look familiar? Well, there is a Given/When/Then text in there. The most important question: is that useful? Can you learn the tool just by reading through that? That text is coming from RSpec's own source code. The RSpec developers packaged up their Cucumber scenarios and presented it in an elegant, nicely formatted, searchable app. Relish app is the prime example of executable documentation.

Publishing our more than 200 scenarios is my next goal. We use Cucumber for feature discovery, automated acceptance testing, we should use it for documentation as well.

Thursday, January 1, 2015

Fast TDD in Clojure with Fireplace.vim

I've been looking at Clojure for the past 18 months. I prefer learning and practicing by writing a failing test first, but unfortunately, the feedback loop through TDD in Clojure is slow. How slow? Well, it's between 4-10 seconds depending on the size of the project. I am still new to the language, I want to tweak my code a tiny bit and see if that change broke my tests. I am used to Ruby's almost immediate test execution, and the long wait for running the tests in Clojure makes it less effective.

In Ruby Land, I am used to running a large number of tests (958 examples in our application) in about 3.8 seconds. In a brand new Clojure project, it takes about 4 seconds to run the only failing test. This is no surprise: Clojure code has to be be compiled to Java byte code, where the compilation takes time.

I bumped into Ben Orenstein's great "Tips for Clojure Beginners" blog post a few weeks ago. It's a must read if you're new to Clojure. Vim is my preferred editor, and he wrote about a vim plugin by Tim Pope, called fireplace.vim. I remember looking at it briefly, but for some reason, I did not give it a try at that time.

A few days later I hacked on some code in Clojure again, and it reached a point where I threw my hands in the air and declared: "enough is enough!" I caught myself checking out Twitter and other websites as I had to wait about 10 seconds to run the tests after a simple change. I went through this blog post, where the author talks about using fireplace.vim for test execution. I gave it a try, and there is no turning back!

I installed fireplace.vim with pathogen. I opened another tab in my terminal, navigated to the root directory of my Clojure project. Fired up lein repl there and noted what the port number was.

In this case, 53844 was the port number for the nREPL server. I connected to that from my vim session in the other terminal tab by typing the vim command :Connect.

Fireplace gently investigated which nREPL server I wanted to connect to. I chose (the obvious) option one, it used localhost and I had to provide the port number from the other tab, which was 53844.

I submitted this option, and I was connected to the nREPL in the other tab. Fireplace lets me run the tests in the currently selected pane by using the :RunTests command. I did that, and much to my surprise the tests executed almost instantaneously. I did it once more (or maybe 5 times) just for the heck of it! This is what I found in the quickfix list:

I made the test pass, the output was terse. I guess there isn't much to say when all my expectations are passing. I included an animated gif here to show you what it feels like running the tests. Super sweet, don't you think!?

When I change a Clojure file in a different buffer (other than the buffer where my tests are), I need to Require! those files again. I get around this by writing all my functions right above the tests in the same buffer, and moving them to their final place when I feel confident about them.

There is an easier way to connect to a REPL by using fireplace.vim's :Piggieback! command. Please read the docs of this great vim plugin, that's how you can learn all the other features (like macroexpand) I have not described in this blog post.

My personal shortcut to run the tests is ,r. Setting it up with vim was easy:
:nmap ,r :RunTests<CR>. With this change, I had the same joy in Clojure as I've had with Ruby and RSpec for years. Bye-bye checking out while I am test driving my code in Clojure!

Update on 01/31/2015

I've been using this keybinding with fireplace in vim recently: :nmap ,r :Require! <bar> Eval (clojure.test/run-tests)<CR>. It picks up any changes I make in the source and the test files as I require packages before every test run. I'd recommend giving this a try.

Saturday, December 6, 2014

Cucumber Ain't Bad, But You Might Be Doing It Wrong

My talk got selected for CodeMash 2015, and I was invited to speak there in January 2015. The best way to prepare for the talk is by practicing it a couple of times before the conference. I've worked hard on my message, on my slides, I even watched Ben Orenstein's presentation on "How to Talk to Developers".

This past Tuesday I gave the talk's initial version at the Chicago Ruby User Group. A good crowd came out to listen to what I had to say about the topic despite the cold, wintery weather. I received a couple of questions, had good conversations with some of the folks afterwards.

Are you unsure about the topic? Here is the pitch for my talk:

The cry from developers is unanimous: “I don’t like Cucumber. My acceptance tests are brittle and hard to maintain!” And in fact, they can be.

This presentation will show you what tools and techniques you should use to write independent, stateless step definitions that can be weaved together to form a testing DSL.

You can review the code examples in this repo.

And here are the slides:

Thursday, August 21, 2014

Unobtrusive Logging with a Decorator

I've recently added logging to LightService. The organizers pipe log messages to whatever you define as your logger.

I, [LS] - calling organizer TestDoubles::MakesTeaAndCappuccino
I, [LS] - @keys in context: :tea, :milk, :coffee
I, [LS] - executing TestDoubles::MakesTeaWithMilkAction
I, [LS -   expects: :tea, :milk
I, [LS] -   promises: :milk_tea
I, [LS] -     keys in context: :tea, :milk, :coffee, :milk_tea
I, [LS] - executing TestDoubles::MakesLatteAction
I, [LS] -   expects: :coffee, :milk
I, [LS] -   promises: :latte
I, [LS] -     keys in context: :tea, :milk, :coffee, :milk_tea, :latte

This blog post tells you the story of how I added logging to this gem I maintain.

What do you think of this code?

class WithReducer
  attr_reader :context

  def with(data = {})
    @context = LightService::Context.make(data)
    self
  end

  def reduce(*actions)
    raise "No action(s) were provided" if actions.empty?
    actions.flatten!

    actions.reduce(context) do |context, action|
      action.execute(context)
    end
  end
end

It's not that hard to figure out what's going on in this object. It has 2 methods, #with and #reduce. The #with method is simple, it creates a Context object through the factory method and returns self for chaining. #reduce takes an arbitrary number of actions and reduces them by calling their #execute method.

Ok, let’s add logging around this:

class WithReducer
  attr_reader :context

  def with(data = {})
    logger.info("[LS] - calling organizer <#{organizer.to_s}>")

    @context = LightService::Context.make(data)

    logger.info("[LS] - keys in context: #{extract_keys(decorated.context.keys)}")

    self
  end

  def reduce(*actions)
    raise "No action(s) were provided" if actions.empty?
    actions.flatten!

    actions.reduce(context) do |context, action|
      result = action.execute(context)

      logger.info("[LS] - executing <#{action.to_s}>")
      if defined? action.expects and action.expects.any?
        logger.info("[LS] - expects: #{extract_keys(action.expects)}")
      end
      if defined? action.promises and action.promises.any?
        logger.info("[LS] - promises: #{extract_keys(action.promises)}")
      end
      logger.info("[LS] - keys in context: #{extract_keys(context.keys)}")

      result
    end
  end

  private

  def extract_keys(keys)
    keys.map {|key| ":#{key}" }.join(', ')
  end
end

This is the exact same core logic as the one in the first example, except it got polluted with logging. This code works, but seeing what the two instance methods do is not obvious, far from simple. What's wrong here? First, two separate concerns (logging and the core functionality) got squashed into one. Second, when the logging logic changes the underlying core logic will be touched violating the Single Responsibility Principle.

Logging and the core functionality of this code should be separated out. The logging needs to wrap around (decorate) the core functionality. It should look something like this:

A decorator - the WithReducerLogDecorator in this case - publishes the exact same interface as the object it decorates, it has the with and reduce methods. I started with those when I went about refactoring this:

class WithReducerLogDecorator
  def with(data = {})
  end

  def reduce(*actions)
  end
end

Then I "wrapped" the original logic with this decorator. The log decorator delegates the calls to the wrapped object, which is WithReducer in this case:

class WithReducerLogDecorator
  attr_reader :decorated

  def initialize(decorated = WithReducer.new)
    @decorated = decorated
  end

  def with(data = {})
    decorated.with(data)
  end

  def reduce(*actions)
    decorated.reduce(*actions)
  end
end

At this point it does not matter which WithReducer class I'd use. Using the original one or the decorated should produce the exact same functionality.
I started adding logging around the decorated methods, refactoring the #with method was first:

class WithReducerLogDecorator
  attr_reader :decorated, :organizer

  def initialize(decorated = WithReducer.new, organizer)
    @decorated, @organizer = decorated, organizer
  end

  def with(data = {})
    logger.info("[LS] - calling organizer <#{organizer.to_s}>")

    decorated.with(data)

    logger.info("[LS] - keys in context: #{extract_keys(decorated.context.keys)}")
    self
  end

  ...
end

Adding logging around the #reduce method was a bit harder. It reduces the provided actions by calling their execute method. There is no easy way to hook into that process just yet. However, calling the provided block within the reduce block opens up the routine for extension (Open/closed principle). I changed the original reduce method like this:

class WithReducer
  ...

  def reduce(*actions)
    raise "No action(s) were provided" if actions.empty?
    actions.flatten!

    actions.reduce(context) do |context, action|
      result = action.execute(context)
      # ::: Invoke the provided block :::
      yield(context, action) if block_given?
      result
    end
  end
end

The logging logic can now wrap around the #reduce method like this:

class WithReducerLogDecorator
  ...

  def reduce(*actions)
    decorated.reduce(*actions) do |context, action|
      # ::: All this code is executed within the decorated reduce block :::
      logger.info("[LS] - executing <#{action.to_s}>")
      if defined? action.expects and action.expects.any?
        logger.info("[LS] - expects: #{extract_keys(action.expects)}")
      end
      if defined? action.promises and action.promises.any?
        logger.info("[LS] - promises: #{extract_keys(action.promises)}")
      end
      logger.info("[LS] - keys in context: #{extract_keys(context.keys)}")
    end
  end

  private

  def extract_keys(keys)
    keys.map {|key| ":#{key}" }.join(', ')
  end
end

And with that I wrapped the original #with and #reduce methods with logging from outside with a decorator. I separated the two concerns: the core logic and the logging around it. I only need to touch the decorator object when the logging logic needs to be changed and I don't even need to look at the logging code when the core logic changes.

I learned about these concepts 9 years ago when I used Aspect Oriented Programming (AOP). I was fascinated by Around Advices, where logging was applied around a routine, beautifully separating the concerns.