SmartLogic Logo (443) 451-3001

The SmartLogic Blog

SmartLogic is a web and mobile product development studio based in Baltimore. Contact us for help building your product or visit our website to learn more about what we do.

TestPilot – Rails Integration Testing Pattern

October 26th, 2010 by

I’ve been thinking about ways to simplify rails integration testing. I wanted to see how well I could do it without cucumber and rspec. I decided to go pure minitest with capybara to help out. What emerged was the TestPilot pattern. Let’s check it out.

Note: all of this code is runnable via the test-pilot-demo on github.

Testing the Index

This one is super simple. It’s just minitest and capybara:

  test "See the index" do
    visit root_path
    assert_see "Blog Posts"
  end

You’ll notice I threw in “assert_see”. It’s a simple method that wraps some capybara code:

  def assert_see(content)
    assert(page.has_content?(content), "Expected page to contain \"#{content}\", but it didn't")
  end

Testing Create

Now we’re getting into some real data business. These were my goals:

  1. Delegate filling in the form to a reusable helper
  2. Make the assertion in helper
  3. Deal with models like models, not like attribute hashes

Here is my test code:

  test "Create a new post" do
    BlogPilot do
      create(
        Blog.new(:title => 'a blog title', :body => 'a blog body')
      )
    end
  end

I delegating filling out the form to BlogPilot#create_blog. I also did it by passing in a blog object to be created. You’ll note there are no assertions. This was a personal choice. I wanted to write out my steps such that they are assumed to succeed. Any step that fails will fail the test.

So, what’s with this pilot business? Let’s check it out:

class BlogPilot < TestPilot::Core
  def create(blog)
    visit root_path
    fill_in "Title", :with => blog.title
    fill_in "Body", :with => blog.body
    click_button 'Create Post'
    assert_on root_path
    assert_see blog.title
    assert_see blog.body
  end
end

That’s pretty straightforward! It’s just a capybara helper. You’ll note the step makes assertions. The only way the test can proceed is if this step passes. That way, we’ve also abstracted the notion of “successfully creating a blog” to a single place. So if we switch between divs, tables, and lists to display our blog, we only change our expectations here.

So what’s this TestPilot::Core business? There are two reasons I had to make TestPilot:

  1. I didn’t want tons of flat helpers (i.e. create_blog, create_comment, create_user)
  2. I didn’t want to have to instantiate the pilot and keep calling it. I wanted “BlogPilot do … end”

TestPilot

Two things should be going through your head right now:

  1. Wait a minute, he said more simple! How is introducing some new framework going to make this simpler?
  2. HOW DO I GET THE GOODNESS IN ME

Luckily, the answer to both questions is the same. This is test pilot:

module TestPilot
  class Core < ActionDispatch::IntegrationTest
    def initialize ; end # override new from inherited class
    def self.inherited(subclass)
      TestPilot::Dsl.send(:define_method, subclass.to_s) do |&block|
        subclass.new.instance_eval(&block)
      end
    end
  end
  module Dsl ; end
end

It’s not a gem. It barely deserves to be in lib. This could go right in your test_helper.rb. What’s going to break? There are two lines of real code in there. And if it breaks, guess what? It’s right there in test_helper, not in some gem you have to maintain your own fork of so you can have “Nick’s raspberry flavored test::pilot”.

Where am I going?

So what was my goal with this? My goal is to write code like this all day:

test "a logged in user can comment on another user's post" do
  joe = Factory :user
  bob = Factory :user
  UserPilot do
    log_in joe
  end
  PostPilot do
    @post = create(Factory.build(:post))
  end
  UserPilot do
    log_out
    log_in bob
  end     
  CommentPilot do
    create(@post, Factory.build(:comment))
  end     
end

Notice I left out any assertions. I think it would be cool to have the pilot steps themselves assert a successful action, since you probably want to check for it every time anyways.

I enjoy that this gives me one of my favorite benefits of cucumber: reusable steps. But, I can carry around instance variables in a highly visible manner, and call specialized factories without creating a one-off step that obfuscates my codebase.

So what do you think? Would you rather code cucumber, rspec, raw test unit, test pilot? Or have I inspired you to roll your own?

Code: test-pilot-demo.

  • http://tripledogdare Evan Light

    Definitely still Test::Unit with Shoulda and Coulda. The lack of explicitly expressing an expectation troubles me. I prefer to roll my own domain-specific assertions per application (e.g., see http://github.com/elight/ruby-conway/blob/master/test/rule_test.rb) where the assertions sometimes include their appropriate setup and, occasionally, yield to a code block that I pass from within the test.

  • http://www.smartlogicsolutions.com/nick Nick Gauthier

    Hey Evan,

    I like that assertion style you have there. Looks great for unit testing. In this post, I was trying to explore the integration testing side of things.

    In integration testing I was always hitting this one wall: DRYing up the interface between my test and my web app, which is where this pattern came in.

    Honestly I drafted a version of this post where the assertions were in the test instead of in the pilot. It can go either way. I’m sure you know better than I the benefits of lean and transparent code: we can do it however we want to!

    Really everything I wrote here was how I felt like doing it. The only thing that test pilot does is encapsulate helper methods so you don’t have to give them funny prefixes, you nest them in a block.

    Thanks for the input!

    -Nick

  • http://tripledogdare Evan Light

    Maybe it’s evil but I tend to be a little inconsistent in integration tests. Sometimes I hide the setup behind a macro and sometimes I don’t. In either case, when I roll my own macro, the most important part (for me) is giving the macro a semantically useful name so that, at a glance, I have a good idea what the macro actually does.

    “UserPilot”, “BlogPilot”, etc throws me for a loop because the names are nouns; they lack a verb that tells me what they’re doing.

    That’s where I tend to write a macro named something like “given_blog_post_with_…” or “given_user_that…”. For me, those methods wrap calls to Coulda but it could just as easily be Shoulda, vanilla Test::Unit, or RSpec.

    Those macros, because they’re just humble methods, easily removed to a helper.

    To each their own. I love seeing how other people test.

  • http://www.smartlogicsolutions.com/nick Nick Gauthier

    Cool. For me the test pilot makes a lot of sense. The Pilots are the “Actors” and the methods inside the block are the “Actions” that they are taking.

    It’s much more like a story this way. I could even imagine you create special pilots for specific actors, for example:

    class Nick < UserPilot
      def update_profile(args = {})
        super(args.merge(:name => 'Nick'))
      end
    end

    And for the record, I don’t test like this. But I want to :-D

  • http://technicalpickles.com Josh Nichols

    This reminds me a lot of Gizmo, or maybe the way I’ve been using Gizmo: http://github.com/icaruswings/gizmo . Considering prior art, that does help validate this as a pattern :)

    This can definitely help with making more re-usable integration steps. One downside is that it is more code and abstraction layers, so eventually you might find yourself asking “Who tests the TestPilot?”

  • http://tripledogdare Evan Light

    Nick: The “Actor” metaphor clarified it for me. The approach makes sense to me when the expectations are explicit.

    Josh’s remark raises a flag for me. The use of several different Pilots and Pilot subclasses in a test starts to smell of what I call the Over-Macro testing anti-pattern: where so many layers of abstraction are added to a piece of code so as to become painful for reader to determine what behavior is occurring.

    And, yes, I’ve named it as a pattern because I’ve committed that same sin numerous times. ;-) The clue for me that I’ve committed the anti-pattern is when I find myself digging through numerous files and/or paging up and down in a single file to understand what the test is doing.

    Not that the above invalidates the pattern. But I could see it getting hairy quickly in the case where the expectations/assertions don’t live in the block passed to the Pilot.

    For that matter, having Pilots solely focused with preconditions of the expectations/assertions would be a good strict adherence to SRP.

  • http://tripledogdare Evan Light

    It almost makes me which that “do” could be substituted with “does”. Then semantically usefully named Pilots would read awesomely:

    class Nick blog.title
    fill_in “Body”, :with => blog.body
    click_button ‘Create Post’
    end
    end

    Where the “Nick” object would handle your authentication.

    I have a little more trouble with abstract Pilots. I could see having a “BlogAuthorPilot” that abstracts the blog creation steps called from the block passed to the “Nick” pilot.

    And then my head starts to hurt because a noun is acting like a verb. ;-)

  • http://tripledogdare Evan Light

    Code formatting fail.