How to write an awesome Active Record bug report

10 July 2011

The main area of Ruby on Rails that I work on is Active Record. As such, I see a lot of bug reports for Active Record. This article will be about how to submit a good bug report to help people like me (Rails committers) help people like you (bug inflicted developers).

Of course, the best bug report will include a well-written test-driven patch. If you’re able to jump into the source code and fix it yourself, please just do that rather than reading this. This article is for people who have found a problem, but are not able to fix it themselves.

Create a minimal test script that will reproduce the bug

When I am trying to fix a bug, I often spend most of my time trying to reproduce what the author has reported. This can be frustrating and difficult, especially when the bug has been described confusingly or ambiguously.

Bug reports also sometimes contain a lot of code that is completely irrelevant to the problem being described. This makes it difficult to hone in on the actual problem and makes the fix take longer.

So here’s what to do:

1. Isolate the problem

Most likely you have discovered the problem in your day-to-day work. Your project probably contains 30 different models and hundreds of lines of code. I’m sure there are some dark corners. Drop into the console (by running rails c) and start trying to reduce the problem. Suppose you encountered an exception when running the following expression:

UserFolder
  .joins(:user)
  .includes(:user, :tags)
  .where(:users => { :role_id => Role.find_by_name('admin') })
  .order('user_folders.name desc')

There’s a lot going on here, and it’s probably not all directly related to producing the bug. So start trying to remove parts. Does the exception still happen if the where is removed? How about the order? Are both the includes necessary? Can the joins be removed?

2. Create a standalone test script

Suppose you reduced the above to:

UserFolder.joins(:user).includes(:user)

That’s great. But the code to reproduce the bug is still entangled in your project. We need to pull it into a separate script that anyone can run. So look at which models are relevant to the bug. It looks like just UserFolder and User matter here, and only the user association is involved. So we can take a good first stab at a standalone script:

gem 'rails', '3.1.0' # change as required
require 'active_record'

# Print out what version we're running
puts "Active Record #{ActiveRecord::VERSION::STRING}"

# Connect to an in-memory sqlite3 database (more on this in a moment)
ActiveRecord::Base.establish_connection(
  :adapter  => 'sqlite3',
  :database => ':memory:'
)

# Create the minimal database schema necessary to reproduce the bug
ActiveRecord::Schema.define do
  create_table :users, :force => true do |t|
  end

  create_table :user_folders, :force => true do |t|
    t.integer :user_id
  end
end

# Create the minimal set of models to reproduce the bug
class User < ActiveRecord::Base
end

class UserFolder < ActiveRecord::Base
  belongs_to :user
end

# Create some test data
#
# If you're demonstrating an exception, then this is probably not necessary,
# but if your bug is to do with the wrong data being returned from the database,
# then you'll probably need some test data to show that.
user = User.create!
user_folder = UserFolder.create(:user => user)

# Reproduce the actual bug!
UserFolder.joins(:user).includes(:user) # => BOOM!

Now you can test the script out. Assuming you have the rails and sqlite3 gems installed, you can simply run ruby path/to/script.rb and see if that reproduces the problem.

If it doesn’t, you need to work out what the difference is.

Notice that we are connecting to a sqlite3 in-memory database. These are great because they are lightweight and created on-the-fly when you run the script. However, perhaps your problem is database-specific, or relies on a feature that sqlite3 does not provide. If so, you’ll need to create a test database on your chosen database server (e.g. postgresql or mysql) and then modify the establish_connection call to connect to it. This hash that is passed is exactly like the details you specify in config/database.yml.

If the difference is not down to the database, perhaps there was other relevant code in your app’s models which should be added to the script. Or perhaps your test data is not quite right.

3. Find out if your bug has been fixed!

Rails development moves fast, and there’s a chance that your bug could have already been fixed in ‘edge Rails’ or a later version than you are running. Using the script we just wrote, we can easily compare the results against different Rails versions.

First, make sure you have the latest stable Rails installed (gem install rails) and then run the script again.

Assuming that does not work, try edge Rails. Use git to get the latest code:

git clone git://github.com/rails/rails.git

Now run bundler to get the dependencies:

bundle

Comment out the gem 'rails', '3.1.0 line in your script, because we will use Bundler for dependency management now. Then run:

bundle exec /path/to/script.rb

4. It hasn’t been fixed? :(

Take the script you just wrote and put it in a Gist. Now, go ahead and file the issue. Be sure to include the following information:

1. A link to your Gist
2. What you expected to happen
3. What actually happened
4. Output from running the script against the latest Rails stable version and from running it against Rails edge

This script also means that if someone else comes along in 6 months time and wants to see if the problem still exists, they have an easy way to do so.

Follow these steps and feel the love

The people who work on Rails have limited amounts of time. So if you want your bug to be fixed, you can make a big difference by providing a good test script. Since Github added support for emoji there’s a decent chance that by doing so your efforts will be rewarded with a :heart: or two. Or at least that your bug will get fixed!