How to Write Smoke Tests for an Ember Rails Stack

The following story shows the importance of smoke tests when writing an app using the Ember+Rails stack. It covers the most basic example, and even then things go awry. In addition, it shows an example of an end-to-end test and some of the difficulty in setting up an environment for smoke tests. Finally, it mentions exhaust, a library that can ease this pain.

Imagine, you’re ready to start writing your next great application. It’s like TodoMVC meets Twitter meets eBay: Users will write down their To Do lists and auction off tasks to other users in 140 characters or less. Investors are very excited. You’re very excited. High-fives all around. You’re going to change the world, and you’re going to do that using all the right technologies: Ember for the client side and Rails for the backend!

Here is your first user story:

Given I am a user
When I go to the todos page
Then I should see a list of all todos

Getting Started with Ember Rails

Piece of cake! You start by installing the latest Ember:

> npm install -g ember bower 
> ember new todo-frontend 
> cd todo-frontend

“I don’t care what @dhh says,” you think out loud. “I’m a rockstar developer, so I’m gonna test-drive it like I stole it!” You don’t have an API to test yet, so you start by mocking it. You install pretender and generate your first acceptance test:

> ember install ember-cli-pretender 
> ember generate acceptance-test todos
// tests/acceptance/todos.js 
import Ember from 'ember'; 
import { module, test } from 'qunit'; 
import startApp from 'emberbook/tests/helpers/start-app'; 
import Pretender from 'pretender';

let TODOS = [ 
  {id: 1, title: "write a blog post"}, 
  {id: 2, title: "let people read it"}, 
  {id: 3, title: "... profit"} 

module('Acceptance | todos', { 
  beforeEach: function() { 
    this.application = startApp();

    this.server = new Pretender(function() {
      this.get('/todos', function(){
        var json = {
          todos: TODOS
        return [200, {}, JSON.stringify(json)];

  afterEach: function() {, 'destroy'); 

test('visiting /todos', function(assert) { 

  andThen(function() { 
    var title = find('h1'); 
    assert.equal(title.text(), 'Todo List');

    var todos = find('.todo-item');
    assert.equal(todos.length, TODOS.length);
    assert.equal(currentURL(), '/todos');

Time to fire up the test runner and watch it all burn:

ember test --serve

Yep, it’s red like the devil. First error, no route.

// app/router.js 
import Ember from 'ember'; 
import config from './config/environment';

var Router = Ember.Router.extend({ 
  location: config.locationType 
}); { 

export default Router;

Now the test can’t find the text “Todo List.” You fix this as well:

{{!- app/templates/todos.hbs }}

# Todo List

{{#each model as |todo|}}
  <p class='todo-item'>

Now the test breaks because there are no To Do items on the page. You move forward:

// app/routes/todos.js 
import Ember from 'ember';

export default Ember.Route.extend({ 
  model() { 

Can’t find the “todo” model. One more fix:

// app/models/todo.js 
import DS from 'ember-data';

export default DS.Model.extend({ 
  title: DS.attr(), 
  complete: DS.attr("boolean") 

Whoa, it’s green! You pat yourself on the back. But you can’t actually deliver the story until there’s an API to back it up. Time to grab the latest version of Rails.

Testing an Ember Rails Application

This is going to be an API rather than a full-fledged application, so you only need a few gems. You spend several minutes pondering a post you just read about building Rails API apps and deciding whether to use the Rails 5 --api flag. However, you want this product to hit the market “yesterday,” so you just roll with what’s stable:

> gem install rails 
> rails new todo_api --skip-test-unit --skip-bundle 
> cd todo_api

You also skipped test-unit because you’d rather use Rspec’s eloquent DSL for this project’s tests.

Time to boil the Gemfile down to the few gems you really need. This API only needs to serve JSON, so out with anything related to the asset pipeline:

# Gemfile
source ''

gem 'rails', '4.2.3' 
gem 'sqlite3' 
gem 'active_model_serializers'

group :development, :test do 
  gem 'rspec-rails' 

Bundle it and get ready to write some specs!

> bundle install 
> rails generate rspec:install

Time to bang out the first request spec:

# spec/requests/todos_spec.rb
require "rails_helper"

RSpec.describe "Todos", :type => :request do

  # letBANG because `before {expected_todos}` looks funny 
  let!(:expected_todos) do do |i| 
      Todo.create(title: "Todo #{i+1}") 

  it "lists all todos" do 
    get "/todos" 
    todos = extract_key_from_response("todos", response)   
    expect(todos.count).to eq expected_todos.count 

  def extract_key_from_response(key, response) 
    json_response = JSON.parse(response.body) 
    expect(json_response).to have_key key 


You run the test. Watch it go red! First gripe: no model. You fix that:

class Todo < ActiveRecord::Base

It’s migration time!

<code class="language-bash rails generate migration CreateTodos title:string</code>
# db/migrate/#{timestamp}\_create\_todos.rb
class CreateTodos < ActiveRecord::Migration 
  def change 
    create_table :todos do |t| 
      t.string :title 
      t.timestamps null: false 
> rake db:migrate

And of course, it needs a route.

# config/routes.rb
Rails.application.routes.draw do 
  resources :todos, only: :index 

The test is still red, because there is no controller. Next fix:

# controllers/todos_controller.rb
class TodosController < ApplicationController 
  def index 
    render json: todos 


  def todos 

All green! Time to boot up the app(s) and bask in your own creative genius. You run the following commands in different terminals:

> rails server 
> ember server

You can’t wait to see your baby running. You open “http://local:4200/todos”… and you get a blank screen. Now that was disappointing. Both test suites are green. What could have gone wrong?

After some inspection into the problem, you notice a GET http://localhost:4200/todos 404 (Not Found) in your logs. Well, crap. Ember doesn’t know the API is on a different domain. Well, that’s an easy fix. You just need to specify a host in the application adapter. And since you are attractive and intelligent, you know better than to hardcode the host. After all, it will change for staging and production. So you open config/environment.js in your Ember app, and you add the following:

// config/environment.js                                                                                             if (environment === 'test') {
} else {
  ENV.API_HOST = (process.env.API_HOST || 'http://localhost:3000')

You set the test environment’s API_HOST to an empty string, so that the current tests keep working. All other environments use the Rails’ default http://localhost:3000 unless an API_HOST environment variable is set. Now you can create an application adapter and fix the bug:

// app/adapters/application.js 
import DS from 'ember-data'; 
import config from '../config/environment';

export default DS.RESTAdapter.extend({ 
  host: config.API_HOST 

W00T! Fixed that problem. Time to run the tests. Once again, you start the Rails and Ember apps in two different terminals.

> rails server 
> ember server

Dang it. The app still doesn’t work. You didn’t set up CORS so that modern browsers can talk to the Rails API. You add rack-cors to the Gemfile, bundle it, and add the simplest policy you can think of to get things working. Allow everything!

# Gemfile
gem 'rack-cors', :require => 'rack/cors'
bundle install
# config/application.rb
config.middleware.insert_before 0, "Rack::Cors", :debug => true, :logger => (-> { Rails.logger }) do 
  allow do 
    origins '*' 
    resource '*', 
      :headers => :any, 
      :methods => [:get, :post, :delete, :put, :options, :head], 
      :max_age => 0 

You cross your fingers, run the tests, and start the servers. When you visit the todos route, you see your app is working. Phew! Even though your app is working now, you aren’t 100 percent happy. You spent the last hour test driving an application that didn’t work, even though all tests were green. And you didn’t even focus unit tests — all of your tests were integration tests. Shouldn’t integration tests prevent things like this?

Writing Smoke Tests for an Ember Rails App

At this point, you remember a post on the Hashrocket blog about test driving elixir apps with cucumber. You decide to try these techniques in practice and write some smoke tests. The type of integration tests that were just written are great for testing application(s) in isolation. These isolated integration tests run very fast. And because they run fast, the majority of your tests should be either request specs or Ember integration. There is no reason to have full coverage (testing every possibility) with smoke tests.

However, in order to ensure the app actually works, there should be at least one smoke test for every API endpoint. Knowing that cucumber-rails already loads your test environment, you think of the simplest solution you can to try writing a smoke test. Simply add cucumber-rails, manually start Ember and Rails, then overwrite how capybara works. This is the solution you come up with. You add cucumber to the Gemfile and install it.

# Gemfile
group :development, :test do 
  gem 'cucumber-rails', require: false 
  gem 'database_cleaner' 
  gem 'capybara-webkit' # ... 
> bundle install 
> rails generate cucumber:install

Then you overwrite capybara’s behavior.

# features/support/env.rb
Capybara.configure do |config| 
  # Don't start rails 
  config.run_server = false 
  # Set capybara's driver. Use your own favorite 
  config.default_driver = :webkit 
  # Make all requests to the Ember app 
  config.app_host = 'http://localhost:4200' 

And you write the first feature and step definitions.

# features/todo.feature
Feature: Todos 
  When a user visits "/todos", they should see all todos

  Scenario: User views todos 
    Given 4 todos 
    When I visit "/todos" 
    Then I should see "Todo List" 
    And I see 4 todos
# features/steps/todo.rb
Given(/^(\d+) todos$/) do |num| 
  num = num.to_i 
  num.times do |i| 
    Todo.create(title: "Title #{i}") 

When(/^I visit "(.*?)"$/) do |path| 
  visit path 

Then(/^I should see "(.*?)"$/) do |text| 
  expect(page).to have_text(text) 

Then(/^I see (\d+) todos$/) do |num| 
  expect(all(".todo-item").length).to eq num.to_i 

Now in separate terminals you run both Rails and Ember. This time Rails is started in the test environment so that Ember requests hit the Rails test database.

> rails server --environment test 
> ember server

You run the test suite with rake, and voilà! You see that the test suite passes. You know that both Ember and Rails are communicating end-to-end. And even though you are super proud of yourself, you’re not happy. “The servers should start automatically. Why can’t I just type rake?” you think to yourself. Here is the first solution you come up with.

# features/support/env.rb
ember_pid = fork do 
  puts "Starting Ember App" 
  Dir.chdir("/your/user/directory/todo-frontend") do 
    exec({"API_HOST" => "http://localhost:3001"}, "ember server --port 4201 --live-reload false") 

rails_pid = fork do 
  puts "Starting Rails App" 
  Dir.chdir(Rails.root) do 
    exec("rails server --port 3001 --environment test") 

sleep 2

at_exit do #kill the Ember and Rails apps 
  puts("Shutting down Ember and Rails App") 
  Process.kill "KILL", rails_pid 
  Process.kill "KILL", ember_pid 

Capybara.configure do |config| 
  config.run_server = false 
  config.default_driver = :webkit 
  config.app_host = 'http://localhost:4201' 

This solution works okay. Rails and Ember run on separate ports, so the development versions of the two servers can keep running. The only problem is the sleep 2. As the Rails and Ember apps grow, so will the time needed to wait for the servers to start. Also, this number is magic; it might take longer on another machine. How long will it take on the CI server?

What you really want to do is halt the tests until you know that Rails and Ember are running. However, after some investigation, you realize there is no way to know if Ember is running successfully. But then you notice how EmberCLI smoke tests itself.

it('ember new foo, server, SIGINT clears tmp/', function() {
    return runCommand(path.join('.', 'node_modules', 'ember-cli', 'bin', 'ember'), 'server', '--port=54323','--live-reload=false', {
        onOutput: function(string, child) {
          if (string.match(/Build successful/)) {
      .catch(function() {
        // just eat the rejection as we are testing what happens

Now you know that Ember has booted when it outputs “Build successful” and have some insight as to how you might wait for Rails.

# features/support/env.rb
  DatabaseCleaner.strategy = :truncation 
rescue NameError 
  raise "You need to add database_cleaner to your Gemfile (in the :test group) if you wish to use it." 

ember_server = nil 
rails_server = nil

Dir.chdir("/Users/mwoods/hashrocket/todo-frontend") do 
  ember_server = IO.popen([{"API_HOST" => "http://localhost:3001"}, "ember", "server", "--port", "4201", "--live-reload", "false", :err => [:child, :out]]) 

Dir.chdir(Rails.root) do 
  rails_server = IO.popen(['rails', 'server', '--port', '3001', '--environment', 'test', :err => [:child, :out]]) 

# Before timeout loop to prevent orphaning processes
at_exit do 
  Process.kill 9,, 

# if it takes longer than 30 seconds to boot throw an error
Timeout::timeout(30) do 
  # wait for the magic words from ember  
  while running = ember_server.gets 
    if running =~ /build successful/i 

  # when rails starts logging, it's running 
  while running = rails_server.gets 
    if running =~ /info/i 

Capybara.configure do |config| 
  config.run_server = false 
  config.default_driver = :webkit 
  config.app_host = 'http://localhost:4201' 

This is not a perfect solution, but it’s good enough for day one. On day two, you might decide to move the smoke tests to their own repository. After all, they aren’t Rails-specific, so they probably should be a separate project.

Problems like these and many others have happened to me. That’s the reason I’m introducing exhaust, a new gem that hopefully alleviates some of the headache associated with smoke testing an Ember+Rails stack. Enjoy!

Reference: How to Write Smoke Tests for an Ember Rails Stack from our WCG partner Florian Motlik at the Codeship Blog blog.

Micah Woods

Micah discovered his passion for Ruby and web development in college. Pair programming at Hashrocket allows him to focus on testing and quality while delivering value at a blazing fast rate.
Notify of

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Inline Feedbacks
View all comments
Back to top button