Test Driven Development of a RESTful JSON API With Rails

In this article, we're going to look at how to set up a simple Rails API-only-application. Rails API-only-applications are slimmed down compared to traditional Rails web applications.

One of the many benefits of using RAILS for JSON APIs is that it provides a set of defaults that allows developers to get up and running quickly, without having to make a lot of trivial decisions.

In order to generate an API-centric framework, Rails makes it possible to exclude functionality that would otherwise be unused or unnecessary. Using the — API flag will;

• Start the application with a limited set of middleware
• Make the ApplicationController inherit from ActionController::API instead of ActionController::Base
• Skip generation of view files

In this three-part article/tutorial, we’ll build a bookstore API where users can manage their favorite book lists.

API Endpoints

Our API will expose the following RESTful endpoints:

Entity Relationship Diagram

The figure above shows a simple design of our database. I used dbdiagram.io, a free online tool that allows you to draw an Entity-Relationship Diagram painlessly by just writing code.

In this article we will cover:

• Project Setup
• Category API
• Books API
  

Project Setup 

Let us quickly generate a new project books-api by running rails new books-api --api -T -d postgresql.

Note: the --api flag tells Rails that we want to build an API-only application. Using -d allows us to specify the database type we wish to use and -T skips the default testing framework (Minitest). Not to worry, we’ll be using RSpec instead to test our API.

Gem Dependencies

We will require the following gems in our Gemfile to help our development process: 

• rspec-rails - A testing framework for Rails 5.0+
• factory_bot_rails - Provides integration between factory_bot and rails 5.0+
• shoulda-matchers - Provides RSpec one-liners to test common Rails functionality that if written by hand, would be much longer, more complex and error-prone
• database_cleaner - Strategies for cleaning databases, it can be used to ensure a clean slate for testing
• faker - Used to easily generate fake data: names, addresses, phone numbers etc.

Now that you have a better understanding of the gems we will be using, let’s set them up in your Gemfile.

First, add rspec-rails to both the :development and :test groups.

{% code-block language="js" %}
# Gemfile
group :development, :test do
  gem 'rspec-rails', '~> 4.0', '>= 4.0.2'
end
{% code-block-end %}

Add factory_bot_rails, shoulda_matchers, database_cleaner and faker to the :test group.

{% code-block language="js" %}
# Gemfile
group :test do
  gem 'database_cleaner'
  gem 'factory_bot_rails', '~> 6.1'
  gem 'faker'
  gem 'shoulda-matchers', '~> 4.5', '>= 4.5.1'
end
{% code-block-end %}

Install the gems by running bundle install.

Rspec Setup

Trusting that all the gems have been successfully installed, let’s initialize the spec directory, where our tests will reside.

{% code-block language="js" %}
rails generate rspec:install
{% code-block-end %}

This will generate the following files (.rspec, spec/spec_helper.rb, spec/rails_helper.rb) that will be used for RSpec configuration.

Now that we're done setting up rspec-rails, let’s create a factories directory (mkdir spec/factories) since the factory bot uses it as the default directory.

Almost there! Let’s add configuration for database_cleaner, factory_bot and shoulda-matchers:

{% code-block language="js" %}
# spec/rails_helper.rb
# require database_cleaner at the top level
require 'database_cleaner'
# configure shoulda matchers to use rspec as the test framework and full matcher libraries for rails
Shoulda::Matchers.configure do |config|
 config.integrate do |with|
   with.test_framework :rspec
   with.library :rails
 end
end
# ...
RSpec.configure do |config|
  # ...
  # add `FactoryBot` methods
  config.include FactoryBot::Syntax::Methods
  # start by truncating all the tables but then use the faster transaction strategy the rest of the time.
  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
    DatabaseCleaner.strategy = :transaction
  end
  # start the transaction strategy as examples are run
  config.around(:each) do |example|
    DatabaseCleaner.cleaning do
      example.run
    end
  end
  # ...
end
{% code-block-end %}

It’s quite a configuration process! I know that was rather long but the rest of the process will be very smooth.

Models

Let’s start by generating a category model and book model. We will generate our user model in the next part of our article since we will only need it for authentication purposes.

{% code-block language="js" %}
rails g model Category name
rails g model Book title author category:references
{% code-block-end %}

By default, active records model attribute is string. Since the name, title, and author is going to be string, there is no need to add one.

Adding category:references informs the generator to set up an association with the Category model. This will add a foreign key column category_id to the books table and set up a belongs_to association in the Book model.

The generator invokes both active record and rspec to generate the migration model and spec respectively. Follow the below steps:

{% code-block language="js" %}
# db/migrate/[timestamp]_create_categories.rb
class CreateCategories < ActiveRecord::Migration[6.1]
 def change
   create_table :categories do |t|
     t.string :name
     t.timestamps
   end
 end
end
# db/migrate/[timestamp]_create_books.rb
class CreateBooks < ActiveRecord::Migration[6.1]
 def change
   create_table :books do |t|
     t.string :title
     t.string :author
     t.references :category, null: false, foreign_key: true
     t.timestamps
   end
 end
end
{% code-block-end %}

Let’s create our database and run the migrations:

{% code-block language="js" %}
rails db:create
rails db:migrate
{% code-block-end %}

Since we are going to follow the test driven approach, let’s write the model specs for category and book first.

{% code-block language="js" %}
# spec/models/category_spec.rb
RSpec.describe Category, type: :model do
 # Association test
 it { should have_many(:books) }
 # Validation tests
 it { should validate_presence_of(:name) }
 it {
   should validate_length_of(:name)
     .is_at_least(3)
 }
end
# spec/models/book_spec.rb
RSpec.describe Book, type: :model do
   # Association test
   it { should belong_to(:category) }
    # Validation tests
   it { should validate_presence_of(:title) }
   it { should validate_presence_of(:author) }
   it {
     should validate_length_of(:title)
       .is_at_least(3)   }
end
{% code-block-end %}

The expressive Domain Specific Language (DSL) of RSpec makes it possible to read the tests just like a paragraph. The shoulda-matchers gem we added earlier provides RSpec stylish association and validation matchers.

Now, try executing the specs by running:

{% code-block language="js" %}
bundle exec rspec
{% code-block-end %}

Just as expected, only one test passed and six failed. 

Let’s go ahead and fix the failures:

{% code-block language="js" %}
# app/models/category.rb
class Category < ApplicationRecord
 has_many :books
 validates :name, presence: true, length: { minimum: 3 }
end
# app/models/book.rb
class Book < ApplicationRecord
 belongs_to :category
 validates :title, :author, presence: true, length: { minimum: 3 }
end
{% code-block-end %}

Good job! Let’s run the tests again. And voila - all green!

Controllers

Our models are ready for this part, so let’s go ahead and generate the controllers.

{% code-block language="js" %}
rails g controller Categories
rails g controller Books
{% code-block-end %}

Yes, your thoughts are right, test first… If you noticed, while generating the controllers spec/requests/categories_request_spec.rb and spec/requests/books_request_spec.rb were automatically generated for us.

The request specs are designed to drive behavior through the full stack, including routing. This means they can hit the applications. Being able to test our HTTP endpoints is exactly the kind of behavior we want from our tests.

Before we proceed, let's add the model factories which will provide the test data.

{% code-block language="js" %}
touch spec/factories/{category.rb,book.rb}
{% code-block-end %}‍

Defining the Factories

{% code-block language="js" %}
#spec/factories/category
FactoryBot.define do
 factory :category do
  name { Faker::Book.genre }
 end
end
#spec/factories/book.rb
FactoryBot.define do
 factory :book do
      title { Faker::Book.title }
   author { Faker::Book.author }
   category { create(:category) }
 end
end
{% code-block-end %}

Now, we will create a custom helper method json, to parse the JSON response to Ruby Hash. This is easier to work with in our tests.

Let’s define it in spec/support/request_spec_helper

{% code-block language="js" %}
mkdir spec/support && touch spec/support/request_spec_helper.rb
# spec/support/request_spec_helper.rb
module RequestSpecHelper
 def json
   JSON.parse(response.body)
 end
end
{% code-block-end %}

The support directory is not autoloaded by default so let’s go ahead and enable this inside the spec/rails_helper.rb:

{% code-block language="js" %}
# spec/rails_helper.rb
# [...]
Dir[Rails.root.join('spec/support/**/*.rb')].each { |f| require f }
# [...]
RSpec.configuration do |config|
  # [...]
  config.include RequestSpecHelper, type: :request
  # [...]
end
{% code-block-end %}

Next, write the specs for category API:

{% code-block language="js" %}
# spec/requests/categories_request_spec.rb
RSpec.describe 'Categories', type: :request do
 # initialize test data
 let!(:categories) { create_list(:category, 5) }
 let!(:category_id) { categories.first.id }
 # Test suite for GET /category
 describe 'GET /categories' do
   # make HTTP get request before each example
   before { get '/api/v1/categories' }
   it 'returns categories' do
     expect(json).not_to be_empty
     expect(json.size).to eq(5)
   end
   it 'returns status code 200' do
     expect(response).to have_http_status(200)
   end
 end
 # Test suite for POST /category
 describe 'POST /category' do
   # valid payload
   let(:valid_name) { { name: 'Horror' } }
   context 'when the request is valid' do
     before { post '/api/v1/categories', params: valid_name }
     it 'creates a category' do
       expect(json['name']).to eq('Horror')
     end
     it 'returns status code 201' do
       expect(response).to have_http_status(201)
     end
   end
   context 'when the request is invalid' do
     before { post '/api/v1/categories', params: { name: '' } }
     it 'returns status code 422' do
       expect(response).to have_http_status(422)
     end
     it 'returns a validation failure message' do
       expect(response.body)
      .to include("is too short (minimum is 3 characters)")
     end
   end
 end
 # Test suite for DELETE /category/:id
 describe 'DELETE /categories/:id' do
   before { delete "/api/v1/categories/#{category_id}" }
   it 'returns status code 204' do
     expect(response).to have_http_status(204)
   end
 end
end
{% code-block-end %}

Thanks to factory bot, we started by populating the database with a list of five categories. 

Now, go ahead and run the test. As usual we are getting a failing test and all issues are routing errors. It’s obvious we haven’t defined the routes yet.

Defining Routes

Let’s go ahead and define them in config/routes.rb

{% code-block language="js" %}
#config/routes
Rails.application.routes.draw do
 namespace :api do
   namespace :v1 do
     resources :categories, only: %i[index create destroy]
   end
 end
end
{% code-block-end %}

If you noticed, our route for categories is wrapped with namespace :api and :v1. That is because it’s usually  good practice to begin versioning your API from the start. This means our URL will be patterned as /api/v1/EndPoint. Going forward we will do the same for our controllers.

Run the tests again and you will notice that the routing error is gone. 

Now we have an ActionController error with uninitialized constant API. Before we define our controller methods let’s go ahead and create a directory to match the api/v1 namespace.

{% code-block language="js" %}
mkdir app/controllers/api && mkdir app/controllers/api/v1
{% code-block-end %}

Now go ahead and move the books_controller.rb and categories_controller.rb, then wrap you class with module API and V1

{% code-block language="js" %}
module Api
   module V1
     class CategoriesController < ApplicationController
     end
   end
end
{% code-block-end %}

When we run the tests again, we see that the uninitialized constant API is gone, and now we have controller failures to deal with. Let’s go ahead and define the controller methods.

{% code-block language="js" %}
# app/controllers/api/v1/categories_controller.rb
module Api
 module V1
   class CategoriesController < ApplicationController
     before_action :set_category, only: :destroy
     # GET /categories
     def index
       @categories = Category.all
       render json: CategoriesRepresenter.new(@categories).as_json
     end
     # POST /category
     def create
       @category = Category.create(category_params)
       if @category.save
         render json: CategoryRepresenter.new(@category).as_json, status: :created
       else
         render json: @category.errors, status: :unprocessable_entity
       end
     end
     # DELETE /categories/:id
     def destroy
       @category.destroy
       head :no_content
     end
     private
     def category_params
       params.permit(:name)
     end
     def set_category
       @category = Category.find(params[:id])
     end
   end
 end
end
{% code-block-end %}

You might have noticed CategoryRepresenter and CategoriesRepresenter. This is our own custom helper that we will use to render the json response just the way we want it. Let’s go ahead and create it.

{% code-block language="js" %}
# app/representers/category_representer.rb
class CategoryRepresenter
 def initialize(category)
   @category = category
 end
 def as_json
   {
     id: category.id,
     name: category.name
   }
 end
 private
 attr_reader :category
end
# app/representers/categories_representer.rb
class CategoriesRepresenter
 def initialize(categories)
   @categories = categories
 end
 def as_json
   categories.map do |category|
     {
       id: category.id,
       name: category.name
     }
   end
 end
 private
 attr_reader :categories
end
{% code-block-end %}

Before we proceed, let’s add a few exception handlers to our application.

{% code-block language="js" %}
# app/controllers/concerns/response.rb
module Response
 def json_response(object, status = :ok)
   render json: object, status: status
 end
end
# app/controllers/concerns/exception_handler.rb
module ExceptionHandler
 extend ActiveSupport::Concern
 included do
   rescue_from ActiveRecord::RecordNotFound do |e|
     json_response({ error: e.message }, :not_found)
   end
   rescue_from ActiveRecord::RecordInvalid do |e|
     json_response({ error: e.message }, :unprocessable_entity)
   end
   rescue_from ActiveRecord::RecordNotDestroyed do |e|
     json_response({ errors: e.record.errors }, :unprocessable_entity)
   end
 end
end
{% code-block-end %}

Let’s include the exceptions in our application_controller.rb.

{% code-block language="js" %}
class ApplicationController < ActionController::API
 include Response
 include ExceptionHandler
end
{% code-block-end %}

Run the test again, you can simply use rspec to run all tests or rspec spec/requests/categories_request_spec.rb to run only the categories request specs to make sure all is green.

Now that we are done with the categories API, let's create the request specs for our books:

{% code-block language="js" %}
# spec/requests/books_request_spec.rb
RSpec.describe 'Books', type: :request do
 # initialize test data
 let!(:books) { create_list(:book, 10) }
 let(:book_id) { books.first.id }
 describe 'GET /books' do
   before { get '/api/v1/books' }
   it 'returns books' do
     expect(json).not_to be_empty
     expect(json.size).to eq(10)
   end
   it 'returns status code 200' do
     expect(response).to have_http_status(200)
   end
 end
 describe 'GET /books/:id' do
   before { get "/api/v1/books/#{book_id}" }
   context 'when book exists' do
     it 'returns status code 200' do
       expect(response).to have_http_status(200)
     end
     it 'returns the book item' do
       expect(json['id']).to eq(book_id)
     end
   end
   context 'when book does not exist' do
     let(:book_id) { 0 }
     it 'returns status code 404' do
       expect(response).to have_http_status(404)
     end
     it 'returns a not found message' do
       expect(response.body).to include("Couldn't find Book with 'id'=0")
     end
   end
 end
 describe 'POST /books/:id' do
   let!(:history) { create(:category) }
   let(:valid_attributes) do
     { title: 'Whispers of Time', author: 'Dr. Krishna Saksena',
       category_id: history.id }
   end
   context 'when request attributes are valid' do
     before { post '/api/v1/books', params: valid_attributes }
     it 'returns status code 201' do
       expect(response).to have_http_status(201)
     end
   end
   context 'when an invalid request' do
     before { post '/api/v1/books', params: {} }
     it 'returns status code 422' do
       expect(response).to have_http_status(422)
     end
     it 'returns a failure message' do
       expect(response.body).to include("can't be blank")
     end
   end
 end
 describe 'PUT /books/:id' do
   let(:valid_attributes) { { title: 'Saffron Swords' } }
   before { put "/api/v1/books/#{book_id}", params: valid_attributes }
   context 'when book exists' do
     it 'returns status code 204' do
       expect(response).to have_http_status(204)
     end
     it 'updates the book' do
       updated_item = Book.find(book_id)
       expect(updated_item.title).to match(/Saffron Swords/)
     end
   end
   context 'when the book does not exist' do
     let(:book_id) { 0 }
     it 'returns status code 404' do
       expect(response).to have_http_status(404)
     end
     it 'returns a not found message' do
       expect(response.body).to include("Couldn't find Book with 'id'=0")
     end
   end
 end
 describe 'DELETE /books/:id' do
   before { delete "/api/v1/books/#{book_id}" }
   it 'returns status code 204' do
     expect(response).to have_http_status(204)
   end
 end
end
{% code-block-end %}

As expected, running the tests at this point should output failing books tests. Be sure they are all ActionController::RoutingError:. 

It’s again obvious we haven’t defined the routes yet. Let’s go ahead and define them in config/routes.rb. Our routes.rb file should now look like this:

{% code-block language="js" %}
Rails.application.routes.draw do
 namespace :api do
   namespace :v1 do
     resources :categories, only: %i[index create destroy]
     resources :books, only: %i[index create show update destroy]
   end
 end
end
{% code-block-end %}

Now that we are on the same page, let’s define the actions inside our books_controller.

{% code-block language="js" %}
# app/controllers/api/v1/books_controller.rb
module Api
 module V1
   class BooksController < ApplicationController
     before_action :set_book, only: %i[update show destroy]
     # GET /books
     def index
       @books = Book.all
       render json: BooksRepresenter.new(@books).as_json
     end
     # POST /book
     def create
       @book = Book.create(book_params)
       if @book.save
         render json: BookRepresenter.new(@book).as_json, status: :created
       else
         render json: @book.errors, status: :unprocessable_entity
       end
     end
     # GET /books/:id
     def show
       render json: BookRepresenter.new(@book).as_json
     end
     # PUT /books/:id
     def update
       @book.update(book_params)
       head :no_content
     end
     # DELETE /books/:id
     def destroy
       @book.destroy
       head :no_content
     end
     private
     def book_params
       params.permit(:title, :author, :category_id)
     end
     def set_book
       @book = Book.find(params[:id])
     end
   end
 end
end
{% code-block-end %}

Just like CategoryRepresenter and CategoriesRepresenter, let’s go ahead and create the BookRepresenter and BooksRepresenter class.

{% code-block language="js" %}
touch app/representers/{book_representer.rb,books_representer.rb}
# app/representers/book_representer.rb
class BookRepresenter
 def initialize(book)
   @book = book
 end
 def as_json
   {     id: book.id,
     title: book.title,
     author: book.author,
     category: Category.find(book.id).name,
     date_added: book.created_at
   }
 end
 private
 attr_reader :book
end
# app/representers/books_representer.rb
class BooksRepresenter
 def initialize(books)
   @books = books
 end def as_json
   books.map do |book|
     {
       id: book.id,
       title: book.title,
       author: book.author,
       category: Category.find(book.id).name,
       date_added: book.created_at
     }
   end
 end
 private
 attr_reader :books
end
{% code-block-end %}

Run the tests again. At this point everything should be green. Phew! That was a long one. But it was worth it.

Instead of the …….. indicating all the passed tests cases, we will add --format documentation to your .rspec file in the root of your document.

{% code-block language="js" %}
# .rspec
--require spec_helper
--format documentation
{% code-block-end %}

Run the tests again to see the results. 

Manual Testing

Apart from writing specs for our endpoints, it’s usually fulfilling if we can manually test our endpoints. There are tons of API testing softwares out there that allows us to determine if our API meets its expectations. Vyom Srivastava in his article “Top 25+ API testing Tools” outlined 25 API testing tools, and the pros and cons.

Postman remains my personal preference for running manual tests on API. It simplifies each step of building an API, as well as generating documentation easily.

Conclusion

If you want a step by step walk through of this tutorial, I've created a video here:

TDD does slow down your development time and it will take some time before we get used to test-driven development. Andrea in his article, “Test Driven Development: what it is, and what it is not” describes the three rules of TDD as: 

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
   

With this approach TDD helps developers get a clearer look at the requirements before actually developing the feature, since the test is a reflection of the requirements. Writing tests first will make developers think of every nook and cranny before writing code.

After reading this, you should feel comfortable: 

• Generating an API application with Rails 6.
• Setting up RSpec testing framework alongside gems like FactoryBot, Database Cleaner, Shoulda-Matchers and Faker to achieve a smooth test.
• Using a Test Driven Development approach to build models and controllers.
• API Versioning.

You can find the full code for everything we went over here. We cover how to build a RESTful API authentication with JWT in this next article. Happy Coding!