Ruby on Rails: Why Upgrading Matters – Part 1

Start Free Trial
May 5, 2020 by , and Updated July 24th, 2020

Ruby on Rails (or Rails) is a web development  framework that gives Rails developers an optimized experience to write their (Ruby) code. Rails is one of many web frameworks in the world of app programming and web development. These frameworks are collections of code libraries that give app and web developers off-the-shelf solutions for time consuming, repetitive tasks—such as building menus, tables, or forms on a website.

Because Rails is a web development framework all security aspects around it need to be taken very seriously.  This creates the need to be on (or near) the latest release in order to take advantage of the most updated functionality and of course, the latest security patches.

Why did we upgrade Rails?

Our app was running on an older Rails version which meant:

  • Security support had ended. This meant that the existing Rails version was no longer receiving updates for bug fixes for critical security issues.
  • We could not directly upgrade to the latest ruby version — here is why.
  • Newer features from newer Rails versions were not available
  • We  could not use the latest gems versions. Older gems also lacked functionality and suffered from security flaws.

How we started the upgrade journey:

Whenever you’re upgrading a software, the most critical points are:

  • Upgrading without blocking feature development
  • Affecting as fewer stakeholders as possible
  • Making sure changes are backward compatible

To fulfil all the above critical points we did the upgrade process as following:

Plan: 

Our aim was to upgrade our existing application to the latest stable version.

Upgrading directly from the current version to the desired stable version is not recommended and also it would be a huge change for us to make in one go. So, we decided to first upgrade Rails to the next available higher version, and then subsequently upgrade to the latest desired Rails version.

So, we followed a staggered upgrade approach where we updated Rails to the desired version in 2 steps.

We started by first doing a small POC by changing rails and many other gem versions and checking if the application is booting properly. There were many deprecations and changes needed for the POC and we did all that in a week.

After a week, we were able to start the rails console and server.

Here is a list of critical changes that we did when upgrading Rails:

  • All views files, especially partials should be named with underscores and not with hyphens.
  • Factorygirl gem has to be updated for RSpec tests and the new version has syntax change for defining factories. Use Factory.define instead of FactoryGirl.define.
  • In models, while defining scopes or relationships, use Procs instead of where conditions.
  • Routes should be written with proper HTTP methods i.e. GET, POST, etc. instead of MATCH.
  • Use attr_accessor or decorators for assigning non-column attributes to an ActiveRecord object to avoid MissingAttributeError exceptions.

You can also find the critical changes in different Rails versions here.

Dual boot:

We made changes in our Gemfile to make the app dual boot. So that the current development work can be done simultaneously with our upgrade work. This also helped us during rollout because it acts as a single switch to change the app’s rails version.

What we did for dual boot:

  • Defined a kernel method named rails_next? method in gemfile so that new Rails version specific changes can be isolated under it.
  • This was an indispensable part for the Rails upgrade as all the changes like bundle install, rails console, rails server and rake assets:precompile can work independent of Rails version.
  • We took help from this article
  • We created a different gemfile.lock for the new Rails version and named it as Gemfile_next.lock. This file was used to keep the gem versions for the newer Rails version.

Gems compatibility: 

We tried to upgrade the gems which were incompatible with newer Rails versions so that it can work both in Rails versions. But approx 23 gems couldn’t be upgraded for the newer Rails version. Some of them were critical to the app like devise, activeadmin, etc.

These gems came under the rails_next? method we had defined in the gemfile.

We added some more gems for compatibility.

Green unit tests: 

You have to depend on unit tests to check if things are working fine. We had around 3.5k tests failing out of 6k tests. We had to incorporate changes in such a way that the unit tests should work with both Rails versions.

We followed the following steps to achieve that:

  • Upgraded rspec to newer version
  • Identify the issue in the failed unit test; Fix; Repeat. (Across full code base)
  • Added a bitbucket pipeline which ran tests in both Rails versions for every Pull-Request. (First we added a pipeline for new Rails version bootup and then for unit tests along with existing Rails version unit tests)

The biggest challenge for rollout:
The biggest challenge that we faced was the cache incompatibility between Rails version and between 2 releases.

Environment Cache setup:
We have around 5 major prod environments and around 3 staging/internal envs.

We use memcached and redis as application cache in these environments.

The Problem:
We had multiple instances in code where we were caching objects directly in the cache(eg. ActiveRecord objects).
Due to this, in the newer Rails version, we were unable to read the cache written via older Rails version and vice versa.

Also, since we follow a blue-green deployment there was a difference in the code in the 2 releases which needed to be compatible to avoid any issues.

There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

The First Solution (Using Openstruct object in place of Rails objects):

We decided to use Openstruct in place of Rails objects since it’s a Ruby class and available in both Rails versions and is accessible with both dot and bracket notation. We also monkey patched the dalli gem to handle this incompatibility. The fix used to invalidate the old cache from the older Rails version  and replace it with the new cache.

For redis, we thought of clearing the old cache and let the new cache be created since the prod env cache was very small.

The patch had changes to handle NameError and in check if the object retrieved from memcached can be converted to an object which Rails understands.

With the following changes, we had handled the following scenarios for production environments since both the releases will be serving traffic while accounts transition from one release to another:

Scenario 1 : Cache value is the older Rails version like active record:

  1. Reading from existing Rails version will work fine in both releases
  2. Reading from the newer Rails version will give nil which will result in creating the cache again. This will force the app to override the cache value with the newer ones (i.e. with OpenStruct).

Scenario 2 : Cache value is newer Rails version. OpenStruct object:

  1. Reading from the existing Rails version will work fine in both releases as we cache the OpenStruct object which is accessible with both dot and bracket notation.
  2. Reading from the newer Rails version will work fine.

All was good and our changes went well in 3 smaller environments. We were ecstatic until we hit a performance issue in the second biggest production environment.

The command execution delay had increased after Rails upgrade. We found that the execution delay has increased from 3-3.5 to 4.5-5s in all the environments where we have done Rails upgrade.

Cause of delay :

OpenStruct was being used to wrap the ActiveRecord object in the application cache.

OpenStruct uses Ruby’s method_missing and define_method to simulate “normal” objects and hence is slower. Hence the deserialization of an OpenStruct object takes time.

Prior to the newer Rails upgrade changes, we were reading and writing Rail’s ActiveRecord objects directly into the cache. To tackle this we had made some changes.

Some of them were easier like just converting into Ruby hash while others were trickier since the retrieved cache object(s) were subsequently being used with dot notation.

To fix it with minimal code change, we wrapped the Hash(of the AR object) in OpenStruct since both bracket and dot notation are supported in OpenStruct.

That said, there is a performance degradation in using OpenStruct.

This was profusely being used being in feature’s check which is done numerous times throughout the whole app. This was the cause of this delay.

Following are some of the numbers to verify the same:

Benchmarking with respect to Feature cache retrieval:

Env: QA_ENV_1
cache_store: memecached
cache_key = "dummy_account_feature_cache_key"
 
#This cache key's value returns a hash of 35 items where the key is the feature_name and value is a hash of db records wrapped in OpenStruct.
 
# latency if the cached Object is OpenStruct
latency_openStruct = Benchmark.realtime do
Rails.cache.read(cache_key)
end * 1000
12.446482 ms
 
# latency if the cached Object is ActiveRecord
latency_with_activeRecord = Benchmark.realtime do
 Rails.cache.read(cache_key)
end * 1000
8.524003 ms
 
# latency if the cached Object is a Ruby hash
latency_hash =Benchmark.realtime do
Rails.cache.read(cache_key)
end * 1000
5.614253 ms

General Benchmarking:

require 'ostruct'
require 'benchmark'
 
COUNT = 100_000
NAME = "Qubole Test"
EMAIL = "[email protected]"
RANK = 100
 
class Person
  attr_accessor :name, :email, :rank
end
 
Benchmark.bm(13) do |obj|
  obj.report("Ruby Hash:") do
    COUNT.times do
      p = {name: NAME, email: EMAIL, rank: RANK}
    end
  end
 
  obj.report("OpenStruct:") do
    COUNT.times do
     p = OpenStruct.new({name: NAME, email: EMAIL, rank: RANK})
   end
  end
 
  obj.report("Ruby Class:") do
    COUNT.times do
      p = Person.new
      p.name = NAME
      p.email = EMAIL
      p.rank = RANK
    end
  end
end

Results:

                    user     system      total        real
Ruby Hash:      0.060000   0.000000   0.060000 (  0.062216)
OpenStruct:     1.620000   0.010000   1.630000 (  1.643002)
Ruby Class:     0.030000   0.000000   0.030000 (  0.024538)

The Solution that worked (Using CustomStruct object in place of Rails objects):

We implemented our own implementation of OpenStruct with a simple Class(CustomStruct) whose object can be accessed with both dot and bracket notation and replaced all the instances of OpenStruct.

Following is our implementation of our custom class CustomStruct :

Benchmarking (CustomStruct vs ActiveRecord vs OpenStruct):

Measurement unit: milliseconds

Note: Our application cache is Read Heavy

MEMCACHED (QA_ENV_1)

  1. CustomStruct (new implementation)
cache_key = "cache:Benchmark:test1"
Benchmark.realtime do
    type_features = Rails.cache.fetch(cache_key) do
    type_features = Hash.new
    Feature.last(1000).each do |type_feature|
      type_features[type_feature.name] = CustomStruct.new(type_feature.attributes)
    end
    type_features
  end
end
 
write : ~130
 read : ~7.1
  1. ActiveRecord (previous implementation)
cache_key2 = "cache:Benchmark:test2"
Benchmark.realtime do
    type_features = Rails.cache.fetch(cache_key2) do
    type_features = Hash.new
    Feature.last(1000).each do |type_feature|
      type_features[type_feature.name] = type_feature
    end
    type_features
  end
end
 
write : ~51
 read : ~12.5
  1. OpenStruct (current implementation)
cache_key3 = "cache:Benchmark:test3"
  Benchmark.realtime do
      type_features = Rails.cache.fetch(cache_key3) do
      type_features = Hash.new
      Feature.last(1000).each do |type_feature|
        type_features[type_feature.name] = OpenStruct.new(type_feature.attributes)
      end
      type_features
    end
  end
 
write : ~218
 read : ~21

REDIS (QA_ENV_2)

  1. CustomStruct (new implementation)
cache_key1 = "cache:redis:Benchmark:test1"
Benchmark.realtime do
    type_features = Rails.cache.fetch(cache_key1) do
    type_features = Hash.new
    Feature.last(1000).each do |type_feature|
      type_features[type_feature.name] = CustomStruct.new(type_feature.attributes)
    end
    type_features
  end
end * 1000
 
write : ~122
 read : ~7.5
  1. ActiveRecord (previous implementation)
cache_key2 = "cache:redis:Benchmark:test2"
Benchmark.realtime do
   type_features = Rails.cache.fetch(cache_key2) do
   type_features = Hash.new
   Feature.last(1000).each do |type_feature|
     type_features[type_feature.name] = type_feature
   end
   type_features
 end
end * 1000
 
write : ~52
 read : ~8.4
  1. OpenStruct (current implementation)
cache_key3 = "cache:redis:Benchmark:test3"
  Benchmark.realtime do
      type_features = Rails.cache.fetch(cache_key3) do
      type_features = Hash.new
      Feature.last(1000).each do |type_feature|
        type_features[type_feature.name] = OpenStruct.new(type_feature.attributes)
      end
      type_features
    end
  end * 1000
 
write : ~270
 read : ~21.7

Result: We could see from the benchmarks that CustomStruct was way faster than the OpenStruct which gave us the requisite performance.

Some other important tasks:

  • Add Rails version tag and request-id for better debugibility. This was very essential for QA, support and other devs so that they can deterministically debug any issue.
  • Make appropriate changes so that devs can easily bring up their dev environment in any of the Rails versions without fail. This was necessary for us to test any changes that we do in both the versions so that the code was backward compatible.
  • Fix custom code that was strictly adhering to existing Rails version’s APIs.

We will publish part 2 shortly. Watch this space for more.

Start Free Trial
  • Blog Subscription

    Get the latest updates on all things big data.
  • Categories

  • Events

    Is your data architecture modern enough to go the distance?

    Aug. 14, 2020 | Virtual Event

    Challenged to maximise AWS Spot utilization and minimize job loss?

    Aug. 18, 2020 | Virtual Event

    How to drop your cloud data lake costs by 50% in 3 months

    Aug. 27, 2020 | Virtual Event

    AWS re:Invent

    Nov. 30, 2020 | Las Vegas, NV