How we're migrating a legacy Rails 3 app to Rails 7

Posted on: 6 June 2024

At Arena Online, our plan this year is to begin migrating our existing legacy codebase off Rails 3 and into a modern Rails 7 app; simultaneously shedding over a decade’s worth of technical debt.

A bit of history

Rails itself is fairly easily upgraded between major versions, with extensive docs and walk-throughs available to help. However, the majority of our code is contained in a series of Rails 3 apps which pre-date even the Git repository they’re contained in. They began life as original Rails 1 apps. Various upgrades over the years took it to Rails 3 but for one reason or another, long before my time, it got unceremoniously stuck in version 3; branded “un-upgradable”.

Various speculative attempts to upgrade to Rails 4 have been made over the years, but it has now largely succumbed to its legacy routes. New apps have been spawned, taking some logic away from the codebase but still a large number of business-critical systems still live within.

Upgrade or rewrite

No app is truly un-upgradeable, given enough time and resource. But in the real world there’s a valid question of which is more efficient: upgrade or a rewrite. Ultimately, we decided on an incremental rewrite.

Incremental being the key word here, as we fully intend to continue business-as-usual operations on the old codebase and architecture, slowly migrating services away as they’re built.

Where to start

The key thing with a monolith app, or in our case, a series of apps in a single repository, is to figure out how to slice it up into manageable units. These should be discrete and functional in isolation. Allowing a seamless switch-over from the old system to the new for that individual unit.

For us, we have a series of “workers” (more “jobs” in todays terminology), which run as background tasks, fulfilling an action on a batch of - typically - orders. These worker classes belong to our CMS/backend application, but are run by Resque, in a separate environment, so don’t have any front end (besides a generic, minimal UI provided by Resque). They are scheduled, queued using a Redis backend, and performed by Resque. Jobs like this are a good candidate for extraction.

A single endpoint in an API-style app could also work as a single unit of code to start with.

Initialising the new app

Starting is very easy, especially if you’ve been used to the relative weeds of navigating a legacy Rails 3 codebase. Start from the latest version of Rails and use the command line tool to generate a brand new app.

The key configuration you will need to set up is access to the legacy database. You may want this as your primary database, however I chose to define a separate “legacy” connection in my database configuration. This left a primary connection, using a database adaptor of our choosing, for if we wanted to use something like Solid Queue down the line.

You will probably have your legacy database running locally already if you’re actively working on the current codebase. For me, I had this available in a Docker container, and just needed to expose the port in the Docker Compose file so I could access it outside of the app’s network.

Configuring an additional database connection is pretty simple in Rails 7. I created a LegacyRecord model, which inherits from the primary ApplicationRecord class, is defined as abstract, and connects to the “legacy” database configuration for reading and writing:

class LegacyRecord < ApplicationRecord
  self.abstract_class = true

  connects_to database: { reading: :legacy, writing: :legacy }
end

Then, in your database configuration YAML file, you’ll need to ensure a “legacy” configuration under the development and test environments:

legacy: &legacy
  adapter: mysql2
  checkout_timeout: 30
  username: <%= ENV['DB_USER'] || 'root' %>
  password: <%= ENV['DB_PASSWORD'] || '' %>
  host: <%= ENV['DB_HOST'] || 'localhost' %>

development:
  legacy:
    <<: *legacy
    database: arena_development

test:
  legacy:
    <<: *legacy
    database: arena_test

I then exported environment variables in my .zshrc file that match the details needed to connect to my local Docker mysql instance. Later on, we’ll define a “primary” database configuration under development and test, but for now, this should work fine.

The useful thing is, as soon as Rails connects to your test database, it will automatically generate a db/legacy_schema.rb. This should largely match the db/schema.rb file in your current legacy codebase.

The first test

My approach to the migration is test driven development. Hopefully you’ll be migrating a codebase with extensive test coverage. Migrating becomes significantly more haphazard without the backing of a good test suite.

With this in mind, you’ll want to first identify the tests that currently cover the code you want to begin migrating. I would suggest copying these files over as-is. Fortunately, from Rails 3 to 7, the makeup of a unit test has not changed much, so Rails was able to understand and at least try to run our existing tests.

Running the test suite (rails test ) at this point, there’s a strong likelihood that it will fail, and fail early. For us, we were referencing the pre-renamed version of the FactoryBot gem, FactoryGirl. At this point we’ve not even included the gem in our Gemfile, but if we had, it would fail due to the old class reference.

The key thing at this stage, and indeed with Test Driven Development in general, is to do the smallest amount of work to fix the current error preventing the test from passing. In this small example, that is specifying the factory_bot_rails gem in our Gemfile and doing a bulk-find and replace in our code editor from FactoryGirl to FactoryBot. This should allow us to re-run our tests and attack the next error.

This test-and-see approach can be time-consuming, but with each fix you’ll be getting closer to where you want to get to - a passing test suite again. You’ll soon come across an actual logic fail in your test. It will attempt to load a class or factory which doesn’t yet exist in the new codebase. Resist the urge to copy and paste this file from old to new.

Shed the deadwood

In our codebase we had a mammoth Order model; a God object, as it’s commonly referred to. Somewhat of a dumping ground over the years. It’s also split across a number of included modules. It’s a lot of code. The last thing we want to do is ship all that technical debt with us to the new codebase. And crucially, we probably don’t know what’s useful and what isn’t. So let your tests guide you.

If a referenced class cannot be found, create it in the new codebase. If a method cannot be found, create it. Add associations & constants when referenced. Although I recommend avoiding copying entire classes, copying over whole methods is generally fine. Methods are usually more focussed and clearer to understand what is useful and what is not.

It’s important at this stage not to try to change too much about the old code. The focus should be on minimal fixes to get the old test suite compatible with the new codebase. You can update old syntax (like .try() for &. and :key ⇒ for key: in hashes) but do not think about significant refactoring at this stage.

Move fast

Getting a passing test suite can be tricky at times. I found it was important not to get held up for too long on this. If a particular test case tested a part of the system I didn’t intend to migrate at this stage, I would simply skip it. This is like putting a post-it on the page to come back to down the line.

If you don’t want to skip the whole test case, don’t be precious about commenting out irrelevant assertions, or use an inline skip block. If it tests areas of the codebase you’re not focussing on right now, just comment it out. But be considerate, leave a note.

It’s even fine at this stage to comment out application code if you deem it irrelevant. For example, I didn’t want to deal with logging to Rollbar, and I knew this would be an easy fix down the line. So simply commenting out this code was not a big deal.

An approach for minimising the visual stream of errors and test failures at first is to --fail-fast. This is a flag you can utilise when running rails test that will stop running as soon as a failure or error is encountered. I found this useful at first when I knew every test was likely to fail, for the same or similar reasons.

Required: good test coverage

Eventually you should end up with a passing test suite, and a collection of minimal, logic-light, related classes. These are classes are far from being production-ready, but you can be reasonably confident they perform the same function as the particular tests expected them to.

It’s important to note, this process is only as good as the coverage the tests provide. If your tests are light and don’t cover the necessary number of scenarios that occur in your application, there’s a chance your new code will be missing crucial bits of the old codebase. But if you’re migrating a rock-solid, battle-hardened codebase that has stood the test of time, there’s a good chance you have a good test suite backing that up.

Forgotten bits

A passing test suite is great, but it’s no substitute for a good, hands-on knowledge of the system you’re migrating. In my case, I knew the Order model was rife with callbacks and validations split over multiple files. This kind of logic may be either not tested completely enough or is covered by some other test file you’re not ready to copy over yet. But some may still be part of the core functionality you’re recreating.

For example, our Order model queues callbacks when its status changes. It also logs a version change when its status or a key attribute changes. At this point you can either put a pin in this functionality, in the form of a ticket, or recreate the logic in the new codebase.

Don’t be tempted to copy and paste modules or old gems that did the heavy lifting. For starters, old gems are unlikely to be compatible with modern Rails versions without some work. And secondly, you want to properly vet all new gems you add to a codebase. Every gem in the Gemfile is a responsibility you must maintain.

Refactor later

I mentioned earlier about resisting the urge to refactor. This step should be secondary to the migration process itself. Your key objective is to get the old codebase working in the new environment. However, there is some nuance here.

Dodgy code patterns and clunky or sub-optimal code will need fixing at some point. You don’t want to bring bad habits over to a new codebase if you can help it. So make a clear note in the code, or better yet, create a ticket/issue detailing the problem with the code.

The primary reason I found for delaying refactoring is it will inevitably get lost in the initial code review. When migrating code, you will end up with a lot of code changes for review. These changes are easier to review if the developers reviewing know they’re only looking at existing code.

A small, focussed Pull Request after the fact has significantly less cognitive overhead than having it jumbled up with swaths of old logic. A distinct PR also provides a much better platform for focussed discussion on the refactor.

Make use of fixtures

There’s a hot debate around factories vs fixtures, but I believe they both have their place in an application. The codebase we’re migrating makes use of factories, but fixtures is now the default in modern Rails apps, so let’s see if we can make use of them.

Because we’re dealing with a legacy application in production, we know the nature of the data the application deals with. Data like this has been around for a while and doesn’t change much. For these reasons, it’s a great candidate for fixtures.

Converting data of this nature from factories to fixtures will provide a decent performance upgrade when running an entire test suite, and will clean up your test setup.

Modernise your code

Use tools like Rubocop to auto-lint your code based on a set of rules you control. You can choose to adopt the rules set by established companies like Shopify and Airbnb too. Rubocop has a plugin for Rails specifically which will autocorrect common anti-patterns in your code and simplify where possible.

Writing code has come a long way in the last decade, so chances are, your old codebase could do with some sprucing up. However, as with refactoring, always make these changes as part of a separate Pull Request. It’s a lot easier to review code when you know you’re only looking for syntax and code style changes, rather than logic.

In conclusion

We’re still very early on in the process. We’re 3 “units” into the migration and are currently now looking at setting up a new environment in which to begin manual testing of the new code. Once we’re happy the code fulfills the same role as it did in the existing codebase, we can look at switching off that code in the old system and switching on in the new.

This is an exciting project and one we expect will continue well into 2025. But hopefully by the end, we’ll have a much more robust, modern base in which to build new functionality into.