Simple DIY database translations in Ruby on Rails

Ruby on Rails has really good static translation support through I18n but if you need to store your dynamic content in multiple locale versions there’s no built-in support in Rails. So what do you do? As always it depends on your use case. One way is to go with the gem globalize. It’s been around for a long time, has a bunch of features and can be dropped into existing projects without even knowing where you’re reading locale-dependent data.

Sounds brilliant, all the features, run a few generators and my giant monolith is all of a sudden internationalized, thanks for the tip!

While this can be true if you’re lucky we’ve found that while it might look all good for a while, edge cases quickly become apparent. We like to compare the trade-offs to that of any of the acts as paranoid gems that allow you to soft delete your ActiveRecord models. Much like the paranoia gems monkey patch destroy and ActiveRecord finder methods gems like globalize also patch your finder method as well as the model attribute accessors.

It’s tempting to adapt to a new default of not returning soft deleted records as it’s probably what you want most of the time. In our experience it’s not adding an extra .undeleted scope that takes time however. Rather it’s the more complex scenarios, e.g. when chaining multiple scopes together or when introducing new developers to your project that tends to be the big time consumers.

The same is true with translations, what looks innocent generates join queries and changes behaviour depending on your current state of I18n.locale.

When you put it like that… So what’s the alternative?

There are a number of much smaller gems that use the database’s json-column support to store translations in the same table. This removes the need for implicit join queries but retains the global state dependency on I18n.locale. As an effort to demystify database internationalization we decided to roll our own implementation. We wanted to try an API that was easy to reason about and contained as few surprises as possible.

We started with creating a model with a postgres jsonb field:

class CreatePosts < ActiveRecord::Migration[5.2]
  def change
    create_table :posts do |t|
      t.jsonb :title, :jsonb, null: false
      t.timestamps
    end

    execute "CREATE INDEX posts_title_index ON posts USING gin (title)"
  end
end

Without needing to do anything else you already you have a working API for translating your model:

post = Post.create(title: { en: "News", sv: "Nyheter" })

post.title # => { en: "News", sv: "Nyheter" }
post.title["en"] # => "News"
post.title["de"] # => nil
post.title.to_json # => "{\"en\":\"News\",\"sv\":\"Nyheter\"}"

post.title = { en: "News", sv: "Nyheter" }
post.title["en"] = "News"

This might be all you need and if it is, we retract what we said about Rails not having built-in support!

In our case we wanted the translatable fields to be easy to edit from a form, in addition we wanted it to be mandatory.

We like the translatable class method that the Globalize gem exposes, allowing you to declare attributes as translatable. Below is our take on an implementation.

module Translates
  def translates(field_name, options = {})
    serialize(field_name, HashSerializer)

    if options[:presence]
      validation_names = I18n.available_locales.map { |locale| "#{field_name}_#{locale}" }
      readers = store_accessor(field_name, *validation_names)
      private *(readers + readers.map { |name| "#{name}=" })

      define_validator(field_name) if options[:presence]
    end
  end

  private

  def define_validator(field)
    validate(
      proc do
        invalid = I18n.available_locales.map do |locale|
          errors.add(:"#{field}_#{locale}", :blank) if self[field][locale.to_s].blank?
        end

        errors.add(field) if invalid.compact.present?
      end
    )
  end
end

class HashSerializer
  def self.dump(hash)
    hash
  end

  def self.load(hash)
    hash = JSON.parse(hash) if hash.is_a?(String)
    (hash || {}).with_indifferent_access
  end
end

We can now declare our title attribute as translatable:

class Post
  translates :title, presence: true
end

This way we can have individual validations per locale version:

post = Post.new(title: { sv: "Nyheter" })
post.save
Post.errors.messages # => name: ["is invalid"], name_en: ["can't be blank"], name_de: ["can't be blank"]

We use two built-in methods in Rails to implement the validations. serialize let’s us override the default serializer so that we can access both name[:en] and name["en"].store_accessor` gives us somewhere to store the validation info on a per-locale basis, but we don’t want them to be used by mistake so we make them private.

In your views simply add the [] part to the name:

<input name="post[title][en]" id="post_title_en">
<input name="post[title][sv]" id="post_title_sv">

You also need to whitelist the nested attributes in your controller:

params.require(:post).permit(title: I18n.available_locales)

We’re big fans of using simple_form to generate form HTML. simple_form has support for extending the default list of input types:

class JsonbInput < ::SimpleForm::Inputs::StringInput
  include ActionView::Helpers::OutputSafetyHelper

  def input(_wrapper_options = {})
    safe_join(
      I18n.available_locales.map do |locale|
        options = {
          input_html: {
            name: "#{object.class.to_s.downcase}[#{attribute_name}][#{locale}]",
            value: object.public_send(attribute_name)[locale]
          },
          as: input_options[:type].presence || :string
        }

        @builder.input(:"#{attribute_name}_#{locale}", options)
      end
    )
  end

  def label(_wrapper_options = {})
    # label is rendered per translatable field
  end
end

Now when we do

<%= simple_form_for(@post) do |form| %>
  <%= form.input :title %>
  <%= form.input :body, type: :text %>
  <%= form.submit %>
<% end %>

simple_form knows that :title is a jsonb field and uses our input that creates all the inputs for us.

What about searching for records? Because of the GIN index we created in our migration we can do indexed searches in our jsonb column. You can probably come up with a nice abstraction but we only needed the search in one place so we simply implemented:

def self.by_title(title)
  names = I18n.available_locales.map { |locale| { locale.to_s => title }.to_json }
  sql = names.map { "posts.title @> ?" }.join(" OR ")

  where(sql, *names)
end

Conclusion

Perhaps you need every single feature available in big gems like globalize, perhaps you’re running an old version of mysql and can’t use json columns. Consider your use case before throwing in that gem "awesome-do-it-all-translation-gem" statement. Perhaps all you need is to create a jsonb column!

Hopefully this blog post has helped demystify database localization and given you a few ideas on how you can add the support you need in a more iterative way.

In the process of writing this blog post I’ve extracted the patterns you’ve seen above into a small gem named Tolken (Swedish for “The Translator”). Use it if you want, feel free to contribute to it or just browse its source for inspiration.

We hope you’ve learned something from this post. We sure did while writing it! If you have any feedback on how it can be improved, or if you spot any errors, please let us know by posting a comment below!

Versions

  • Article publish date: 2019-03-26
  • Article last updated: 2019-03-26
  • Last verified Rails version: 5.2.1
  • Last verified Ruby version: 2.5.0p0 (61468)
  • OS: Mac OS X 10.13.3 (High Sierra)