This is my try at text search described here

There is already redis running in my app as the back end for Rails Cache.The new requirement was the ability to search posts by user.

I wanted to make better use of the redis server that is already running (and is under utilized by the Rails cache).

We first build index of all the content we want to search.That is , like the index you find at the back of a book , all the page numbers where a term can be found.In my case , the page numbers are the id of the post in the database.

Post {:id => 1 , :content => 'Can we go now?'}
Post {:id => 2 ,:content => 'We are here to arrest you'}

My index would be :

'Can' => [1]
'We' => [1,2]

… and so on

Redis Sets can be used for this purpose , we need to use the following from redis :

  • To add an item into a set
redis.sadd 'name_of_a_set', item 
  • To retrieve an item that is in the intersection of set1 and set2
redis.sinter 'name_of_set_1' 'name_of_set_2'

Instead of using the exact words as the keys of index , we use their metaphones , to qoute from the original post that inspired to do this :

…Enter phonetic algorithms, a blessing to the linguistically challenged.

Phonetic algorithms are actually pretty simple. They are just a way of taking a word, dropping out some letters (according to a set of rules), and providing you with a rough representation of how that word sounds. Common phonetic algorithms are Soundex and Metaphone. The former is pretty dated now, but we understand the latter to be more up-to-scratch (any to play nicely with other Western languages)…

Looking for a Metaphone implementation in ruby , I come across metaphone.rb

I planned to build the index every thirty minutes for now .Depending on the use of search and interval new posts are created , we can fine tune that.If there were frequent searches and less frequent new posts , we could update the search index everytime a new post is made.

But for now , we will have a rake task that will run once in every thirty minutes,

namespace :search do
  desc 'Build index'
  task :build_index  => :environment do 

    require './lib/metaphone.rb'
    $r = unless $r

    last_indexed_time = $r.get('last_indexed_time')
    if last_indexed_time.nil? 
      #Never been indexed
      last_indexed_time = Post.order(:updated_at).select(:updated_at).first.updated_at

    puts "Indexing from #{last_indexed_time}"

    Post.where('updated_at > ?',last_indexed_time).each do |p|
      Text::Metaphone.metaphone(p.content).split(' ').each do |mtphn|
        $r.sadd "word:#{mtphn}",

    last_indexed_time =


We will have an index build by now , time to query the index :

class Search
  require 'metaphone.rb'

  def initialize(query)
    $r = unless $r
    @query = query

  def results
    index_keys = Text::Metaphone.metaphone(@query).gsub(/(\w+)/,'word:\1').split


Now we can perform the search as'arest').results

which would give the result ‘2’.