Scott Rutherford - Home

Exclude Records With UltraSphinx

2008-08-20T17:47:00Z

So been using Sphinx with the UltraSphinx plugin and I came across the requirement to use filters to reject records. Unfortunately in the plugin the filters are hardwired to submit the ‘exclude’ parameter as ‘false’ i.e include. Time for a quick hack I think…....

Very easy to fix, just open up the ‘internals.rb’ file in the UltraSphinx plugin or include the gem using ‘rake gems:unpack GEM=ultrasphinx’ (need to have config.gem “ultrasphinx” set in envrinoment.rb) and then find that file.

Ok, so found the file? On line 97 is this loop:

Array(opts['filters']).each do |field, value|          
  ....
 begin
     case value
     when Integer, Float, BigDecimal, NilClass, Array
       # XXX Hack to force floats to be floats
       value = value.to_f if type == 'float'
       # Just bomb the filter in there
       request.filters << Riddle::Client::Filter.new(field, Array(value), false)
     when Range
     ....
end

The ‘Riddle::Client::Filter.new(field, Array(value), false)’ call is the one we need to change (the ‘false’ is the exclude param).

So what I did was to change that loop to:

        Array(opts['filters']).each do |field, value|          

          field = field.to_s          
          type = Fields.instance.types[field]             

          # Special derived attribute
          if field == 'distance' and options['location']
            field, type = '@geodist', 'float'
          end

          raise UsageError, "field #{field.inspect} is invalid" unless type

# ADDED FROM HERE
          exclude = false
          # check for exclude flag
          if value.is_a? Hash            
            exclude = value[:exclude]
            value = value[:value]
          end
# TO HERE

          begin
            case value
              when Integer, Float, BigDecimal, NilClass, Array
                # XXX Hack to force floats to be floats
                value = value.to_f if type == 'float'
                # Just bomb the filter in there

# CHANGE CALL HERE TO PASS THE exclude PARAMETER
                request.filters << Riddle::Client::Filter.new(field, Array(value), exclude)
              when Range
                # Make sure ranges point in the right direction
                min, max = [value.begin, value.end].map {|x| x._to_numeric }
                raise NoMethodError unless min <=> max and max <=> min
                min, max = max, min if min > max
                # XXX Hack to force floats to be floats
                min, max = min.to_f, max.to_f if type == 'float'

# CHANGE CALL HERE TO PASS THE exclude PARAMETER
                request.filters << Riddle::Client::Filter.new(field, min..max, exclude)
              when String
                # XXX Hack to move text filters into the query
                opts['parsed_query'] << " @#{field} #{value}" 
              else
                raise NoMethodError
            end
          rescue NoMethodError => e
            raise UsageError, "Filter value #{value.inspect} for field #{field.inspect} is invalid" 
          end
        end

Now I can do searches like

    @search = Ultrasphinx::Search.new(
      :query => params[:q], 
      :page => page,
      :class_names => ["User"], 
      :filters => {:id => {:value => current_user.id, :exclude => true}}
    )

And the current_user gets excluded, cool.

Hop Toad

2008-08-20T15:49:00Z

I've been using the Hop Toad web app for a while now to collect errors from my Rails apps and its been working out great. It drops in very simply and replaces the Exception Notifier plugin I was using.

Big advantages are the archive of errors and the ability to mark them as dealt with. Of course I could do that with email folders and the like, but I'm lazy and this appears to do it all. Oh and its free!!!

Linux - Just say no.......

2008-08-01T01:13:00Z

Mashup of the Day - Tweetlists

2008-07-30T15:42:00Z

So, I got a great email about an hour ago:

To the creative genius[es] behind TweetLists,

We'd like to congratulate you on being selected as our MASHUP OF THE DAY at MashupAwards.com for July 30, 2008.

So, not all that familiar with The Mashup Awards, but others clearly are as the resulting traffic killed the site (not that that would of been hard)! I would like to say thank you to them for the award and also take this opportunity to thank my manager, parents, producer, jesus, god, twitter!, Tim Berners Lee, Matz........

Right then off to 'enjoy basking in the glow of my well deserved recognition' as suggested by 'The Mashup Awards Judging Panel'

Parsing Rails log files with JSON

2008-07-28T12:38:00Z

So I needed to parse the log files for SlimTimer this weekend to correct a data loss issue due to a small issue with implementing https for subscribers. The issue required looking for requests that had returned something other than “200 OK” collecting the parameters and entering any data that had got lost. After a bit of messing around it occured to me the the format of the parameters string in the Rail’s logs was not a million miles from the JSON format, so I came up with this to turn the string into a useable hash:

params =~ /.*: Parameters: (\{.*\})$/
str_hash = JSON.parse(params.gsub('=>',':'))

This provides a hash of the parameters used in the request. Of course the keys here are strings so to convert to symbols we can then use:

def create_symbol_hash(input)
  ret = input
  if input.is_a? Hash
    ret = {}
    input.each do |k, v|
      ret[k.to_sym] = create_symbol_hash(v)
    end
    ret
  else
    ret
  end    
end

and simply pass in the output from the JSON library. Seemed quite neat to me anyway.

How not to return your hire car!!!

2008-07-18T08:22:00Z

So just got back from a great 2 weeks in California, went to Foo camp, Social Media Camp, Mashable and a couple of other meetups – all excellent. One slight hiccup with the hire car…..... I claim it was the taxi’s fault (of course), although its a bit of a blur to be honest.

On the up side I can highly recommend Enterprise (and taking out the full coverage), they were very helpful and polite. Even offered me a new car – which would of seemed like a better idea if I could move properly!! At least no one was badly hurt.

TweetLists Comedy

2008-06-22T14:52:00Z

So, couple of pretty funny links creeping up the popular chart over at TweetLists

1) Yahoo Resigner (by Mat Honan)

2) Instant Rim Shot (by Scott Carver)

Goosh

2008-06-21T22:10:00Z

One of the top links that came from Tweetlists over the past couple of weeks is Goosh, an online google shell. Its great, well worth checking out. I really like the “translate” mode.

FCKeditor Plugin 0.5.1 Released

2008-06-21T22:04:00Z

Thanks to George Chatzigeorgiou there is now a version of the FCKeditor plugin that works with Rails 2.1. I have put the packages on Ruby Forge and updated the repository. If you are interested as to what the issue was check out George’s comments on the previous post.

Wordle

2008-06-19T15:55:00Z

My Delicious tag cloud from Wordle very cool

FCKeditor Plugin 0.5.0

2008-06-04T16:53:00Z

I created a new release for the fckeditor plugin today. It upgrades the version of the editor to the lastest 2.6 and also fixes the SanitizeHelper include issue which was causing problems with the spell check.

Other than that, little has changed. I think it is all still working ok, although I have only had time to complete a cursory check on the editor’s functionality.

The plugin is available from Ruby Forge or by doing this:

ruby script/plugin install svn://rubyforge.org//var/svn/fckeditorp/trunk/fckeditor

Twitter API: User IDs

2008-05-29T15:31:00Z

So, next lesson from TweetLists: the user id given in the xml from the public timeline is not a unique identifier. The screen name is the only way to identify a user. I ended up with 17 different ids for one user!!! Hence, gave up fixed it and dropped the database – too much effort to fix when its only been running for 12 hours.

TweetLists

2008-05-29T09:13:00Z

I released TweetLists last night, well actually this morning (about 3am), the idea being to try and capture a little of the zeitgeist on Twitter by aggregating the links people are talking about. At the moment there are 3 lists, the live feed, a popular feed (the links ordered by the number of times they appeared) and one I have chosen to call Twitterati (quite proud of that!) which is the links ordered by the number of followers people have.

At the moment there is no time aspect to the Twitterati or Popular lists (not relevant on the live one) but I guess I will make them for the last 7 days?

Lesson one form this has been: don’t try and be clever with your scheduling – cron just works. I tried to use the Rufus Scheduler and it had stopped running by the time I got up again this morning.

Anyhoo, I’ve found it to be fun and interesting so far, any ideas for other lists?

Plugin of the Day - GemsOnRails

2008-05-27T11:16:00Z

So this plugin by Dr Nik Williams (no link available) does this:

Link or freeze RubyGems into your rails apps, instead of just plugins. This allows you to ‘vendor everything’ – pushing all dependent gems into your rails app thus ensuring your application will be guaranteed to work when deployed. Your application is no longer dependent on the gems that are/aren’t available on your target deployment environment.

I found this very useful, mainly because for some gems I want to see and maybe play with the code, usual I just copy the gem to lib. This provides a simple way of bringing in only one gem. I know there are features in 2.1 that deal with gem dependencies and do some cool stuff, but from what I know (from Ryan Bates) I don’t think they provide this particular functionality.

Its available on RubyForge. Or just by using the “sudo gem install gemsonrails” command.

Xkcd: GeoHashing and Stuff

2008-05-26T00:37:00Z

Just been flicking through the cartoons at xkcd.com again and came across this one which made me chuckle:

Also found the GeoHashing, Spontaneous Adventure Generator to be very interesting idea. Check it out.