CRAZ8

I recently read jPod the book by Douglas Coupland. Its a pretty good book, kind of weird – perhaps Quirky is a good description.

As a programmer in the video game industry, its kind of required reading

It turns our that CBC in Canada turned this into a TV Show)

This story works much better as a TV show!

I’m diagnosing a load problem with a server, and ATOP has helped me a lot.

For Ubuntu – ‘apt-get install atop’ works fine.

Here is the ATOP home page

I tracked down a performance issue in my code today that was pretty subtle.

I have a table with a varchar column that contains a number, mainly for size reasons. This column has an index all to itself, so queries using the column should be pretty snappy, but they weren’t.

My code went something like this:

  find(:all, :conditions => "my_col = #{value}")

I know I shouldn’t have built the conditions that way, but that’s the way the code was. And the performance sucked!

Since my_col is actually a string, I need quotes around the value otherwise MySQL has to try to figure it out, and it misses the index completely!

So, in future, use the :conditions the correct way, as Active Record does a pretty good job of getting the quotes right!

So, why did I write the code this way?

The code I actually wrote is for the parameters to a call to update_all, and the documentation for update_all isn’t as clear as it could be on how the parameters work. The key is that both of the first 2 parameters to update_all are the same as a :conditions parameter to find (but without the :conditions symbol). So they can be a string, or an array that has Active Record parameter substitution.

As far as I can tell, Facebook is always broken. Ok, they do seem to struggle on serving users, but a lot of functionality breaks on Facebook every day. I’ve seen no press on this, I guess the vultures are all circling around Twitter.

Here’s some examples over the last 2 months:

• Today – The category selection tabs in the Application directory broke. They sent you back to the home page.
• Every week when they update their servers on a Tuesday, the script in my IE browser breaks and I need to restart the browser.
• When they were rolling out the Chat code in April, there were times where every second page load would get a ‘connection reset’ error.
• Sometimes login doesn’t work
• Too often to mention, the API calls will get a Connection Reset
• The ‘infinite session’ key applications can save and use later often fail in the future for no reason
• Getting an app into the Application Directory can take forever
• The new Import for RSS into your Feed seems to not work at all.
• On one of my apps, Facebook doesn’t always do the right thing for the Google Analytics script and comes to my server to get the urchin.js file if the current directory, causing Routing Errors all over my logs.

Now, when things break, its usually not all the servers, so some apps for some users will be broken, while others will be working just fine. Facebook does tend to fix these things quite quickly, but I’ve seen very little communication about what is broken when its broken.

It’s always exciting in Facebook Land!

Update: Less than an hour after writing this post, Facebook broke CSS stylesheets for apps.

In the Facebook Developer Application, the act of deleting an application does have a confirmation page, but it isn’t very clear on what application is being deleted – there is the name of the application, but not the picture or icon.

And the delete works even if you have thousands of users.

Ask me how I found this out, I dare you!

Many would say yes, but Tom Standage would beg to differ in his book The Victorian Internet

Mr Standage tells the story of the people who created the telegraph system starting at the end of the 18th century, and leading to Samuel Morse and the first electric system in the US.  He goes into the social aspects of suddenly being able to be in touch across the world, the doom and gloom that came from the newspaper industry, and the rise and fall of the operators of the equipment.

I found out about this book from Rick Segal’s blog The Post Money Value

I got my copy from the library, but Amazon seems to have a recent reprint available

One of the interesting things about Facebook is that, of the interesting pages on Facebook, everyone sees a different version of that page.

Search is clearly filtered to show stuff that is more relevant to you first. Your ‘facebook’ page has stuff filtered out depending on what Facebook thinks is interesting to you (with input from the tiny ‘Preferences’ settings that I expect no-one actually changes)

Search is the tricky one. If someone tells you to type something into Google, you’ll see the same results as them. On Facebook, probably not! This could give Facebook great opportunities for advertising, but their search actually sucks! e.g. I searched for an app the other day and couldn’t find it – I was using the singular form of the name when I should have used the plural. Now I have that app installed, it shows up higher in the search, but now it’s too late.

I’ve been working on a project for a while. The site has been live for a month or so, and seems to be stable. If you visit my site to read the posts, you may have noticed the top section of the left sidebar has been showing a widget from my new site VideoSlush:http://videoslush.com

VideoSlush:http://videoslush.com is a trading game for YouTube videos. You find cool videos that you think may get a lot of views in a short amount of time. You ‘buy’ the video and also take a guess at how many views it may get by noon the next day to earn a bonus. Then you see how your video does over time, using the graphs to see when the price peaks and then selling.

Sign-up is free and open, so check it out.

Caching is a real pain to get right. Actually, the caching part is quite easy with Rails, its the expiry that causes the trouble. All the simple cases are easy! Its when you have a page with various pieces of data, with different expiry conditions that the trouble starts.

I’ve written some code to use versioning in my fragment cache that is working for my most recent project. This technique relies on using Memcache to work correctly. I’ve been thinking about packaging this up as a plugin and releasing it. Luckily, I never got around to it.

Today, I read this post on how to do better caching. This plugin has documentation and tests, and is easier to configure than mine.

Go check it out.

http://blog.evanweaver.com/articles/2007/12/13/better-rails-caching/

If you are using the Rails implementation of validates_uniqueness_of in your model to ensure duplicate data doesn’t get into your database, your application is broken. If you don’t use the database to ensure uniqueness with a key, then your app will fail at some point in the future. Probably just when you start getting some decent traffic and would like the app to not fail.

How do I know this? It happened to me this last weekend.

In checking this out, I find that Michael Koziarski of the Rails core team recently wrote

validates_uniqueness_of gives a nice error message, and does an ok job at guaranteeing uniqueness. Validates uniqueness of + a unique index does both.

In my app, does a nice job isn’t really good enough when it comes to uniqueness.

Michael has also posted that, hey, if you don’t like it, fix it and submit a patch, which is a great idea, but for this case is a really hard problem to solve for the general case across many databases.

I didn’t find a lot of specific fixes with code that I liked, so here’s how I fixed it in my app.

For my case, I already had a do some stuff and save the model method. In this method, I added some code (I stripped out my app specific code in the example).

  DUPLICATE_ERROR_MESSAGES = [
      "Duplicate entry"
  ]

  def save_new
    begin
      save    
    rescue ActiveRecord::StatementInvalid => error
      if DUPLICATE_ERROR_MESSAGES.any? { |msg| error.message =~ /#{Regexp.escape(msg)}/ }
        logger.info "Duplicate Entry exception from DB"
        errors.add_to_base('Duplicate item not allow')
        return false
      else
        raise
      end
    end
  end

I also added a unique key to the table to ensure the MySQL database throws the exception that the above code relies on.

I still keep the validates_uniqueness_of call, as this does give a more specific error message, but now I don’t rely on it to enforce uniqueness.

Shh, don’t tell DHH that I’m putting integrity checks into my database.