We’ve been building Offbeat Creations for over a year now, and we just managed to find enough time to actually put together a web site for the company!

We’re using the Radiant CMS to run this, to allow us to change the content quickly and easily. I’m still not very comfortable with the way Radiant works, but it does get the job done.

Check us out at Offbeat Creations

(The Offbeat Creations logo was designed by my friend John Betts at Glass Apple)

Lisa got me a few cool t-shirts for my birthday. I think my favorite is the Rock-Paper-Scissors-Lizard-Spock shirt pictured left. This shirt can be got from ThinkGeek , so you too can have the best geek shirt in town!

I’ve been using Rails for so long without caring about how Time Zone’s are processed, that when I recently built a new application with Rails 2.3, I decided that I needed to care about the support for Time Zones since Rails 2.1

Technically, my new app doesn’t really care about time zones, so I could have just left the whole thing alone, but I wanted to see how these worked and get some experience.

The first big surprise is that dates are stored in the database as UTC time. Well, duh! This is usually seamless, but I noticed this detail when my custom code to group by created date on a set of objects suddenly started showing items for tomorrow at about 5pm Pacific time.

Here’s the code I was using:

1       Items.find(:all,
2                  :select => "DATE(created_at) as created_on, count(*) as c",
3                  :group => "DATE(created_at)",
4                  :conditions => [ "created_at > ?", 14.days.ago.to_date ],
5                  :order => 'created_on desc')

I realized that I needed to adjust for the current Time Zone when retrieving this data.

With some investigation – Google search wasn’t entirely helpful here – I ended up with the following code that retrieves the Time Zone offset in seconds to add to the stored time (for Pacific Time, this is a negative number!)

1       zone = 'PDT'
2       Items.find(:all,
3                  :select => "DATE(date_add(created_at, interval #{Time.zone_offset(zone)} second)) as created_on, count(*) as c",
4                  :group => "DATE(date_add(created_at, interval #{Time.zone_offset(zone)} second))",
5                  :conditions => [ "created_at > ?", 14.days.ago.to_date ],
6                  :order => 'created_on desc')

Now my reported counts by day rollover to the next day at midnight Pacific Time. Adjust your zone setting to suit what results you need.

I was interested in using the new Rails Metal system for a project I’ve been building. What I want this code to do is take parameters out of the URL and write a record to my database to record these parameters.

This is my first application built with Rails 2.3, and the first application in my company to use Rails Metal in a production environment.

Here’s what I wrote:

1   def self.call(env)
2       if env["PATH_INFO"] =~ /^\/i\/(\d+)\/(\d+)\/(\d+)\/(\d+)/
3         Record.track($1.to_i, $2.to_i, $3.to_i, $4.to_i)
4         [ 200, { "Content-Type" => "text/html" }, [ "OK" ] ]
5       else
6         [ 404, { "Content-Type" => "text/html" }, [ "Not Found" ] ]
7       end
8   end

This code works great. Runs fast, does what I need.

Until I run it in production under some load!

Suddenly, I see requests backing up and monit is killing my Mongrel’s as the heartbeat call is timing out!

Diagnosing this problem was a bit painful. The RAILS_DEFAULT_LOGGER isn’t available, so I have to create my own logger, and then I used Benchmark.measure to writing out some timings.

The one thing I see is that every 5 calls, a call stalls for exactly 5 seconds. My superfast recording system is taking on average 1 second per request!

The number 5 rings a bell – where did I see that recent;y? I remember now, its in the database.yml file. There’s a new setting for connection pooling. I wonder how that works?

Well, it turns out that Rails Metal really doesn’t set up anything for you, nor does it tear anything down at the end of the request. For Connection Pooling the connection is setup automatically when you access your database, but outside of a Rails Controller, you’d better release this connection before you’re done.

Here’s a simplified version of how I did this:

 1   def self.call(env)
 2     if env["PATH_INFO"] =~ /^\/i\/(\d+)\/(\d+)\/(\d+)\/(\d+)/
 3       begin
 4         Record.track($1.to_i, $2.to_i, $3.to_i, $4.to_i)
 5       ensure
 6         # Release the connections back to the pool
 7         ActiveRecord::Base.clear_active_connections!
 8       end
 9       [ 200, { "Content-Type" => "text/html" }, [ "OK" ] ]
10     else
11       [ 404, { "Content-Type" => "text/html" }, [ "Not Found" ] ]
12     end
13   end

If you’re using the will_paginate Rails plugin to manage the amount of data that you show to the user on a single page, there is a gotcha that comes into play as your data gets bigger.

The default display of the pagination buttons for shows links that can take you to the last page of the set. If you only have a few thousand records, you’ll never notice the problem, but try it with a million records and check out the MySQL slow log file.

The SQL query generated by your paginate call will look something like:

1   SELECT * from foos ORDER BY created_at LIMIT 1000000,50

In the mysql-slow.log file you’ll eventually start seeing an entry that looks like this:

1   # Query_time: 6  Lock_time: 0  Rows_sent: 50  Rows_examined: 100050
2   SELECT * from foos ORDER BY created_at LIMIT 1000000,50

The key here is that MySQL had to look at 1M rows to return the 50 you requested!

This is a Very Bad Thing.

You must do all you can to avoid this case. What you do will depend on your specific application. The easiest is to not let page numbers go above a certain maximum, and configure the will_paginate view helper to not show all the possible page numbers.

I ran into a problem today where I had SQL queries taking a really long time to execute and I had pages timing out. After a lot of investigation, I found a strange behavior that I didn’t expect that was killing my application.

I’m using Rails 2.1.1 and I have a named scope that looks like this:

1 @data = Foo.something.another_thing.limited(10)

The limited scope adds a “limit X” to the query to reduce the number of rows returned

In my view, I have some code that looks like this:

1  render :partial => 'foo', :collection => @data 

This code works great.

Except when the Foo table starts to get large.

In the MySQL slow log, I was seeing almost this generated query, except it looked like the code was running it as a count(). Since my table is quite large (hence only getting the first 10 items!), this count was taking a really long time.

Checking through the code, I find that the partial rendering code calls empty? on the collection. For Named Scopes, this partial optimization actually causes the Named Scope implementation to call count if it doesn’t yet have data, which causes a huge database hit in my case.

So how do I fix this?

I think the best way to fix this is to use the all method on the named scope object to cause the data to be loaded by the time the partial rendering checks for for empty?

1  render :partial => 'foo', :collection => @data.all 

Its interesting that for a count() query with a limit X that MySQL doesn’t stop when it gets to X, but ultimately, its the Rails code performing an optimization that causes a database access that is to blame here.

My current desktop computer just had a power supply failure due to having a too powerful graphics card in the machine. Due to it being a non-standard Dell, replacing the power supply doesn’t solve my problem, and I can’t put a bigger one in that machine.

Time to build a new PC!

Recently, Intel introduced their new i7 processor. Quad core, 2.66 – 3.2 Ghz, tri-channel memory, built in memory controller and a new X58 chipset. Off to Google and NewEgg to get some parts together.

CPU

Intel i7 2.66 Ghz for $299 for the retail version

I got the slowest and cheapest version – at some future time I can upgrade to the 3.2 Ghz and keep everything else if I need to

Motherboard

ASUS P6T Deluxe – $299 from NewEgg

I got the Deluxe version to get the built in SAS controller, I did not get the version with the external module to aid the overclocking enthusiast.

Memory

Corsair 6GB DDR3 1600 memory in 3 sticks – $209 from NewEgg

The cool thing with this new chip is the DDR3 – the CPU memory controller supports this directly and allows faster access than DDR2 for the same amount of memory. The downside is that you need to install 3 sticks at a time to get the best performance. The ASUS Motherboard supports 6 sticks and 12GB currently.

Power Supply

CORSAIR CMPSU-750TX 750W supply – $119 from NewEgg

Need power for the graphics card that killed the old machine!

Hard Drives

2 X Fujitsu 147GB 15K RPM SAS drives – $250 each at NewEgg

I’ll mirror these drives with the built in RAID configuration to get reliability and put the OS and application installations here. I’ll be moving the 500GB drive from the old computer into this machine for storing data.

These 15K RPM drives are screaming fast compared to standard SATA drives.

Graphics Card

I recently bought a new Nvidia GTX 280 video card. This will transfer over easily. The ASUS motherboard supports running 3 of these in SLI mode. I’ll stick with a single card for now.

Operating System

Vista Ultimate Edition – The Microsoft Company Store will help me out with this for a good price. We need to be able to run all sorts of PC games on this machine, so Windows is the only way for our desktops.

Optical Drives

I’ll just transfer these from the old computer.

Construction

The parts have been ordered and should be here early this week. Hopefully I’ll have this running before the new year.

Why on earth would I want to run a Facebook application on my laptop? Surely these apps are meant to be used by lots of users and they shouldn’t stop running when I turn my computer off!

The only reason you’ll need to do this is during development. I run my apps on my laptop, behind a firewall, either at home, or on the road – so my IP address isn’t static. This allows me to make changes locally and immediately test them out without needing to copy to a server.

For this technique, you do need to have access to a unix server of some sort that does have a static IP address (or a fixed DNS name) that allows you to create SSH tunnels. With SSH client access, this should be possible with the low end hosting providers. Dreamhost costs less than $10 per month.

Facebook Application Setup

1. Create a new application with the callback url http://ip-address:15000/ and a name of my-app-name-development
2. Set your application configuration to use the app key and secret from step 1 when running on your laptop
3. Run your application locally on port 5000

SSH Tunnel setup

Now we setup to get the remote server machine to listen on port 15000 and forward requests to this machine on port 5000.

1 ssh -p 22 -NT -g -R *:15000:127.0.0.1:5000 <user>@<server> -v

I add the -v parameter so I can see a verbose output. Sometimes it takes Facebook a few seconds to connect to the tunnel, and the verbose option will allow you to see that when it works.

I add this command to a tunnel script I have and anytime I want to test out my app, I fire up the tunnel, then my application code, then access it on Facebook to try it out. If I have internet access, I can work on my Facebook applications.

I don’t use Rails, how does this help me?

If you can run a web server on your laptop, you can use this technique. This is not a Rails specific solution. I’ve seen information on the web for forwarding SMTP ports for email servers.

Windows?

I run this on Windows using a copy of SSH that comes with cwRsync