CRAZ8

Today’s Yakima Herald-Republic has an article about cheese made in the Yakima area.

Although it’s not obvious, the Yakima Herald-Republic web site is one of the early Ruby on Rails sites, written by Ezra

Cheese and Rails – can’t go wrong with either.

After setting up a Linux machine using these excellent instructions I wanted to expand my configuration to run some memcached processes to test session storage and caching

Having followed the original instructions, I knew that this ‘runsv…’ thing would be useful. It turns out to be something called runit that allows apps to be started, stopped, logged etc. Sounds similar in scope to Windows Services, but much easier to setup for an application.

How runit works is by monitoring a specific directory for sub-directories. For each subdirectory, it will run the run command until it exits for some reason, then it will run the finish command, and then go back and execute the run command again. If there is a log sub-directory, it will do the same in there too.

It will also do this at startup.

To get memcached working, I followed the instructions to build and install it. I then created a memcache1 directory in my services directory. In there I added a run command that looks like this:

exec 2>&1 /usr/local/bin/memcached -m 64 -p 2222

I now have a memcache instance running. Some testing with my Rails app confirmed this.

I added a second one – memcache2 – that looks like this:

exec 2>&1 /usr/local/bin/memcached -m 64 -p 2223

Now my Rails app is actually distributing sessions across multiple instances. In a production environment, these could be different machines, but in my test, this is all on the same machine.

Note: The Ruby memcache gem is not the correct version. I chased and reported a bug that is fixed in the most recent code

I also chased an issue with runit. I was looking at the top display, and I noticed runsv processes coming and going every second. This indicates something wrong. In my case, it was an original lighttpd configuration problem with the name of the logging directory being wrong. Make sure you get all the path names correct!

I’ve been noticing 404 errors on my blog for pages that should be there. After digging around, I noticed that the page cache is getting in the way of delivering pages.

Here’s what I think is going on.

I have a page with a URL http://blog.craz8.com/articles/2005/10/26/typo-has-caching-issues. I also have a page with a URL of http://blog.craz8.com/articles/2005/10/26 and one with a URL of http://blog.craz8.com/articles/2005/10.

If the page with the shorter URL is requested before the one with the longer URL, then the longer URL will return a 404. If the longer URL is requested first, then nothing bad happens.

Why is this?

First, here’s how Rails page cache works.

1. URL is requested from Rails, output is generated
2. An After filter takes that URL and the generated content and writes out a file in the cache directory called /a/b/c.html for the URL /a/b/c – creating the directories as needed.
3. A RewriteRule in the .htaccess file tells Apache to serve up the /a/b/c.html file for URLs of the form /a/b/c
4. Apache serves up the static HTML file if it exists, otherwise it calls into Rails.

What seems to be happening is, if /a/b.html exists, then a request for /a/b/c will be converted into a request for /a/b.html/c unless a directory called ‘b’ exists in ‘a’. So, the actual URL used by Apache depends on whether a subdirectory exists or not. If the shorter URL is requested from Rails before the longer URL, then the subdirectory will never be created and requests for the longer URL will always fail.

The order of hits to your site determines whether your users will see 404 errors for some pages

There maybe something that can be done in the Apache configuration to avoid this problem, but for now, I’ve disabled the RewriteRule entry in my .htaccess file that tries to perform this magic, and turned caching off in my Typo to avoid writing out cache files that won’t be used.

There was a DNS issue with blog.craz8.com that is now resolved, and Typo page caching has been a little tricky (since I’m not the owner of the generated cache files)

My blog has moved to Planet Argon for hosting. If you can see this entry, then all is well!

For christmas, Lisa got me Electronic Gadgets for the Evil Genius This thing has plans (sadly, not detailed) for an EMP, just like in the movies!

My plans for World Domination are set in motion, bwah-ha-ha-ha.

Next on my list: Fluffy White Cat.

Due to issues with page caching, I just started looking at the Rails action cache. There are a few things that could be done better by the built in code, but since this is written in Ruby, I can make those changes in my code and override the pieces I want.

The core of Rails Action Caching is the around filter applied to all actions that are to be cached. I re-implemented the before and after methods to make the changes I needed.

Here are the things I changed:

• Store cache entries as YAML streams so the Response headers from the original response can be returned with cache hits
• Add a ‘last-modified’ header to the response to get the client to use a get-if-modified request
• If the client has the response we have cached, don’t send it again, send a ‘304 Not Modified’ response to reduce data on the wire
• Fix a bug in the original Rails code where responses other than ‘200 OK’ are cached (since the headers aren’t cached in the original, all the clients would get is an empty ‘200 OK’ response from subsequent requests). Patch submitted

The code can be downloaded here. Place action-cache.rb in your lib directory and require ‘action-cache’ in your environment.rb to hook this in. Since this extends Rails Action Caching, all the documentation for that code still applies, e.g.

  caches_action :foo, :bar

The OutputCompression filter works with this code if it is set to run before the action cache. If you put the OutputCompression filter in your Application.rb file, and the action caching in specific controllers, both will work correctly.

Update: It looks like IE doesn’t send the If-Modified header unless the response contained a Cache-Control header. The code will now set the Cache-Control to max-age=1 if the action did not set it.

The Rails documentation has passing references to a filter class called OutputCompressionFilter. However, this code is nowhere to be found.

Since I have some code to do output compression (see my earlier post), I decided to repackage it as this class:

require 'stringio'
require 'zlib'

class OutputCompressionFilter

  # Do output compression if the client supports it
  def self.filter(controller)
    if controller.request.env['HTTP_ACCEPT_ENCODING'].match(/gzip/)
      if controller.response.headers["Content-Transfer-Encoding"] != 'binary'
        begin
          controller.logger.info "Compressing"
          ostream = StringIO.new
          gz = Zlib::GzipWriter.new(ostream)
          gz.write(controller.response.body)
          controller.response.body = ostream.string
          controller.response.headers['Content-Encoding'] = 'gzip'
        ensure
          gz.close if gz
        end
      end
    end
  end

end

Here is the source file – put it in your lib directory.

Here is how you use this code:

require "compress.rb"

class ApplicationController < ActionController::Base
  after_filter OutputCompressionFilter
  ...
end

This code works just like the Rails Filter documentation indicates.

Update: I’ve updated the source file to include the major updates in the comments from Dan, and my updates to that code.

<!--adsense-->

If I have a collection of things that are output like this:

<div id="things">
<% @things.each do |thing| %>
  <%= render :partial => 'thing' %>
<% end %>
</div>

or

<div id="things">
<%= render :partial => 'thing', :collection => @things %>
</div>

I can use AJAX to insert a new ‘thing’ by implementing an RJS template that does this, reusing the same partial layout:

  page.insert_html :bottom, :partial => 'thing'

but I can’t then replace the inserted thing by doing this, as this is implemented as element.innerHTML:

  page.replace_html "thing-id", :partial => 'thing'

There are ways around this, but they involve moving the outer element of the thing out of the partial code, splitting the HTML across two files, or removing and re-adding the element:

  page.remove "thing-id"
  page.insert_html :bottom, :partial => 'thing'

I don’t like either of these ideas, so I came up with an improvement that replaces the entire element in the DOM, similar to IE’s element.outerHTML:

  page.replace_html_element "thing-id", :partial => 'thing'

• The addition to the JavaScriptGenerator class to add a replace_html_element method
• An update to the Prototype Element implementation to perform the client side update.

# Update the JavaScriptGenerator to add our own functionality
module ActionView
  module Helpers
    module PrototypeHelper
      class JavaScriptGenerator
        def replace_html_element(id, *options_for_render)
          html = render(*options_for_render)
          record "Element.replace(#{id.inspect}, #{html.inspect})"
        end
      end
    end
  end
end  

Add this code to your application javascript file:

// Extend the object for our RJS extension to work
Object.extend(Element, {
  replace: function(element, html) {
      var el = $(element);
      if (el.outerHTML) { // IE
        el.outerHTML = html.stripScripts();
    } else {  // Mozilla
          var range = el.ownerDocument.createRange();
        range.selectNodeContents(el);
        el.parentNode.replaceChild(range.createContextualFragment(html.stripScripts()), el);
    }
    setTimeout(function() {html.evalScripts()}, 10);
  }
 }
);

I want to be able to compress my output text that I send to the browser from my Rails app. This will include HTML, javascript, Ajax responses, plain text etc. Lighttpd will perform this function for the static files it serves, but not for the dynamic files, so I went looking for a solution.

There is a README for ActionPack that has this snippet of code:

after_filter { |c| c.response.body = GZip::compress(c.response.body) }

I added this to my ApplicationController, and it doesn’t work.

I can’t find any GZip object in my Ruby or Rails code. I did find some output compression in the Ruby SOAP implementation, so I re-purposed that code for my after_filter. Here’s what I ended up with:

require 'stringio'
require 'zlib'

class ApplicationController < ActionController::Base
  after_filter :compress

  def compress
    if self.request.env['HTTP_ACCEPT_ENCODING'].match(/gzip/)
      if self.response.headers["Content-Transfer-Encoding"] != 'binary'
        begin 
          ostream = StringIO.new
          gz = Zlib::GzipWriter.new(ostream)
          gz.write(self.response.body)
          self.response.body = ostream.string
          self.response.headers['Content-Encoding'] = 'gzip'
        ensure
          gz.close
        end
      end
    end
  end
end

• The request must be checked to see if the browser supports gzip encoding before sending the compressed data
• I don’t do the gzip detection entirely to spec, if q=0 in the accept encoding header, GZIP should be disabled.

Since the request accept-encoding header must be checked, and the content-encoding header added, the original code example could never have worked correctly. My version of the code should work in most cases.

Update I added a check for the Content-Transfer-Encoding header. If this is ‘binary’, then don’t compress, as the response is an image, or something we don’t want to compress. Rails leaves this header off for text responses, like HTML.