Rails memory usage and long running background tasks

September 1, 2006 Link to post  Permalink

I have a backround task for my rails app to send a bulk mailing to a list of email addresses and receive the incoming replies to those mailings to detect bounces and de-activate email addresses. The mailing is controlled by my rails app, with the relevant model indicating what to send and when.

I’m currently using rails_cron for this task, running as a daemon, and polling for the outgoing mailing and incoming mail in the user’s mailbox. When poking around with top, I noticed this process is using about 23MB of virtual space. Since I’m expecting to send maybe two mailings a week, using 23MB all the time seems to be a little excessive.

I wrote a small test ruby script to see the ruby minimum app size:

3while true
4  sleep 60

This app takes up just over 3MB while running. Large for doing nothing, but I can live with that for now.

I then added an include for my enivronment.rb, and this process now takes up 20MB of virtual space. Now we’re using too much memory for something that will have work to do maybe twice a week.

Since I have to poll for the incoming mail, I can’t just kick this task off based on user action, I need to have a lightweight app that can run my heavyweight poll code from time to time.

I think I’m going to replace my rails_cron script with a long-running shell script that runs my rails script on a schedule to poll the inbox and for an outgoing mailing, and live with the rails startup overhead for this.

Update: I was looking at the wrong process when checking my memory usage. My app is taking 23MB not the original 50MB. I’ve updated the above text with the correct numbers