Microsoft WebDav opens document as Read-Only when using RailsDav

July 1st, 2008 by Joseph Jakuta

I had been working on a project in which we wanted to utilize WebDAV (namely for editing Word & Excel Documents that were saved in our application). In order to do this we decided to use a plugin from liverail.net that can be found here. It was pretty easy to hook up after a little direction from a guy over at Benryan Inc [apologies I cannot find a link for them], but there was a major issue. When opening a document through the ActiveX controller for editing it was opening in Read-Only mode.

After a few starts and stops, many hours of reading through the webdav documentation, and browsing through the http traffic using Fiddler - it was determined that locking was the issue.

Read the rest of this entry »

Rails 2.1 broke my mysql foreign keys!

June 24th, 2008 by John Trupiano

Rails 2.1 introduced in the MySQL Adapter “smart integer columns.” The idea was to use the :limit option to determine whether a smallint, int, or bigint should be used. This is something that the Postgres adapter had already previously implemented. The relevant code in activerecord/lib/active_record/connection_adapters/mysql_adapter.rb is:

  # Maps logical Rails types to MySQL-specific data types.
  def type_to_sql(type, limit = nil, precision = nil, scale = nil)
    return super unless type.to_s == 'integer'

    case limit
    when 0..3
      "smallint(#{limit})"
    when 4..8
      "int(#{limit})"
    when 9..20
      "bigint(#{limit})"
    else
      'int(11)'
    end
  end

Mirko Froehlich suggests monkey patching this function. Timothy Jones blogged about it.

To monkey-patch this, just drop a file (fix_mysql_adapter.rb) into your initializers/ directory, as such:

module ActiveRecord
  module ConnectionAdapters
    class MysqlAdapter < AbstractAdapter
      # Maps logical Rails types to MySQL-specific data types.
      def type_to_sql(type, limit = nil, precision = nil, scale = nil)
        return super unless type.to_s == 'integer'

        case limit
        when 0..3
          "smallint(#{limit})"
        when 4..11
          "int(#{limit})"
        when 12..20
          "bigint(#{limit})"
        else
          'int(11)'
        end
      end
    end
  end
end

All we’ve done is change how the mysql adapter interprets the limit attribute. In my opinion, this isn’t a particularly great solution, as it’s more appropriate for non-foreign key integers to behave as the rails team suggested. This really only affects us when we’re dealing with foreign keys, because mysql enforces our column types to match.

The Real Problem
So, if the problem isn’t here, where is it? The problem actually lies in the schema dumper. Notice in the previous code samples the else clause in the case statement. If limit is nil (or outside of 0..20), then this falls back to int(11). Curiously, when I have a migration such as the following, the schema dumper adds a :limit => 11, even though I didn’t specify it!

$> rails test -d mysql
$> cd test
$> ruby script/generate model user name:string
$> ruby script/generate model game name:string user_id:integer
$> rake db:create db:migrate

First, let’s check our game migration to verify that it doesn’t specify :limit => 11.

class CreateGames < ActiveRecord::Migration
  def self.up
    create_table :games do |t|
      t.string :name
      t.integer :user_id

      t.timestamps
    end
  end

  def self.down
    drop_table :games
  end
end

So, there’s no limit specified. But let’s take a look at schema.rb.

ActiveRecord::Schema.define(:version => 20080624161220) do

  create_table "games", :force => true do |t|
    t.string   "name"
    t.integer  "user_id",    :limit => 11
    t.datetime "created_at"
    t.datetime "updated_at"
  end

  create_table "users", :force => true do |t|
    t.string   "name"
    t.datetime "created_at"
    t.datetime "updated_at"
  end

end

Ah ha! Where’d that :limit => 11 come from? The schema generator!!! This is our culprit. We need to isolate the schema generation code, and ensure that isn’t forcing :limit => 11 onto our integers that don’t explicitly set a limit.

As I started down this track, I hooked up with Rob Sterner from ITA Software on IRC. He filed the original ticket. By the time I got in touch with him, he pretty much had the problem solved.

First, let’s fully illustrate the problem. When you run migrations, naked integer statements

t.integer :myvalue

are inserted into the database with :limit => 11 (this is handled by the extract_limit() function in the adapters). SchemaDumper.dump is called after your migrations are finished, and creates schema.rb by inspecting the database (not your migrations — which makes sense). However, when extracting the information from the database, the SchemaDumper picks up this limit of 11, which causes it to write this out in your schema.rb file:

t.integer :myvalue, :limit => 11

See a problem? Yup, where in our migration we did not explicitly set a limit, the SchemaDumper did!

The Solution
So what’s the solution? Well, what we need to do is alter SchemaDumper to identify int(11), and special case the output (so that :limit => 11 is not appended). At first, this seemed very hacky to me. What if I want a bigint(11) and not int(11)? Well, turns out this isn’t a valid concern. Why? If we really wanted bigint(11), our migration would look different:

t.integer :myvalue, :limit => 11

So, when migrations are run, this field would be placed into the database as a bigint(11) column. And, when SchemaDumper encounters it, it will also see it as bigint(11), As long as our special case discriminates between bigint(11) and int(11), then we’re in the clear! The fact is, in rails 2.1, there’s no way to get an int(11) column in mysql unless you’ve left off the :limit in your migration.

Where’s the patch?
Funny (read: agonizing) story…..we were testing all of this against 2.1.0. When we checked out edge rails, all of this code had changed!!! In fact, looks like Jeremy Kemper (bitsweat) already fixed it.

For the sake of completeness, I’m including the patch that Rob threw together that does the trick for 2.1.0. This patch applied to /activerecord-2.1.0/lib/active_record/connection_adapters/mysql_adapter.rb does the trick.

127c127,129
<           else
---
>           elsif sql_type == ‘int(11)’
>             nil # special case for :integer columns w/ no explicit :limit set in their migration
>           else

The Lesson
Check out edge rails to make sure that the problem is still unresolved before you spend all day on something!!!

Also, be aware that edge has changed the meaning of :limit. See the ticket for Jeremy’s explanation. Up through Rails 2.1, the :limit attribute for integers dictated the display width. In other words, it did not specify the storage size, but rather the amount of space mysql would use to display it when returning query results. Moving forward, it will be used to indicate the number of bytes to use for storage. Rails will now effectively use tinyint (1), smallint (2), mediumint (3), int (4) and bigint (8). So, prepare yourselves accordingly.

Special thanks to Rob Sterner for spending lots of time today working through this with me.

Don’t Abuse the Session

June 23rd, 2008 by John Trupiano

Never, ever, ever, ever, ever store an ActiveRecord model in the session. Just store the id and load it into an instance variable from the database on every request. Why? A couple reasons…

First, you’re susceptible to staleness. Consider this. User A logs in, and you store their user object in the session. Administrator X logs in and deactivates User A’s account. User A can still muck around your site because you’re reading the user data from the session, which has stale data.

Second, the default in Rails these days is to store your session data in cookies (honestly, I don’t know why…..it only clutters up your requests, forcing the session to be passed back and forth on _every_ request, and opening up the possibility that the encryption key could be brute-forced……this is a rant for another day). You just don’t want to be storing whole ActiveRecord objects in the session. They’re big and clunky. The extra database call to reload the object in a before_filter on every request is practically trivial, and you’ll keep the “tubes” less clogged.

This practice is certainly not rails-specific, and should be adopted no matter the server-side technology.

Google AJAX Libraries on Rails

June 20th, 2008 by Nick Gauthier

If you’re reading this blog you’ve probably already heard about Google’s AJAX Library API on many other news sites like Slashdot.

I’m going to describe my simple process for setting up a RoR app to use Google to pull the APIs in a rails friendly way, throughout layouts, views, and helpers.
Read the rest of this entry »

Automating Flex Compilation Using ANT

June 15th, 2008 by Greg Jastrab

When we first started developing Flex applications for clients when the time would come to send the SWF over, I would build the application in Flex Builder and send off the generated SWF. This got the job done, but it imposed a few limitations since I was the only Flex developer in our office:

  • I was the only one that knew how to compile the application
  • If someone else wanted to try to compile the application, they’d have to install Flex Builder

After reading a blog post by Marc Hughes I realized it was time we put in place a more versatile environment for building Flex applications.

Read the rest of this entry »

Ruby on Rails Polymorphic Association Benchmarks

June 13th, 2008 by Nick Gauthier

Polymorphic relationships in Ruby on Rails are great. If you don’t know what they are, check them out here:

Understanding Polymorphic Associations

John and I were curious about the speed of these relations, since the linking between objects searches on both the ID of the foreign object, and a string which is the model name. So if you have two tables, ChildA and ChildB, your parent has a reference to child which is acutally the combination of child_id (the ID in the ChildA or ChildB table) and child_type (equal to “ChildA” or “ChildB”).

The old-school way of doing this involves creating a lookup table and using integer IDs for type, instead of strings. So you’d have another table mapping “ChildA” to “1″ and “ChildB” to “2″, then when you do your query, you are matching against the number “1″ and not the string “ChildA”.

The down side of doing it that way is that you don’t get to use Rails’ snazzy polymorphism, which makes life a lot easier. So we decided to run some tests to see how much faster it would be, and therefore, if it was worth it.

Read the rest of this entry »

Multithreading in Ruby on Rails

June 11th, 2008 by Nick Gauthier

Don’t you hate it when sites say “Please Wait” when you’d rather just come back later? I am always worried my browser will close and it won’t work. Or maybe I want to shut my computer down but I have to leave my task running. Read on!

Read the rest of this entry »

Subversion Timestamps + Capistrano finalize_update

June 7th, 2008 by John Trupiano

Update 2008/06/13: Jamis released Capistrano 2.4.0, and it includes the :normalize_asset_timestamps patch that I submitted!

Update 2008/06/11: Here’s a link back to the Google Groups Discussion regarding this topic.

Subversion has a lesser-known feature that allows you to specify that checkouts/exports/switches/reverts should timestamp files with the last committed timestamp. By default, this setting is turned off. As such, when you checkout a repository, every file is timestamped with the current time on your local machine.

To be honest, I’m not quite sure why this is the default. The pertinent section of the Subversion manual (you have to scroll to the bottom) describes the setting as such:

———-
use-commit-times

Normally your working copy files have timestamps that reflect the last time they were touched by any process, whether that be your own editor or by some svn subcommand. This is generally convenient for people developing software, because build systems often look at timestamps as a way of deciding which files need to be recompiled.

In other situations, however, it’s sometimes nice for the working copy files to have timestamps that reflect the last time they were changed in the repository. The svn export command always places these “last-commit timestamps” on trees that it produces. By setting this config variable to yes, the svn checkout, svn update, svn switch, and svn revert commands will also set last-commit timestamps on files that they touch.

———-

It’s not clear to me why the default aids in Makefiles. What is clear to me though is that Jamis Buck has taken this default behavior into account in Capistrano, the wonderful deployment tool we use at SLS.

The following code snippets will require a bit of understanding of the built-in Capistrano deployment recipes. Let’s take a look at the code for the :finalize_update task. This task is invoked after the code has been updated on the server (for Subversion, either by an export or update).

  task :finalize_update, :except => { :no_release => true } do
    # ... other details omitted

    stamp = Time.now.utc.strftime("%Y%m%d%H%M.%S")
    asset_paths = %w(images stylesheets javascripts).map { |p| "#{latest_release}/public/#{p}" }.join(" ")
    run "find #{asset_paths} -exec touch -t #{stamp} {} ';'; true", :env => { "TZ" => "UTC" }
  end

What this snippet of code does is compute a timestamp, and then touch each asset file on the server with that timestamp (-t #{stamp}). The intention for doing this was to handle the scenario where you have multiple asset servers. Since an export/checkout updates the timestamp with the local machine’s current time, it’s possible for the same asset to have different timestamps on separate asset servers.

So what’s the big deal? First, rails serves up images (when using the image_tag helper) with a querystring appended to it. This querystring is simply a timestamp. The reason for this is to support client-side caching (you’re doing this, right?). This basically allows you to set the “expires” attribute of that file several years (decades or millenniums, in fact) into the future. The reason this is so is because if that file ever changes, it’s last modified attribute will also change, effectively changing the querystring rails appends automatically, and causing your browser to download a ‘new asset.’ So, when the finalize_update task is invoked (which happens every time you re-deploy), all of these last-modified timestamps are reset, causing any repeat visitors to re-download these very same assets again.

I have submitted a patch to Jamis (which I hope he’ll apply soon!) that exposes an extra Capistrano parameter (:normalize_asset_timestamps), which would be set to true by default, leaving the original behavior in tact. The new :finalize_update task looks like:

  task :finalize_update, :except => { :no_release => true } do
    # ... other details omitted

    if fetch(:normalize_asset_timestamps, true)
      stamp = Time.now.utc.strftime("%Y%m%d%H%M.%S")
      asset_paths = %w(images stylesheets javascripts).map { |p| "#{latest_release}/public/#{p}" }.join(" ")
      run "find #{asset_paths} -exec touch -t #{stamp} {} ';'; true", :env => { "TZ" => "UTC" }
    end
  end

I’ll follow up when/if Jamis accepts the patch. Hopefully it can make it into version 2.4!

Deploying Rails Apps with Capistrano without root or sudo Privileges

June 6th, 2008 by John Trupiano

In an effort to prepare for my presentation on Rails Deployment with Capistrano and Phusion Passenger at the next Bmore on Rails ruby users meetup, I’m writing a series of blog posts to help illustrate some concepts. This represents the second installment of the series. Better setup for environments in rails addressed the set of structural changes that I make to any fresh Rails app. This post will focus on some general principles and useful security considerations to take into account when deploying Rails apps with Capistrano.

The primary point of this post is this: You don’t need to deploy using root. And you don’t need to grant sudo access to the user used for deployment.

Our primary deployment setup is either a single or two-box solution (web server, asset server, database server spread across two). We generally use MySQL for the backend, and Phusion Passenger to serve Rails. We deploy to either Ubuntu Server (Hardy 7.10) or CentOS 5. We also generally disallow root ssh access.

First of all, it’s important to categorize tasks into two types: privileged tasks and unprivileged tasks. The nice part about a rails app is that, for the most part, it’s pretty self-sufficient, and rarely ever needs to venture out of its own tree in the filesystem. This means that we can get away with deploying with an unprivileged account. There are, however, certain ’setup’ tasks that likely need to be executed with root/privileged access. Fortunately, all of our privileged tasks can be performed before we ever deploy the app!

Privileged Tasks
For our baseline deployment, this includes the following:
1) Install any necessary software (Ruby, RubyGems, ImageMagick, MySQL, etc.)

2) Create the Rails app structure. For us, we create the following structure:

/var/vhosts/myapp
  /releases
  /shared
    /content

The default Capistrano setup task performs similar functionality. The only additional folder here is /shared/content. We use this folder to hold all of our uploaded assets (mostly via the File Column plugin). Then, on successive deployments, we set up symbolic links from the public directory up to this shared folder. This allows these assets to live outside of the context of a specific release.

3) Create a log directory at the system level: /var/log/myapp.

4) Create a symlink from the apache config directory (generally /etc/apache2/sites-enabled on Ubuntu, /etc/httpd/conf.d on CentOS) to /var/vhosts/myapp/shared/passenger.conf. Note that at this stage, passenger.conf does not exist. This is ok, as our cold deployment will address this, and each successive deployment will exploit this. This passenger.conf file will actually just be another symlink out to the current release. What this allows us is the ability to alter our apache/passenger configuration for this app on successive deployments. The apache config directories will not be visible to non-privileged users, and thus, we won’t be able to update those symlinks using a nonprivileged account.

5) Create (if it doesn’t already exist) a deploy user, and assign it to the same group that apache runs (www-data on Ubuntu, apache on CentOS).

$> adduser --group www-data deploy

6) chown the app root (/var/vhosts/myapp) and log directory (/var/log/myapp), and give both user/group write permissions (775, 774, or 770). Apache’s user will be able to write to these directories by virtue of them being owned by the group.

$> chown 775 deploy:www-data -R /var/vhosts/myapp /var/log/myapp

7) Create your database, and grant all privileges to a non-root user.

mysql> CREATE DATABASE myapp;
mysql> GRANT ALL PRIVILEGES ON myapp.* TO 'myappuser'@localhost IDENTIFIED BY 'asecretpassword';

8) Install Passenger and the GemInstaller gem (as long as we keep our geminstaller.yml file up to date, we can ensure that our server will use those exact gem versions).

In order to execute these tasks, you’re going to need root access. Note, however, that it is ill-advised to ever include your root password in your deploy script (lest someone accidentally commits it to the repo). One way to handle this is to implement some/all of this functionality in cap deploy:setup. You can then temporarily run this with the root user, but no password (this only works if root can ssh in), which will force you to enter your password. I like this approach, because at this point, we’ll never need the root password again. Put it in a lockbox and leave it alone! Your other option (if root can’t login), is just to login with the unprivileged account, su -, then perform these steps manually. Similarly, you can temporarily grant your deploy user unrestricted sudo access temporarily (but don’t forget to undo this!) in the sudoers file. Either way, these are one time steps, and it’s really not much of a hassle if you know what you’re doing.

Unprivileged Tasks
Everything else you’ll ever have to do (unless you’re adding a new feature that requires installing other software, etc.) with Capistrano can now be completed with the unprivileged deploy user that we created in step 5 above.

At this stage, the only difference between a cold deployment and a successive deployment is the fact that your app was never running in the first place. In essence, it’s the difference between a hard restart of apache and a soft restart. All other steps are the same:

1) create a new release (with the updated code)

2) update the current and filecolumn symlinks (these were the symlinks I mentioned above that point out to the shared/content directory)

3) ensure that all of our necessary gems are installed (reading geminstaller.yml and executing geminstaller if necessary) — more on this in the next post

4) run any pending migrations

5) update /var/vhosts/myapp/shared/passenger.conf to point to the config snippet in the latest release

6) restart apache (hard if cold deploy or if the apache config is updated, soft otherwise)

We overwrite more or less all of the default Capistrano recipes. Actual code will be released probably just after the presentation (Tuesday, Jun 10, 2008, 7:00 PM at Medical Decision Logic, 1216 E. Baltimore Street, Baltimore, MD 21202), as I’m still tweaking things here and there.

The important take home note is this: yeah, it’s nice to be able to just pull out your Capistrano recipes and build an app on a brand new server from scratch with a few command line calls. However, it is borderline impossible to securely pull this off. The line between server setup and application deployment becomes blurred when you try to put this together. The server setup process, by nature, requires root/privileged access. Incremental deployment, however, does not require this level of privilege. Capistrano was not designed to build a server from scratch. Rather, a better approach is to develop a server image that you will use for all of your client app servers.

My next post will elaborate (with more code samples, particularly recipe snippets and some capistrano/rails extensions I’ve been working on) on much of what was covered here. Additionally, I’ll go into further detail regarding other ways to make maintaining your production apps easier with Capistrano.

map.resources and custom nested routes

June 5th, 2008 by Scott Davis

I encountered an error in rails trying to create a nested route in rails 2.x

map.import_time_cards 'users/:user_id/time_cards/import',
:controller => 'time_cards',
:action => 'import'

Wasn’t setting up a route for users because this route was being setup automatically and overwritten by:

map.resources :users,
:has_many => [:notes, :addresses, :expenses, :time_cards] ,
:collection => [:login, :logout, :disable, :enable]

So after digging around on the rails api I discovered that map.resources takes a block so my solution to this problem was :

map.resources(:users,
:has_many => [:notes, :addresses, :expenses] ,
:collection => [:login, :logout, :disable, :enable]) do |user|
user.resources :time_cards, :collection => [:import]
end

By using a block this tells rails to include route to ‘users/1/time_cards/import’ instead of appending import as the id for the show route.