SmartLogic Logo (443) 451-3001

The SmartLogic Blog

SmartLogic is a web and mobile product development studio based in Baltimore. Contact us for help building your product or visit our website to learn more about what we do.

Better setup for environments in Rails

June 2nd, 2008 by

Update: I have created a gem (environmentalist) to create this configuration structure for you. Read Introducing environmentalist for an introduction.

I will be presenting at the next Baltimore Ruby Meetup (Tuesday, 6/10/08) on deploying applications with Capistrano and Phusion Passenger. In an effort to prepare (and perhaps induce a little bit of interest), I am writing a series of blog posts that help set the stage for the presentation.

In this first post, I discuss a common set of changes I make to the config structure of a fresh Rails app. This is pertinent because it has some (minor) effects on our deployment procedure, namely within my core capistrano recipes.

rails test -d mysql

One of the things that bothered me about the default config structure is the database.yml file. The file contains the database credentials for all of our environments. As you should know, the default file looks like:

# ...
# And be sure to use new-style password hashing:
#   http://dev.mysql.com/doc/refman/5.0/en/old-client.html
development:
  adapter: mysql
  encoding: utf8
  database: test_development
  username: root
  password:
  host: localhost

# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
  adapter: mysql
  encoding: utf8
  database: test_test
  username: root
  password:
  host: localhost

production:
  adapter: mysql
  encoding: utf8
  database: test_production
  username: root
  password: 
  host: localhost

We’ve got development, test and production credentials in this file. So…should this file be added to your repository? Well, localized settings files are generally not suitable for the repo…but what about that production block? That sure looks like it belongs in the repo considering there’s no reason it should differ amongst developers. Your production environment is, after all, the same as everyone else’s on your team. What if we wanted to add another environment (e.g. a staging environment)? That should probably go into the repo as well.

Some make the argument against putting the file into the repo. I’ve seen several capistrano scripts that echo out the contents of this local file onto the production server. This is fine and all (particularly if you set your recipe to read from the local copy), but then every developer needs to make sure that their local copy has the exact same credentials in that file. Another method I’ve seen is just copying those credentials directly into your cap recipe…..but that isn’t very DRY, and forces the developer(s) to remember that the attributes are repeated in multiple files.

Furthermore, how would we handle other pieces of configuration? For instance, with Passenger, each app stores locally an apache config :

<VirtualHost *:80>
  ServerName test.smartlogicsolutions.com
  RewriteEngine on
  RewriteCond %{SERVER_PORT} !^443$
  RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [L,R]
</VirtualHost>

<VirtualHost *:443>
  ServerName test.smartlogicsolutions.com
  DocumentRoot /var/vhosts/test/current/public

  PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.0.0
  RailsRuby /usr/local/bin/ruby
  RailsEnv production

  SSLEngine on
  SSLCertificateFile /etc/apache2/ssl/apache.pem
  CustomLog /var/log/apache2/test.log combined
</VirtualHost>

This is not a problem if we only ever deploy to a single environment (production). But what about that staging environment? The naive solution is to pollute the config directory with several apache config files (one for each deployable environment). Prior to Passenger, we used mongrel_cluster, which causes basically the same problem when you need to keep distinct copies for separate environments. Several other plugins/gems require configuration as well that will not necessarily be the same for all of your deployable environments.

The default config directory of a rails app looks like:

config/
  boot.rb
  database.yml
  environment.rb
  environments/
    development.rb
    production.rb
    staging.rb
    test.rb
  initializers/
  routes.rb

One option would be to add more top-level folders (similar to environments/) for each of these pieces of configuration:

config/
  boot.rb
  environment.rb
  apaches/
    production.conf
    staging.conf
  databases/
    development.yml
    production.yml
    staging.yml
    test.yml
  environments/
    development.rb
    production.rb
    staging.rb
    test.rb    
  initializers/
  routes.rb

I don’t know why, but this just feels wrong to me. I like to rearrange my directory structure such that I have a directory for each of my environments:

config/
  boot.rb
  development/
    database.yml
    environment.rb
  environment.rb
  initializers/
  production/
    apache.conf
    database.yml
    environment.rb
  routes.rb
  staging/
    apache.conf
    database.yml
    environment.rb
  test/
    database.yml
    environment.rb

Now, each of my environments has its specific settings grouped together. It’s cleanly organized and obvious when looking at the contents of the config directory which environments exist and where their configurations are stored. There are two things in particular to note:


1) I now have 4 database.yml files. You might argue that this isn’t DRY, but in reality I have exactly the same number of lines of code written as I would have had I used a single database.yml file. e.g. config/production/database.yml looks like:

production:
  adapter: mysql
  database: myapp
  username: myapp_user
  password: supersecret
  socket:  /var/run/mysqld/mysqld.sock

Furthermore, this file can (and should) be checked into Subversion! And developers won’t have to mess with svn:ignores et al. (Note that config/test/database.yml and config/development/database.yml do not belong in the repo.)

2) I have changed the name and location of each of my environment-specific configuration files. e.g. config/environments/development.rb is now located at config/development/environment.rb.

So, now our config directory is a little better organized. Unfortunately, rails is going to complain about not being able to find config/database.yml and config/environments/development.rb. All we need to do is override where rails looks for them, and we’ll be good to go.

I create a new file config/postboot.rb:

# Be sure to restart your server when you modify this file.

rails_env = ENV['RAILS_ENV'] || 'development'

env_dir  = File.join(RAILS_ROOT, 'config', rails_env)
db_file  = File.join(env_dir, 'database.yml')
env_file = File.join(env_dir, 'environment.rb')

raise "#{env_dir} environment directory cannot be found." unless File.exists?(env_dir)
raise "#{db_file} is missing.  You cannot continue without this." unless File.exists?(db_file)
raise "#{env_file} environment file is missing." unless File.exists?(env_file)

# Now, let's open up Rails and tell it to find our environment files elsewhere.
module Rails
  class Configuration
    
    # Tell rails our database.yml file is elsewhere
    def database_configuration_file
      File.join(root_path, 'config', environment, 'database.yml')
    end
    
    # Tell rails our environment file is elsewhere
    def environment_path
      "#{root_path}/config/#{environment}/environment.rb"
    end
  end
end

Lastly, I need to hook it in right after boot.rb is run. As such, I add a require for the file just after boot.rb is required in config/environment.rb:

# Be sure to restart your web server when you modify this file.

# Uncomment below to force Rails into production mode when 
# you don't control web/app server and can't set it the proper way
# ENV['RAILS_ENV'] ||= 'production'

# Specifies gem version of Rails to use when vendor/rails is not present
RAILS_GEM_VERSION = '2.0.2' unless defined? RAILS_GEM_VERSION

# Bootstrap the Rails environment, frameworks, and default configuration
require File.join(File.dirname(__FILE__), 'boot')

# Pull in the postboot initializer
require File.join(File.dirname(__FILE__), 'postboot')

Rails::Initializer.run do |config|
...

Now, our rails app is ready to go! I have not yet created a script to automate all of this. As I compile resources for my presentation next week, I will include a script to do just this…..and obviously share it with you then!

  • rick

    My approach to this issue is a bit different. After working with some smart sysadmins on a certain client project, and working with Engineyard, one common thread popped up: Using git to store things like your various server config files. In the same way, you can also store your production specific database.yml and any config initializers in git also. You can symlink these in the after callback for deploy:symlink_configs.

    I only like my approach slightly better because a) it doesn’t require any changes to the framework or app config files, and b) I’m not checking those files into the same git repo as my application.

  • http://www.smartlogicsolutions.com/wiki/John_Trupiano John Trupiano

    Nice– so you more or less simulate an svn:externals. I do like keeping configuration out of the same repository, but does it add extra steps when a new developer hops onto the project?

    People raised some interesting points at Bmore on Rails. One I particularly liked was completely leaving SCM off of production servers by doing local checkouts and rsync’ing. Your procedure could fit in here when the local copy is checked out. Another thing to point out is that I’m currently leaving all of my non-production configuration on the production box (shame on me). It’s not simple to restrict a single directory from being checked out (an ignore won’t work, because you want those files in dev). I suppose your process would also aid me here.

  • rick

    New developers have to create a database.yml (we have an app:bootstrap task that sets up the database.yml, the schema, and any initial data that’s needed for a quick script/server demo), but that’s it. Usually they don’t need the staging or production info.

    But who knows, we have maybe 5 developers in my small company, usually all working on different things. I imagine the practices would have to evolve as the size of the team grows.

  • Pingback: SmartLogic Solutions Blog » Blog Archive » Introducing environmentalize: an intuitive, environment-focused config structure for your rails applications

  • Sam

    This is a very easy and simple way to separate the environment database files. I would make one fix to anyone using this method, in config/postboot.rb, exit should not be used and raise should be. Raising allows rails to produce a trace and will not break console interactions within Aptana.

    So instead of the unless… exit(1) end sections, use:
    raise “Environment directory `#{env_dir}’ does not exist” unless File.directory?(env_dir)
    raise “Environment file `#{env_file}’ does not exist” unless File.exists?(env_file)
    raise “Database file `#{db_file}’ does not exist” unless File.exists?(db_file)

  • http://www.smartlogicsolutions.com/wiki/John_Trupiano John Trupiano

    Hi Sam,

    Thanks for the suggestion to use raise in lieu of exit. It is valuable to maintain the stack trace. I’ve updated the post (and the environmentalist library at http://github.com/jtrupiano/environmentalist) to reflect this change.

    -John

  • http://www.tonyamoyal.com/blog Tony Amoyal

    Hey John,

    You raise some interesting points here and this is a well written post. I am currently building and deploying my first production rails app.

    However, something really bothers me about keeping a production config in the repo. Mainly that I don’t believe every developer should have access to production credentials. I plan to implement a method close to the one described in “Agile Web Development with Rails”. I will keep the database.yml out of the repo and put it in a protected directory on my server then use a capistrano hook to copy the database.yml to the prod repo after deployment.

    I think following this route will allow me to better secure the production credentials.

John Trupiano co-founded SmartLogic with Yair Flicker in May 2005 and was co-president through 2011. Check out his GitHub Projects or follow @jtrupiano on Twitter.

John Trupiano's posts