SmartLogic Logo (443) 451-3001

The SmartLogic Blog

SmartLogic is a web and mobile product development studio based in Baltimore. Contact us for help building your product or visit our website to learn more about what we do.

Nick’s Highlights from Windy City Rails

September 11th, 2010 by

I attended and spoke at Windy City Rails. I tried to take more notes this time. Out of 6 talks an lightning talks, I have ~500 lines of notes. Enjoy!.

Jake Scrugges – Metrics Based Refactoring: What To Do With Your Code Metrics

  • Metric-Fu is a ruby metric bundle
  • rake metrics:all
    => generates a master report of a bunch of metrics
  • There are a lot of metrics, where do you start?
  • Everyone wants to refactor, but where do you start?
  • Flay
    • Analyzes code duplication
    • Finds violations of DRY
    • Solutions: Extract Method, Modules, etc.
    • Single Responsibility Principle
  • Churn
    • Looks through your commits to see what stuff changes the most
    • Files that change the most are “God Objects” – stuff that does everything
    • Any time anything changes, the God Object has to change
    • Bad smell, should be smaller objects with greater focus
    • Tells you where your most volatile code is
  • Rcov
    • Main reason metric-fu blows up
    • Problem is, metric-fu is mostly static analysis, but rcov is running the code
    • Rcov runs the test suite, which is sometimes hard to get working
    • Rcov makes sure your coverage is good where it needs to be (most complicated and volatile code)
    • Writing tests for the sake of coverage is harmful
    • Write tests when you are in the code and you understand it (not a week later, “let’s write some tests!”)
  • Flog and Saikuro
    • Both are complexity management
    • Saikuro (pronounced cyclo as in cyclomatic complexity) measures cyclomatic complexity
    • Saikuro = number of paths through the code, and cyclomatic complexity is exponential in growth
    • Flow looks at branching, but uses ABC
    • Assignments
    • Branches
    • Calls
    • Flog generates a score based on ABC
    • < 20 is good, 20-40 is a grey area, 40-60 is a warning, 60+ is FIX IT!
    • How do you fix it?
    • Extract method
    • Missing object (need a new class to handle this behavior)
    • If that doesn’t work, re-architect
    • Seriously, you don’t need complicated methods in any app
    • Director method: calls a couple of sub methods, doesn’t do much actual work
    • Take care not to over-factor. If you are a guest in the code, just clean it
      as necessary. You don’t need to fully re-organize.
  • Reek, Roodi, and RailsBestPractices
    • Reek: if it could possibly be a problem, I’ll tell you
    • Roodi: only if I’m really sure, I’ll tell you
    • RailsBestPractices: just rails stuff
    • All parse the code and look for design problems
    • Very ambitious! Code trying to understand what you’re trying to do
    • Awesome because you can look up the definition of the problem and see how to fix it.
    • Trouble because it’s a machine trying to identify human problems, which is difficult
  • Continuous Integration
    • Metric-fu on your code
    • Most CI’s allow you to have artifacts for a build
    • J.S. runs his nightly at midnight
  • Pay attention to trends over time
    • Is your refactoring making things better?
    • Did the last feature skyrocket flay scores?
    • How is the influx of new developers affecting the code quality
    • Did the highly paid consultants produce good work?
  • Take metrics as advice (not law)
    • It is a tool, use it intelligently (like a GPS system)
    • Don’t let a manager get in charge of this
    • Up to you and your team to establish conventions and stick to them
  • Coming soon
    • Fail the build on bad metrics
    • Resisted for a while because there may be false negatives
    • Failing a build is a serious incident, needs to mean something
    • Metrics are not as definitive as tests
  • Coming later (and/or soonish)
    • Meta metrics
    • Imagine a report that flagged a method as having high churn, poor coverage, and high complexity
    • This method is trouble waiting to happen
    • No so easy to write this feature
    • J.S. needs help!
    • Need to re-architect the system to understand classes and methods, not just lines of code
  • Questions
    • Can you use other source control?
    • I think so, check out the churn gem
    • How is the performance?
    • Not great, but you only have to run it once a day
    • If you don’t have test coverage, now what?
    • start w/ rehacktoring (refactoring w/o tests)
    • do that until you understand it enough to write tests
    • write tests!
    • What about front-end heavy stuff? Javascript?
    • No idea! These are all ruby tools. Maybe there are JS tools out there?
    • You should test your javascript. Sapphire is the “new hot thing”.

John McCaffrey – Analyzing and Improving the Performance of your Rails Application

  • http://railsperformance.blogspot.com
  • O(n) queries is bad
  • Code he refactored went from 2811 to 17 queries per action
  • Leverage: small amounts of work for large amounts of value
  • Delay directly correlates w/ loss of revenue
    • Bing: 1s = 2.8% decrease, 2s = 4.3%
    • Google: users that had a delay for a short period of time, actually searched less even after the delay was removed.
    • Yahoo: added a 400ms delay and users bailed before the pages even loaded
    • Shopzilla: reduced delay from 6s to 1.2s.
    • Conversion rate rate up 7-12%
    • Saved infrastructure costs by 50%
    • Release cost decreased
  • Fred Wilson (VC): Speed is #1 of the 10 golden principles of successful web apps
  • So where to begin?
  • Anti-patterns:
    • “Scalability can be sprinkled on later”
    • Guessing, not testing
    • Ad-hoc, unstructured
  • “Premature optimization is the root of all evil” – Donald Knuth
  • “If you can not measure it, you can not improve it” – Lord Kelvin
  • Measure
  • Repeatable tests
  • Isolate your changes
  • (Scientific method!)
  • Visibility is important for working on a team
  • Response time = latency
  • Requests per second = throughput
  • Load, utilization, scalability, throughput, concurrency, capacity
  • Performance != Scalability
    • Performance = response time
    • Scalability = ability to grow to more requests
    • Adding more workers will not improve latency
  • Twitter: dynamic html = 250ms, but 3369ms is static assets
    • Don’t waste your time on the 250ms, go for the big slow chunks
  • Walmart: 3 redirects, 388ms dynamic, 4580ms static content
  • Use YSlow
    • Do less work
    • Distribute content
    • Cache
    • Compress
    • Fight queued work
  • Google page speed
    • similar to yslow
    • includes paint events
    • separates out ads and trackers from your site
    • “how fast it feels”
    • If the web is faster, they make more money
  • http://www.showslow.com/ tracks your site (or competitors, will check every 24h)
  • http://webpagetest.com/ runs page speed and yslow for you, and show progressive rendering
  • http://gtmetrix.org
  • http://zoompf.com
  • http://loadimpact.com
  • http://gomez.com
  • See Greg Pollack’s Scaling Rails series
  • Majority of the Alexa top 1,000 sites don’t do even the easiest things
    • 42% don’t gzip
    • 44% >2 css files
    • 56% serve css from a cookied domain
    • 62% don’t minify
  • Gzip is the simplest and smartest thing you can do
  • Minify js
    • jsmin, yuicompressor, sprockets
    • Asset packager
    • javascript_include_tag :cache => “cache/all”
  • Sprited images
  • Spriteme.org
  • JqueryUI gives you sprited images
  • Image optimization
    • Smush.it
  • Expires and browser caching
  • Set a far-future expire to tell the browser it can cache it
  • Combined with a query string lets you version a file
  • Rails Caching: make the static parts of your dynamic content cached
  • railslab.newrelic.com
  • request-log-analyzer
  • Rack::Bug
  • NewRelic
  • Scout
  • Get on the heroku mailing list / blog to learn about scaling issues
  • Apache bench
  • httperf
  • jmeter
  • Database issues
  • http://github.com/eladmeidar/rails_indexes
    • rake tasks to find missing indexes
    • columns that need to be sorted
    • lookup fields
    • columns used in a group-by
  • http://github.com/samdanavia/ambitions_query_indexer haven’t tried it yet, check it out
  • http://github.com/sdsykes/slim_scrooge Instruments your code and looks for bad “select *”
  • Aman Gupta
  • Joe Damato
  • http://timetobleed.com/
  • http://memprof.com/
  • yajl-ruby for C++ extension json parsing
  • Ruby 1.9 for massive performance

Kevin Gisi – It’s time to repay your debt

  • We’re leveraging an awesome framework and language, and it’s time to give back
  • Releasing gems is easy
  • (Showing some code to refactor out into a gem)
  • Ruby community has an incredible (and overwhelming) amount of choice
  • Choice is good and important
  • There exist incomplete and unstable solutions
  • You should release a “product”, not an incomplete solution
    • Aesthetics
    • Utility
  • Agile: Do the bare minimum to get something delivered
    • (Nick: isn’t it to embrace change?)
  • Delivered: coded, deployed, supported, tested, extensible, usable by the client
  • Treat open source libraries and client deliverables
  • The Lazy programmer’s manifesto
    • I want to write code once
    • I want to be able to read code w/o docs
    • I want to be able to rely on libraries
    • I wants to use adaptable solutions for numerous problems
    • I want to always be working on something new
  • Idea filtering
    • To share
    • business value
    • practical use
    • To learn
    • no requirements
    • don’t post it publicly (community does not value your stupid web framework)
  • Testing
    • We all say we do it (it’s time to start)
    • RSpec and Cucumber are easy to set up
    • Design verification tool
  • (really turning into a rant at this point)
  • Extensible
    • Write clean code (don’t post spikes)
    • Peer review
    • Follow the Unix philosophy: do one thing really well
    • Leverage other gems
  • Documentation
    • Who’s looking at your code?
    • RDoc, YARD, etc
    • Getting Started
    • Blog post!
  • Supported
    • If you can’t maintain your app, don’t post it
  • Be Honest
    • Hand over the reigns if you can’t support it
    • Be a net positive
    • Don’t announce things that aren’t done
  • If you have to
    • Put it in a limited venue
    • Add a disclaimer
  • The internet doesn’t filter bad ideas very well
    • It does make things hard to do if they don’t work well
    • Having enough eyes
  • “Ruby treats you like a grown up programmer” – Matz
  • What happens when grown-ups have too much fun?
  • How to be a good parent
    • Avoid sensory overload
    • Sometimes spoon feed
    • Remember you’re used to the status quo
    • Be a mentor http://railsmentors.org
    • Provide solutions to problems, not blocks of code
  • (I wish this was a “how to do a good job” and not “you suck, stop sucking”. I think he wants to be the king of the rubygems app store)

Lightning Talks

Micah Martin – Limelight

  • Tool to build GUIs in ruby
  • Built a gui calculator in 5 minutes

Sean Scofield – The Awesome Power of Rails Engines

  • Engines make it easy for frameworks to be built on top of rails
  • Engine is a complete application that can provide the stuff rails apps provide
  • app directory is auto loaded
  • You can supply models, views, controllers in your engine
  • You can override locale by dropping it in app
  • Engines extending engines, for example Spree
  • Bundler ties it all together

Mike Buselli – Brush::Pipeline

  • Running external OS commands
  • sys
    • lower level, more convenient for specific takss
    • chaining commands
  • pipeline
    • better at chaining commands
    • pipeline ['head', 'README'], ['tail', '-n2']
    • Each part of the pipeline can run in different directories
    • stderr and stdout redirection
    • argv[0] renaming
    • restrict access to open files
    • Works great on POSIX including cygwin

Chris Hallendy and Nick Lewis – Realtime Apps

  • Demo of node.js <-> Redis <-> Rails

Joe Fiorini (sp?) – Web Katas

  • Problem: web developers don’t know http
  • Rails is big and awesome, but it’s hard to troubleshoot
  • It allows us to ignore how http works.
  • Code Katas http://bit.ly/codekatas
  • No-framework webapp development (Rack) to learn how http cookies work
  • More Katas

Jim Remsik – Improv: Comedy to Coding

  • Programming is a people problem
  • You write code for other people
  • We use ruby because it is expressive and easy for others to read and maintain
  • Improv after-hours at hashrock
  • Yes-and, drop your preconcieved notions at the door
  • Support your team

Malcolm Arnold – Ruby Nuby

  • Brand new social movement dedicated to teaching ruby on rails to disadvantaged youths
  • Maximize social good by spreading ruby

Praveen Alavilli – Monetizing your apps

  • “chase your dream, money will follow”
  • Still a mystery to a lot of people how the money follows
  • Indirect
    • advertising
    • offers
    • referalls
  • Direct
    • e/m-commerce
    • freemium
    • causium (donate money to a cause)
    • pay as you use
    • free to use pay for service
    • premium content
    • digital goods
    • virtual currency
    • subscriptions
  • Challenge: apply these models to your use cases
  • Gateway (i.e. Payflow, Authorize.net)
  • Checkout (Google, PayPal, Amazon)
  • Platform (PayPal, Amazon)
  • Also services on top (Chargify, Spreedly, Freshbooks)
  • Choose one that works the best
  • Your users need to be able to pay somehow, should not be hard to pay
  • PCI compliance for charging directly
  • secure and privacy enabling
  • Adaptive Payments API

Ryan Singer – Weaving Design and Development

  • UI Design and Programming, working together
  • (telling a story about working with a startup and doing design for them)
  • Designer says “let’s change this button” while pairing with dev, dev makes the change instantly
  • XP is focused on delivery
  • In XP, feedback loops are not as important
  • Feedback can be difficult if you are purely focused on delivery
  • Shared code base, code runs on designers computers
  • Avoid the designer w/ folders full of PSDs and static content
    • Becomes a spec, then the dev has to implement it
  • Static files are like spikes, for exploration and experimentation
  • Actual changes need to keep up with the codebase
  • At 37Signals, no sync problem, because the designers are working in the code base
  • HAML and Mustache are out, because designers can’t deal with them
  • Templates with logic
  • Avoid helpers writing HTML and presenters writing html
  • Start together, don’t bring the designers in late, or the developers in late
  • Architecture before design leaves UX to too late in the process
  • Designers starting everything before programmers start, it may be impossible or impractical to code
  • P&D decide on domain language and basic model
  • P stub models, controllers, routes, and templates
  • D few days lead on designing and building templates
  • Simultaneous
    • P design and build models, controllers, routes, helpers, dynamic templates
    • D design and build templates and helpers
    • Repeat this process continuously
  • Share the codebase
  • All views are in templates w/ logic (ERB or similar, not HAML)
  • Start together for domain language and models for communication
  • constantly committing to the same codebase
  • Train designers in git and ruby and other things
  • Designers are motivated to learn so they can make changes directly: it gives them more power

Nick Gauthier – Grease your Suite

Les Hill and Jim Remsik – Sustainably Awesome

  • Comfortable and productive environment is crucial
  • sustainable pace is important. 40h weeks.
  • cargo culting
  • Pet friendly
  • The point is not to do what we do, but to recognize our goals. Don’t cargo cult these tips!
  • Casual
  • No Micromanagement
  • Don’t penny pinch
  • Keep a budget and spend it wisely
  • $400 per week grocery budget
  • No death marches
  • No meetings (except daily standup and project standup and monthly company status meeting)
  • No hierarchy
  • No future-proofing (software and organizational practices)
  • What do you do when a developer rm -rfs?
    • No sudo for devs?
    • 3 manager approval before running a script?
    • Develop a backup solution and a common machine image?
  • “Process is an embedded reaction to prior stupidity” – Clay Shirky
  • A project came along that needed work done ASAP
  • Crafted an MVP and built it without changing their process
  • The MVP did not overlap with their mockups and specs
  • Started with story cards and team boards
  • Moved to pivotal tracker
  • Tim Pope had a suggestion about how to work stories in pivotal that worked really well
  • Always open to try new things and experiment with new processes
  • Enjoy your work
  • Don’t hire people you don’t like, or without passion
  • Hire smart people that get things done
  • Must be a great cultural fit
  • Hiring
    • phone screening
    • week long “test drive”
    • gems you use are important
    • editors you use are important
    • test driven (and test-first) style
    • hang out after hours to check for a cultural fit
  • “If I’m not saying ‘hell yeah!’ about something, then I say no” – Derek Sivers
  • Firing: due to lack of communication or mistake, probably on both sides
  • Fire fast, but with compassion
  • Approach building a team the same way you build software

Yehuda Katz – Rails 3

  • Rails core family is getting bigger
  • Rails 2 was very intertwined
  • Broke things up cleanly
  • Instead of Railties being a big glue layer w/ initialization stack, it became a small part or Rails 3 and specifies the framework for applications and engines
  • Engines are special plugins now that inherit from Railties
  • Applications inherit from Engines
  • Railtie lets you add features to rails
  • Railties provide a series of Hooks
  • Inheriting from Railtie doesn’t do anything, but it has lots of functionality
  • Railtie lets you configure the application, such as pull in generators
  • Rake Tasks are another Railtie Hook
    • Extra stuff you want to have happen in rake, and not on boot
      rake_tasks do
      load “my_stuff/my_tasks.rake”
      end
  • before_configuration hook = right inside Rails::Application, before user stuff happens, but you know about rails root and stuff.
  • config.to_prepare : in dev, run every time, in production, run once (i.e. compass template compilation)
  • Framework loading: ActiveSupport.on_load = i.e. when action mailer loads, monkey patch in some code. Then you don’t have to try to detect ActionMailer, you just say, when it loads do stuff.
  • Don’t do if(defined) because it depends on order
  • Decouple via Instrumentation
  • ActiveSupport::Notifications.instrument(“receive.action_mailer”) do |payload|; #callback; end
  • Worth Noting: we worked really hard making things mostly backwards compatible
  • The old way may appear to work, even if it doesn’t
  • AbstractController is decoupled from ActionController
  • ActionDispatch = Rack++
  • ActiveModel lets other persistence engines connect to ActionController. Only need to support ~5 methods
  • Tight Integration without Coupling
  • Integration first, decouple later
  • Migrating:
    1. Install Rails 2.3.9.
    2. bundler
    3. rails_xss
    4. Deprecations – get the count to 0
    5. Upgrade
    6. Deprecations again!
  • Helpers should call partials, not generate html
  • Rails 3.1: We’re not burned out!
    • Engines
    • @drogus did all the engine work
    • Engine Migrations
    • Static assets (w/o copying). Leave it in the engine, and Rack::Static will find it
    • Self-contained engine
    • Built-in http caching, to get away from filesystem caching
    • fresh_when? calculates an etag from the post and returns 304 if not fresh
    • caching at the browser layer
    • etag can be based on an AR object
    • If you have Varnish, you can turn off Rack::Cache and fresh_when will be built in
    • This also means that when you turn on Akamai, you already have all the code that makes it work well
    • Enables Edge-Cached SCSS
    • Rack::Cache is like Memcache, but uses HTTP semantics
    • Assets
    • ERB == SCSS (I have content that needs to be processed before it hits the browser)
    • Solution: build in compilation pipeline
    • app.css.scss (like app.html.erb)
    • posts/index.css.scss
    • Treats CSS the same as HTML for templating
    • SCSS template handler => http response => rack::cache
    • It just works w/ varnish and the like
    • Spriting
    • Compass and lemonade are merging w/ a new sprite api
    • Put all your images in images/sprites/ 1, 2, 3, 4, .png
    • Lets you include all the images as a sprite
    • Gives you a bunch of css classes to use the sprites
    • SCSS can take CSS, and then add stuff in
    • You don’t need imagemagick, there is a pure ruby PNG spriting library
    • AutoFlush – it’s cool, read his blog, running out of time
    • flush content to the browser as it is rendering
    • Tradeoffs you have to deal with, like exceptions
    • Browser can be working on your CSS while you are doing SQL queries
    • Keep content_for at the top of the template
    • Exceptron
    • Stolen from merb
    • ExceptionsController to render and take actions when exceptions happen
    • Performance
    • We did pretty well in 3.0, definitely some trouble spots
    • More mechanisms for performance
    • Big Arel rewrite for 3.1
    • Possible to get to 2x faster over 3.0
  • Dave Zachrich

    Nick,
    Thanks for posting. Your notes are a lot more complete than mine. I really enjoyed the conference and sharing conversation with you during lunch. Best wishes.
    Dave