Paluy on Miscellaneous

Of course I'm an optimist - I don't see much point in being anything else. - Winston Churchill

Rails Code Quality Tools

The only valid measurement of code quality: WTFs/minute
source

I would like to cover code quality tools I use while developing Rails applications.

Of course you should try CodeClimate service to monitor your project, but…

Let`s overview some open source solutions:

1. Rails Best Practises - RBP

It’s a static code analysis tool that checks the quality of rails code. It can be customized for your needs and even add your own checklist

I prefer to add RBP analysis to CI tasks, as following:

1
rails_best_practices -f html --template config/my_rails_best_practices.html.erb .

while I use my custom RBP template report config/my_rails_best_practices.html.erb. The RBP report is a part of my CI artifacts.

Alternative 1: ROODI - Ruby Object Oriented Design Inferometer. Usage: roodi -config=config/roodi.yml "app/**/*.rb" "lib/**/*.rb" > roodi_report.txt

Alternative 2: Rubocop

Alternative 3: Cane

Alternative 4: Ruby Lint

2. Code Coverage - SimpleCov

Add the following code to spec_helper.rb

1
2
3
4
5
6
7
8
require 'rubygems'

if ENV["COVERAGE"]
  require 'simplecov'
  SimpleCov.start 'rails'
end

# ...

Note: Add export COVERAGE=true before you run your tests.

3. Ruby Style Guide, Rails Style Guide and JavaScript Guide

Your team should apply those guides or define your own. You should also check those Github guides.

4. Rails Security Scanner - Brakeman

Note:

Read this blog post by CodeClimate

RSpec - Less Known Features

I would like to summarize some of less known RSpec feature I use during my development.

Execute specific test from command line

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# spec/controllers/user_controller_spec.rb
require 'spec_helper'

describe UsersController do

  before (:each) do
    @user = FactoryGirl.create(:user)
    sign_in @user
  end

  describe "GET 'show'" do

    it "should be successful" do
      get :show, id: @user.id
      response.should be_success
    end

    it "should find the right user" do
      get :show, id: @user.id
      assigns(:user).should == @user
    end

  end

end

Suppose you want to test only the test at line #17. You can append the line number to file pathrun:

1
rspec spec/controllers/user_controller_spec.rb:17

It will run only the test inside the scoped line. The same can be used with describe/context.

Temporary pending test

Suppose you need temporary disable a test at line #8. You can add x before it and the test is pending now.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
describe "GET 'show'" do

  it "should be successful" do
    get :show, id: @user.id
    response.should be_success
  end

  # THIS TEST IS PENDING NOW
  xit "should find the right user" do
    get :show, id: @user.id
    assigns(:user).should == @user
  end

end

Focus on specific test

Add a tag to the selected test:

1
2
3
4
5
6
7
8
9
10
11
12
13
describe "GET 'show'" do

  it "should be successful", focus: true do
    get :show, id: @user.id
    response.should be_success
  end

  it "should find the right user" do
    get :show, id: @user.id
    assigns(:user).should == @user
  end

end

Then, run rspec as following: rspec -t focus

Rails.logger stub

If you need to stub Rails logger, you should do the following:

1
2
3
4
5
6
7
8
9
10
# Expecting to receive Rails.logger.debug some_message
let(:logger_mock) { double("Rails.logger").as_null_object }
Rails.stub_chain(:logger, :debug).and_return(logger_mock)

# your test goes here
# ...
#

# Don't forget to unstub the logger at the end of test!
Rails.unstub(:logger)

More about stub_chain

SteelCity Ruby - My First International Ruby Conference

TL;DR

You have to visit at least one Ruby conference in USA! Check this summary

SteelCity Ruby

At the beginning of this year (2013) I decided that I have to participate in at least one international Ruby conference. I already participated in Development conferences in Europe and even did several talks. But I have never been in USA Ruby conference yet.

Some one tweeted about SteelCity Ruby conference and I decided to register immediately. I found amazing appartment via AirBnB just near the venue for a half price of a room in hotel.

View from my appartment

Pittsburgh is a very nice city with amazing Ruby community. They have a pub inside a church (The Church Brew Works)

The Church Brew Works

and Pittsburgh Sandwich with French fries

Pittsburgh Sandwich

But let’s go back to the Ruby. Did you know that Steve Klabnik lived in Pittsburgh till last year?

Ruby Friends - Steve Klabnik with David Paluy

I really like the “community feel” of this event. The organizers did amazing job! It’s all done by local volunteers with help of sponsors.

I was introduced to other at the beginning of first day, when the conference moderator asked, who did more than 10,000 miles to arrive to that event.

I met a lot of great developers and had some #RubyFriends pictures. BTW, if you still don’t have a RubyFriends picture, go to a local meetup and do at least one!

I had an interesting talk with founder of Gittip Chad Whitacre. He founded amazing product to support Open Source community.

Ruby Friends - Chad Whitacre with David Paluy

I also invited Avdi Grimm to come to Israel and give a talk in one of our events.

Ruby Friends - Avdi Grimm with David Paluy

Summary

More conference pictures and other info can be found here.

You can find the confernece videos here. But I would like to advise you to check at least those talks:

My Weekly Read

I have several weekly subscriptions that you, as a web software leader, should read too

  • Software Lead Weekly - Friday emails by Oren Ellenbogen, with good overview of interesting posts, relevant for Software Leaders.

Example: What Dan Ariely can teach us about Software Development

Other weeklies by Peter Cooper:

  1. JavaScript Weekly - all about JavaScript
  2. HTML5 Weekly - all about HTML5
  3. Ruby Weekly - all about Ruby

Another weeklies that could be relevant for some of you:

My other sources comes from:

What is yours?

Process Information

I found it very useful to see my application process info on an admin page, as following:

You can do it inside your Rails app.

Inside your controller/admin_controller.rb:

1
2
3
4
5
6
def index
  @processes = `ps -g #{Process.getsid($$)} -o pid,wchan,%cpu,%mem,rss,vsz,command`.strip.split("\n")
  @proc_headers = @processes.shift.split(/\s+/)
  @processes.map! { |line| line.strip.split(/\s+/, @proc_headers.size) }.
      reject! { |line| line.last.start_with?('ps -g') }
end

and add the following to your view/admin/index.html.erb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<table>
  <thead>
    <tr>
    <% @proc_headers.each do |th| %>
      <th><%= th %></th>
    <% end %>
    </tr>
  </thead>
  <tbody>
    <% @processes.each do |line| %>
      <tr>
      <% line.each do |td| %>
        <td><%= td %></td>
      <% end %>
      </tr>
    <% end %>
  </tbody>
</table>

I would like to thank @romanbsd for this source.

Protect Email From Spam Bots

TL;DR

Spam bots are getting a lot smarter at harvesting email addresses from web pages. Use encode option while using mail_to

Ways to protect your email: me@domain.com

  • Weak solution - substitute characters: me_at_domain_dot_com
  • Weak solution - HEX encoding: %6d%65@%64%6f%6d%61%69%6e.%63%6f%6d
  • Better solution - Javascript encoding:
1
2
3
4
5
mail_to "me@domain.com", "Contact us", :encode => "javascript"
# => <script type="text/javascript">eval(decodeURIComponent('%64%6f%63...%27%29%3b'))</script>
#
# The JavaScript evaluates the following:
# document.write('<a href=\"mailto:me@domain.com\">Contact us<\/a>');

Check more details here.

Another way to encode your mail_to with JavaScript

Email obfuscator script 2.1 by Tim Williams

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<script type="text/javascript" language="javascript">
<!--
// Email obfuscator script 2.1 by Tim Williams, University of Arizona
// Random encryption key feature by Andrew Moulden, Site Engineering Ltd
// This code is freeware provided these four comment lines remain intact
// A wizard to generate this code is at http://www.jottings.com/obfuscator/
{
  coded = "B6@LSBeJv.ASB"
  key = "78ncPmS9z0hseXrvAKBlgi1OC6tjYU2qdIJw5MVfQyb3ZLH4ExRWNTpoFkuGDa"
  shift=coded.length
  link=""
  for (i=0; i<coded.length; i++) {
    if (key.indexOf(coded.charAt(i))==-1) {
      ltr = coded.charAt(i)
      link += (ltr)
    }
    else {
      ltr = (key.indexOf(coded.charAt(i))-shift+key.length) % key.length
      link += (key.charAt(ltr))
    }
  }
  document.write("<a href='mailto:"+link+"'>Contact us</a>")
}
//-->
</script><noscript>Sorry, you need Javascript on to email me.</noscript>

Important! You can’t protect your email 100%

If the bot evaluates JavaScript before harvesting email addresses, you can’t resist it.

For example, it can be done using PhantomJS

Caution - To_time in Rails

TL;DR

Attention on time zone while using today.to_time in your Rails application!

Ruby to_time in your Rails application

Suppose, you need to convert some date to datetime. You can use the Ruby to_time method:

1
2
3
4
5
6
7
date = Date.new(2013, 04, 15)

date.to_time            # 2013-04-15 00:00:00 +0300
date.to_time(:local)    # 2013-04-15 00:00:00 +0300
# The Time Zone depends on your local machine

date.to_time(:utc)      # 2013-04-15 00:00:00 +0000

If your server time zone is UTC, but Rails application has different time zone, for example:

1
config.time_zone = 'Central Time (US & Canada)'

converting from date to time will ignore Rails settings!

It happens, when you need to output your date in epoch format (for example: in json).

I suggest to convert your date to utc and than to epoch.

1
epoch = date.to_time(:utc).to_i

Then, apply local time zone using JavaScript::

1
var date = new Date(epoch);

Profiling Rails Applications

Your finaly got clients and everything works fine.

But users expect several performance issues.

How can I profile my Ruby on Rails application?

Available Tools

GC::Profiler

The GC profiler provides access to information on GC runs including time, length and object space size.

1
2
3
4
5
6
7
8
GC.start # clear GC before profiling
GC::Profiler.enable

# your code

GC.start
puts GC::Profiler.report
GC::Profiler.disable

Ruby Benchmark

Ruby has core module to Benchmark your code. It helps you to mesure running time of your code.

1
2
3
4
5
6
7
8
9
10
11
12
13
require 'benchmark'

Benchmark.bm do|b|
  b.report("+= ") do
    a = ""
    500_000.times { a += "." }
  end

  b.report("<< ") do
    a = ""
    500_000.times { a << "." }
  end
end

I suggest you to add benchmark_suite gem. There is a set of enhancements to the standard library benchmark.rb

ObjectSpace

ObjectSpace class already described in previous post. You should also check Visualizing memory leaks in Ruby using ObjectSpace methods.

RUBY-PROF

How to use in Rails:

  • Add the ruby-prof to your gemfile
1
2
3
group :profile do
  gem 'ruby-prof'
end

Note: verify your /config/environments/profile.rb has the following settings:

1
2
config.cache_classes = true
config.cache_template_loading = true
  • Add rack adapter to config.ru:
1
2
3
if Rails.env.profile?
  use Rack::RubyProf, :path => '/temp/profile'
end

Note: By default the rack adapter will generate a html call graph report and flat text report. You can add printer option as following:

1
use Rack::RubyProf, :path => '/temp/profile', :printers => {RubyProf::GraphHtmlPrinter => 'graph.html'} # or any other printer as described

More printers options can be found here

  • Add Profile to your code:
1
2
3
4
5
6
7
8
9
# define mesurment type
# as described: https://github.com/ruby-prof/ruby-prof#measurements
RubyProf.measure_mode = RubyProf::PROCESS_TIME

RubyProf.start

# code to profile

result = RubyProf.stop

Perftools.rb

While ruby-prof is a good tool, I, personally, prefere to use perftools.rb. It also has a rack gem.

The main advantage of perftools: it samples your process using setitimer() so it can be used in production with minimal overhead.

How to use in Rails:

  • Add the rack-perftools_profiler to your gemfile
1
gem 'rack-perftools_profiler', :require => 'rack/perftools_profiler'
  • Add middleware to config/environment.rb:
1
config.middleware.use ::Rack::PerftoolsProfiler, :default_printer => 'pdf', :bundler => true

See more options here.

  • Profile your application
1
2
3
curl http://localhost:3000/__start__
curl http://localhost:3000/__stop__
curl http://localhost:3000/__data__

Visualization of your report

  • ruby-prof and perftools.rb has support for different output formats. One of them is KCacheGrind
  • In Ruby 2 you can gather your stack traces. Read more here

Summary

  • Remember: More Objects => Longer GC => Slow application
  • Always examine your HEAP
  • Use Tools!

Debugging Rails Application - Part 2

In previous post, I described which tools I use for debugging Rails (Ruby) applications.

In this post I will show you how to debug your Garbage Collector.

Why should I debug my GC?

First, you should understand how ruby allocates memory for new objects. What happened inside your OS, each time you write book = Book.new?

How Ruby Memory Works

RVM defines Process Heap for each Ruby process. Than, it creates first Ruby Heap. Each Ruby Heap contains allocated objects. There is a “Free List” array inside the heap.

Ruby Heap - Free List

While new object allocated, the RVM looks for an empty place in “Free List” and allocates it for a newly creted one. If the “Free List” is empty (all slots are allocated), RVM calls to Garbage Collector (GC).

The Garbage Collector finds non-reachable objects and remove them. If “Free List” is still empty, another Heap allocated.

Conclusion: The more objects we have, the longer GC process. And it’s a cause for slow application.

How should I debug my GC?

I suggest to use pry-debugger.

There are several tools we can use while debugging:

  • ObjectSpace class
  • Heap dump (via GC patch)

But to get it all functionality you need to use patched version of Ruby.

To enable heap dump support, pass the -enable-gcdebug option to the rvm install command:

1
rvm install 1.9.3 -C --enable-gcdebug -n gcdebug

ObjectSpace class

An example of method I used to get a better output:

1
2
3
4
5
6
7
8
9
10
11
12
13
def get_object_stats
  return_value = Hash.new
  ObjectSpace::each_object(Object) {|my_object|
    unless return_value[my_object.class.to_s.downcase].nil?
      return_value[my_object.class.to_s.downcase][:count] += 1
    else
      return_value[my_object.class.to_s.downcase] = Hash.new
      return_value[my_object.class.to_s.downcase][:name] = my_object.class
      return_value[my_object.class.to_s.downcase][:count] = 1
    end
  }
  return_value.sort_by {|k,v| -v[:count]}
end

Source

Heap dump

1
2
3
4
5
6
7
8
GC.enable_stats if defined?(GC) && GC.respond_to?(:enable_stats)
GC.start # clean up heap
# ...
# YOUR CODE GOES HERE
# ...
# dump it, including class names of heap objects
GC.dump_file_and_line_info(filename: "heap.dump", include_class_names: true)
stat_string = "allocated: #{GC.allocated_size/1024}K total in #{GC.num_allocations} allocations, GC calls: #{GC.collections}, GC time: #{GC.time / 1000} msec"

Other interesting solutions to check:

Note: the memprof works only with Ruby 1.8.x

My slides at RubyUnderground