Rails performance scripts: profiler and benchmarker

Filed Under (Performance, Rails, Ruby) by Leonardo Borges on 20-11-2008

Tagged Under : , ,

There are several ways you can measure your rails application’s performance. The techniques range from filling your code with “puts” statements - :p - to fancy ones like NewRelic - which is quite nice, I must say.

But what many people don’t know is that rails ships with a handful of scripts to help you out. One of which is called profiler, located under your application’s scripts/performance directory.

By default it uses the standard ruby profiler but if you want more speed - and additional reporting options - , consider installing the ruby-prof gem.

So if you execute it without params, you’ll get a clue of how it works:

$ script/performance/profiler
Usage: ./script/performance/profiler 'Person.expensive_method(10)' [times] [flat|graph|graph_html]

Pretty self explanatory, right?

As a sample code, I have in my rails app a dumb model with a really dumb method I wanna profile:

class Article < ActiveRecord::Base
  def self.find_all_with_delay
    sleep 10
    self.find(:all)
  end
end

Clearly this method doesn’t perform well and is a bottle neck in our super application! But let’s see what rails’ profiler tells us:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ script/performance/profiler 'Article.find_all_with_delay' 1 graph > text_graph.perf
Loading Rails...
Using the ruby-prof extension.
Thread ID: 109440
Total Time: 10.147995
 
  %total   %self     total      self      wait     child            calls   Name
--------------------------------------------------------------------------------
 100.00%   0.00%     10.15      0.00      0.00     10.15                1     Global#[No method] (/Users/leo/projects/test/vendor/rails/railties/lib/commands/performance/profiler.rb:24}  /Users/leo/projects/test/vendor/rails/railties/lib/commands/performance/profiler.rb:24
                     10.15      0.00      0.00     10.15              1/1     Object#profile_me
--------------------------------------------------------------------------------
                     10.15      0.00      0.00     10.15              1/1     Global#[No method]
 100.00%   0.00%     10.15      0.00      0.00     10.15                1     Object#profile_me ((eval):1}  (eval):1
                      0.00      0.00      0.00      0.00              1/1     Class#const_missing
                     10.15      0.00      0.00     10.15              1/1     <Class::Article(id: integer, name: string, content: string, created_at: datetime, updated_at: datetime)>#find_all_with_delay
--------------------------------------------------------------------------------
                     10.15      0.00      0.00     10.15              1/1     Object#profile_me
  99.97%   0.00%     10.15      0.00      0.00     10.15                1     <Class::Article(id: integer, name: string, content: string, created_at: datetime, updated_at: datetime)>#find_all_with_delay (/Users/leo/projects/test/app/models/article.rb:2}  /Users/leo/projects/test/app/models/article.rb:2
                      0.15      0.00      0.00      0.15              1/1     <Class::ActiveRecord::Base>#find
                     10.00     10.00      0.00      0.00              1/1     Kernel#sleep
--------------------------------------------------------------------------------
                     10.00     10.00      0.00      0.00              1/1     <Class::Article(id: integer, name: string, content: string, created_at: datetime, updated_at: datetime)>#find_all_with_delay
  98.54%  98.54%     10.00     10.00      0.00      0.00                1     Kernel#sleep (ruby_runtime:0}  ruby_runtime:0
--------------------------------------------------------------------------------
...

As you can see, the group around line 13 is where most of the time is spent, going through our stupid call to Kernel#sleep and detailing every and each call from the very beginning. The report is much larger, so I recommend you give it a try. It’s really useful.

Now, displeased with my method’s performance, I wrote a new one that I think performs much better:

  def self.find_all_with_less_delay
    sleep 5
    self.find(:all)
  end

Nice huh? :) But how can we be sure it performs better? It turns out that under scripts/performance there is another useful script: benchmarker

Again, running it without arguments reveals it’s usage:

$ script/performance/benchmarker 
Usage: ./script/performance/benchmarker [times] 'Person.expensive_way' 'Person.another_expensive_way' ...

So, ready to see which one of my methods performs better? Let’s check:

$ script/performance/benchmarker 1 'Article.find_all_with_delay' 'Article.find_all_with_less_delay'
            user     system      total        real
#1      0.020000   0.000000   0.020000 ( 10.016033)
#2      0.010000   0.000000   0.010000 (  5.015390)

Pretty neat way to benchmark your methods huh?
Profiler and benchmarker are a powerful combination that have been helping me a lot in the projects I’m working on.

Hope you like it! See u soon! ;)

Merb turns 1.0, and started driving me crazy

Filed Under (Merb, Ruby) by Leonardo Borges on 18-11-2008

Tagged Under : ,

As you may already know, Merb turned 1.0 a while ago and I decided to resume my studies to learn the framework.

So I just built a new app with a few resources and fired the migration scripts:

$ rake db:automigrate

And this is what I got:

Loading init file from /Users/leo/projects/merb/my-first-app/config/init.rb
Loading /Users/leo/projects/merb/my-first-app/config/environments/development.rb
 ~ Connecting to database...
/opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.6/lib/dm-core/adapters/data_objects_adapter.rb:137:in `initialize': wrong number of arguments (8 for 1) (ArgumentError)
	from /opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.6/lib/dm-core/adapters/data_objects_adapter.rb:137:in `new'
	from /opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.6/lib/dm-core/adapters/data_objects_adapter.rb:137:in `normalize_uri'
	from /opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.6/lib/dm-core/adapters/abstract_adapter.rb:44:in `initialize'
	from /opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.6/lib/dm-core/adapters/data_objects_adapter.rb:159:in `initialize'
	from /opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.6/lib/dm-core.rb:157:in `new'
	from /opt/local/lib/ruby/gems/1.8/gems/dm-core-0.9.6/lib/dm-core.rb:157:in `setup'
	from /opt/local/lib/ruby/gems/1.8/gems/merb_datamapper-1.0/lib/merb/orms/data_mapper/connection.rb:44:in `setup_connections'
	from /opt/local/lib/ruby/gems/1.8/gems/merb_datamapper-1.0/lib/merb/orms/data_mapper/connection.rb:27:in `connect'
	from /opt/local/lib/ruby/gems/1.8/gems/merb_datamapper-1.0/lib/merb_datamapper.rb:17:in `run'
	from /opt/local/lib/ruby/gems/1.8/gems/merb-core-1.0/lib/merb-core/bootloader.rb:99:in `run'
	from /opt/local/lib/ruby/gems/1.8/gems/merb-core-1.0/lib/merb-core/server.rb:172:in `bootup'
	from /opt/local/lib/ruby/gems/1.8/gems/merb-core-1.0/lib/merb-core/server.rb:42:in `start'
	from /opt/local/lib/ruby/gems/1.8/gems/merb-core-1.0/lib/merb-core.rb:169:in `start'
	from /opt/local/lib/ruby/gems/1.8/gems/merb-core-1.0/bin/merb:11
	from /opt/local/bin/merb:19:in `load'
	from /opt/local/bin/merb:19

Lovely, isn’t it?
After a fair amount of googling around it turns out that there seems to be a problem with the URI parser Merb uses underneath, called Addressable.

As of the installation of Merb 1.0, the installed version of this library was 2.0.0.
The solution? Install a older Addressable version:

$ sudo gem uninstall addressable
 
You have requested to uninstall the gem:
	addressable-2.0.0
dm-core-0.9.6 depends on [addressable (>= 1.0.4)]
data_objects-0.9.6 depends on [addressable (>= 1.0.3)]
If you remove this gems, one or more dependencies will not be met.
Continue with Uninstall? [Yn]  Y
Successfully uninstalled addressable-2.0.0
 
$ sudo gem install addressable -v 1.0.4
Successfully installed addressable-1.0.4
1 gem installed
Installing ri documentation for addressable-1.0.4...
Installing RDoc documentation for addressable-1.0.4...

I haven’t figured out the reason for this issue, so please do share if you know. I’ll be glad to update the post. :)

Understanding Ruby threads

Filed Under (Ruby) by Leonardo Borges on 28-10-2008

Tagged Under :

This post is just to clarify some confusion I’ve noticed reading some posts around the web. There is some misunderstanding of the differences between Ruby 1.8 and Ruby 1.9 regarding threads. And the difference between them and JRuby.

So I decided to write this small summary:

Ruby 1.8 - Supports only Green Threads

This means that the ruby interpreter has its own scheduler. No matter how many threads you create in your ruby program, there will be only one native thread on your OS. Therefore, your program cannot take advantage of multiple core environments.

Ruby 1.9 - Supports native threads (GIL)

Ruby 1.9 adopted YARV as the new VM implementation, which supports native threads. This means that now your ruby programs can take advantage of multiple core environments, but no truly parallel execution is achieved.

The catch is GIL - Global Interpreter Lock - and it means that each ruby thread runs on its own native thread, but only one of them can be executed at a time.

JRuby - Ruby 1.8 compatible

I think this is the easy part. JRuby runs on the Java VM, which supports native threads and parallel execution.

On this environment, ruby threads are java threads.

The future

There doesn’t seem to be any decision about what’s gonna happen on the next versions of Ruby. At least to the extent of my research. But this is today’s snapshot of Ruby threads and I hope it’ll be useful to some folks.

c u around

Don’t use REXML. I mean it.

Filed Under (Rails, Ruby) by Leonardo Borges on 08-10-2008

Tagged Under : ,

REXML is the standard XML processing library for Ruby. It’s on Ruby’s core and is terribly slow.

Yeah, I know it’s pretty simple to use, got a nice interface and, again, it’s just there. And it is a good library, for most things. But if you, as me, came to a point that processing XML is taking 50% of the time to render a rails action, it’s time to change.

My tip? Use libxml instead. The numbers on their home page speak for themselves. Try it yourself, you won’t be disappointed. And I’m really happy with the performance increase on our app.

RailsConf Europe 2008: impressions and highlights

Filed Under (Conferences, Rails, Ruby) by Leonardo Borges on 09-09-2008

I’m back in Madrid again after the RailsConf and I think it’s time to say something about it. :)

First off, the infrastructure provided by the conference was really great. The rooms, WiFi connection, food…  Really well organized.

Now to the sessions, highlights:

Tutorials (Tuesday)
- Meta-programming Ruby for fun and profit (Neal Ford, Patrick Farley)
The old and good techniques that made Ruby so powerful. Here Neal and Patrick walked us through the main tricks to meta programming like open classes - and conditionally open them - , dynamically define methods, sending messages to objects and how Ruby can help test your Java code in a much easier way.

I’ve put the link to the slides but honestly I don’t think they’re too much useful without the talking.

Sessions (Wednessday)
- EC2, MapReduce and Distributed processing (Jonathan Dahl)
Jonathan explained the theory behind MapReduce using very simple ruby examples, providing the basics on how to distribute and paralelize tasks accross multiple machines.

He also introduced Hadoop, a platform built in Java that “lets one easily write and run applications that process vast amounts of data”. What I liked the most was the simplicity he explained this subject. As of today, his presentation is not available online. Stay tuned as I’m gonna update this post with the links, as soon as they’re available.

Sessions (Thursday)
- Debugging & Testing the Web Tier (Neal Ford)
If you’ve been concerned about testing your app’s web tier lately, this presentation would probably not show you anything new. Neal talks about the need to debug and test javascript behaviour accross multiple browsers, using tools like Firebug, JSUnit and Selenium. If you have no idea about what these tools are, please stop now and go evaluate them!

We are pretty concerned about testing on my actual job, but selenium tests can be a pain sometimes - a.k.a extremely slow. And what ends up happening is that they are forgotten. Developers only run the test suite if it’s not painful and it’s lightning fast. Here’s is where the highlight for this session comes: CrossCheck.

The idea is to be able to test your javascript code accross multiple browsers without the need to launch them. In fact, you don’t even need a browser installed. The negative point is that it’s kinda fallen behind because now you can only test older versions of browsers. But since the project is getting a lot of traction, I’m pretty sure this will be solved soon.

Conclusion

My overall impression of the other sessions I attended is that some speakers just didn’t have time to properly prepare themselves, what made me think this years’s RailsConf wasn’t all that I expected.

But I also met interesting people and after all one of the key points in a conference is networking. :)

Definitely worth it though. And that’s why I took the time to provide this highlights.

c u soon

RailsConf Europe 2008: heading to berlin!

Filed Under (Conferences, Rails, Ruby) by Leonardo Borges on 30-08-2008

Tagged Under : , ,

The title says it already.

On monday I’ll be going to Berlin to attend this year´s RailsConf.

This will be my first one and of course my expectations are pretty high!

As usual, after the conference I’ll try and give a summary of what happened there, providing as much content as I can.

Anyone else’s going??? :)

C u there!

Mac OS X: Getting MySQL and Rails to work

Filed Under (Mac, Rails, Ruby) by Leonardo Borges on 28-08-2008

Tagged Under : , ,

So I couldn’t resist and bought myself a MacBook Pro! It’s my first week with my new toy and I’m really enjoying it.

But I need to do something useful with it so I started to prepare it to be my new development platform, starting with Ruby/Rails + MySQL: Here is where the fun begins!

After I installed both Rails and MySQL, I fired up a terminal an typed:

sudo gem install mysql

…and here is what u get

ERROR: Failed to build gem native extension.

If you google this error you will find a couple solutions and this is the one that worked for me:

ARCHFLAGS="-Os -arch x86_64 -fno-common"
sudo gem install mysql -- --with-mysql-dir=/usr/local/mysql
--with-mysql-config=/usr/local/mysql/bin/mysql_config

Now, confident enough, I created a sample rails app and tried to create the development database:

leo$ rake db:create (in /Users/leo/projects/test)
dyld: lazy symbol binding failed: Symbol not found: _mysql_init

Doesn’t look happy yet huh? This took me a while to figure out but it turned out to be fairly simple.
I have no idea why but after I installed the gem I had the file mysql.bundle in two different places:

/Library/Ruby/Gems/1.8/gems/mysql-2.7/lib/mysql.bundle
/Library/Ruby/Gems/1.8/gems/mysql-2.7/mysql.bundle

The solution was to remove the first copy of the file. Now everything is working fine at this end!
I really hope this is useful to someone!

Rails: Vulnerability on REXML

Filed Under (Rails, Ruby, Security) by Leonardo Borges on 24-08-2008

Tagged Under : , ,

REXML, the XML library uses by many ruby apps, including rails, has a vulnerability that requires an immediate patch on whatever rails version you’re using.

Details and instructions on the official rails weblog, here.

But basically, this is what you need to do:

gem install rexml-expansion-fix

Then, require rexml-expansion-fix in your rails’s app environment.rb file.

Why I like Ruby #1: alias_method

Filed Under (Ruby, Why I Like Ruby) by Leonardo Borges on 07-08-2008

Tagged Under : ,

So you found yourself in the need to override a method but still count on it’s old behaviour?

No problem! Override it with your new code, call super and…. Uh oh!! Suddenly this turned into a problem… Let me give some more context.

I was testing Ferret (and the acts_as_ferret plugin) in a project to provide full text search capabilities to our models. One of the things the plugin does is to add a new method to ActiveRecord, called find_with_ferret. That way, every model can use it. Great!

So I thought that would make sense for me to remove all diatrictics from the input text before letting ferret do its job. You know, like removing umlauts and all that.

I could do that by overriding this method with code to remove the undesired chars and then call its older version to finally do the search - something like calling super, but not quite. And I didn’t want my models to inherit from anything else than ActiveRecord::Base. That wouldn’t make any sense.

alias_method to the rescue!

You know that to redefine a method in an existing class you can open it up and rewrite it. But since you don’t wanna loose the behaviour provided by the original method, this is how you can achieve this:

1
2
3
4
5
6
7
8
9
10
module ActiveRecord
  class Base
    alias_method :find_with_ferret_original, :find_with_ferret
 
    def find_with_ferret(q, options = {}, find_options = {})
      remove_diatrictics!(q)
      find_with_ferret_original(q, options, find_options)
    end
  end
end

And you’re good to go. On line 3 you’re just giving the original method an alias, making a copy of it.

Then you redefine it the way you like and on line 6 you call the old version to make sure u still got the same behaviour.
Now all my models can benefit of this change without requiring them to call another method nor inherit from another class.

Cool, huh? :)

The biggest Rails event in latin america

Filed Under (Conferences, Rails, Ruby) by Leonardo Borges on 04-08-2008

Tagged Under : , ,

Behold latin american railers!

This year we will have the Rails Summit Latin America on October, 15th and 16th, in São Paulo, Brazil.

It’s by far the biggest Rails event we’ve ever had, including many of the speakers that were present at RailsConf.

Fábio Akita is also one of the speakers and provides more details on his blog.

If you’re a assumed rails geek don’t miss the opportunity to hear from the big names and to know a beautiful country like Brazil.

Oh, btw, if you’re brazilian, like me, you have no excuse to miss this party!

Enjoy!!!


Rails Summit Latin America