flouri.sh

Thanks to a nice suggestion by the community which I pretty much used verbatim. Still works with rails 2.0 and 1.2.3. Check out the project homepage.

http://code.google.com/p/query-reviewer/

Kevin Hall and I have released a new version of the query_reviewer plugin. You should start by looking at my first post, to see what the basic premise is before reading this article. The single largest improvment is the ability to analyze the database requests of AJAX requests, which is accomplished by piggy backing javascript or HTML into the ends of AJAX responses. Here's what it currently looks like, analyzing my project on a page that does lots of database requests:

The improvements are:

• View query analysis for AJAX requests
• Take into account the duration of a query (if production data)
• Show PROFILE information from mysql
• Warnings for long key lengths (which can be bad even when you hit an index)

Lots of information is now available for a single slow query:

I'm really excited that this plugin is getting attention, and welcome feedback, suggestions, and bug reports on the Google Code project homepage.

To install it, simply visit the project homepage.

Using ActiveRecord can be tricky, here's how

Data migrations are a major headache in every rails project I've worked on. Developers typically write straight SQL migrations which take much longer to create and test, or they use ActiveRecord and run into problems. I recent wrote a data migration which took 7 existing tables and compressed them into 4. Some of the new 4 had the same names as the existing models, so I finally figured out how to do this safely with ActiveRecord. Here's my advice:

• When changing the schema of an existing set of tables, create new tables and then rename them.
• Include active record fragments inside the migration class at the top of your migrations.
• Put your data migrations inside transactions.
• Make your data migrations completely reversible (which is a lot easier when you follow the first rule)

Here's an example of an ActiveRecord class fragment at the top of a migration:

class MarkPrimaryBits < ActiveRecord::Migration
  class LessonVersion < ActiveRecord::Base
    belongs_to :lesson
  end

  class Lesson < ActiveRecord::Base
    has_many :lesson_versions
  end
 
  def self.up
    ...

By putting the Lesson and LessonVersion classes at the top of the migration, I'm allowing myself to use those classes inside my migration and be completely independent of any changes made to the real model from then on (including the deletion of the class itself). Furthermore, I can have another migration which uses those same class names with completely different meanings and they won't conflict with each other.

How to build custom methods on associations that aren't slow

The "has_many do" syntax has been widely adopted in rails, but I often see it going wrong. Consider the following association extension:

has_many :versions do
  def primary
    find(:first, :conditions => {:primary => true})
  end
end

What's wrong with this code is that the finder has to execute a database request every time it's invoked. True, rails has a query cache, but it results in more database requests then one should need. We can make it better by caching the result in an instance variable attached to the association:

has_many :versions do
  def primary
    @primary ||= find(:first, :conditions => {:primary => true})
  end
end

But what if you already have the association loaded? Inside the block, you can access various methods of the AssociationCollection and AssociationProxy classes. Notable methods are the following:

• proxy_owner - the module that contains the association
• proxy_reflection - the Reflection object that contains the association options (FK, :dependent, etc)
• proxy_target - the cached association data, if the association has been loaded
• loaded? - returns true if the association has been loaded
• reset - delete the cached association data and forget it has been loaded

Using these methods, we can rewrite our method to be the most efficient possible, by making use of loaded association data when present:

has_many :versions do
  def primary
    if loaded?
      @primary ||= proxy_target.detect { |ver| ver.primary? }
    else
      @primary ||= find(:first, :conditions => {:primary => true})
    end
  end
end

Now our method is guarenteed to make only 1 database call, no matter how many times invoked. It also will make zero database calls if the entire association has already been loaded. The only downside is that "how to determine if primary" logic has to be written once in rails_sql and once in pure ruby.

All the flickr functionality with better iPhoto integration!

I just finished a conversion of my existing flickr ajax mephisto plugin to use mac.com web galleries! I finally ended up paying for mac.com, because of the awesome integration with everything.

Features:

• Pull photos from a specific album or your entire gallery
• Uses AJAX to pull feed data, so it won't bottleneck your main page rendering
• Randomly chooses X photos to display every time
• A variety of supported images sizes: square, small, medium, and large
• Images automatically link to Apple's awesome web gallery interface

Issues (depending on size of gallery, mine has 400 images at the time):

• The RSS feed takes between 2 and 10 seconds to deliver from apple.
• The downloaded XML file takes 2 to 5 seconds process.

Usage

How I used it in my site:

<p>Pictures:</p>
{% macgallery feed: http://gallery.mac.com/dsboulder/?webdav-method=truthget&feedfmt=photocastrss  count: 4  format: square  width: 75  height: 75 %} 
{% endmacgallery %}

Supported options to the "macgallery" liquid tag:

• feed: REQUIRED. The url of the RSS feed to your gallery, or to a specific album.
• count: OPTIONAL (default: 6). The number of images to display.
• format: OPTIONAL (default: square). Also valid are "small", "medium", and "large".
• width/height: OPTIONAL (default: none). If specified, will add width and height attributes to the IMG tag. Useful for resizing the square images from their 160x160 default size.

Where to get the plugin

Using CodePress to make writing blog entries easier

After I got Metphisto working with Ultraviolet syntax highlighting, I decided to make writing blog entries easier! I already had some experience with my TicTacToe project using CodePress for real-time syntax highlighting. From there, I was able to modify Mephisto to use syntax highlighting while I was writing my articles... the goal being to make the HTML easier to read and easier to write:

The first step was to install CodePress from codepress.org into the mephistro public directory. I ended up putting the css in /stylesheets, and the javascript in /javascripts... big mistake! Codepress likes to be kept all together (probably best in /public), and you shouldn't have to rewrite any of the paths in codepress.html or codepress.js, like I did.

The second step was to include codepress.js in your layout.

Then I made the following modifications to /app/views/admin/articles/_form.rhtml replacing only the dd tag that defines the "body" textarea:

<dd><%= form.text_area :body, :class => 'fat codepress html linenumbers-off', :rows => 25  %>
  <br/><%= check_box_tag "enable_codepress", "1", true, :onchange => "article_body.toggleEditor();" %>
&nbsp;  Use codepress for syntax highlighting 
  <%= hidden_field_tag "article[body_hidden]", @article.body, :id => "article_body_hidden" %>
</dd>

Just changing the class of the text_area is enough to active codepress... pretty cool huh? The checkbox is cool too, allowing you to turn codepress on/off on the fly. Unfortunately, I had to do some craziness with a hidden field. Codepress does not submit the edited data with the form, like you'd expect. Because it creates an design-mode iframe, the browser does not submit your code the way you want it to. However, it provides an object on which you can call a method to get the unformatted code. We can copy the code into the hidden field with an onsubmit in /app/views/admin/articles/new.rhtml and /app/views/admin/articles/edit.rhtml by adding an :onsubmit option to the form like so:

{:id => 'article-form', :multipart => true, 
  :onsubmit => "document.getElementById('article_body_hidden').value = article_body.getCode();"}

Of course, we've now got a hidden field named article[body_hidden] which contains the actual body, rather than the regular field article[body], so we need to deal with that somewhere. I chose /app/models/article.rb, adding the following public method:

def body_hidden=(newval)
  self.body = newval
end

This is still mostly untested, but I'm really enjoying it so far. Uploading an asset inline might not submit the body of the article anymore, so you'll have to a similar piece of javascript to copy the value from the codepress object or lose your changes when you upload an image.

Being a mac rails user, I love textmate. Naturally, I went crazy when I found out that Ultraviolet is a syntax highlighting gem for ruby that reads textmate theme and textmate syntax files to create xhtml highlighted code. After using this in my tictactoe project, I integrated it into mephisto. Because I wanted to use the filter:code tag that's normally used by coderay, I did this by deleting all the files in the filtered_column_code_macro/lib plugin except code_macro.rb, which I replaced as follows

require 'uv'
class CodeMacro < FilteredColumn::Macros::Base
  def self.filter(attributes, inner_text = '', text = '')
    lang = attributes.delete(:lang) || "ruby_on_rails"
    theme = attributes.delete(:theme) || "cobalt"
    begin
      Uv.parse( inner_text, "xhtml", lang, false, theme) 
    rescue
        RAILS_DEFAULT_LOGGER.warn "UltraViolet Error: #{$!.message}"
        RAILS_DEFAULT_LOGGER.debug $!.backtrace.join("\n")
    end
  end
end

Then I copied in the xhtml CSS files for Ultraviolet xhtml rendering (as shown in the UV instructions) into my stylesheets directory. I included the themes I liked into my layout.liquid as follows:

<link rel="stylesheet" href="/stylesheets/uv/blackboard.css" type="text/css" media="screen" />
<link rel="stylesheet" href="/stylesheets/uv/dawn.css" type="text/css" media="screen" />
<link rel="stylesheet" href="/stylesheets/uv/cobalt.css" type="text/css" media="screen" />

Finally, you have to remove a reference to 'include "coderay"' in environment.rb. For each code block in your blog, you can change the theme (there's about 12 included themes):

import java.net;

public class Bob extends BobParent implements Bobliness {
  private final int SOMETHING=0;
  public static void main(String[] args) {
    System.out.println("Hello world!" + 5.5)
  }
}

UV comes with syntax highlightinb for 158 languages:

actionscript, active4d_html, active4d_ini, active4d_library, active4d, ada, antlr, apache, applescript, asp, asp_vb.net, bibtex, blog_html, blog_markdown, blog_textile, blog_text, build, bulletin_board, cake, camlp4, cm, coldfusion, context_free, css_experimental, css, cs, csv, c, c++, diff, dokuwiki, dot, doxygen, d, dylan, eiffel, erlang, fortran, f-script, fxscript, greasemonkey, gri, groovy, gtdalt, gtd, haml, haskell, html-asp, html_django, html_for_asp.net, html_mason, html_rails, html, html_tcl, icalendar, inform, ini, installer_distribution_script, io, javaproperties, javascript_+_prototype_bracketed, javascript_+_prototype, javascript, java, jquery_javascript, json, languagedefinition, latex_beamer, latex_log, latex_memoir, latex, lexflex, lighttpd, lilypond, lisp, literate_haskell, logo, logtalk, lua, macports_portfile, mail, makefile, man, markdown, mediawiki, mel, mips, mod_perl, modula-3, moinmoin, mootools, movable_type, m, multimarkdown, objective-c, objective-c++, ocamllex, ocaml, ocamlyacc, opengl, pascal, perl, php, plain_text, pmwiki, postscript, processing, prolog, property_list, python_django, python, qmake_project, qt_c++, quake3_config, ragel, r_console, rd_r_documentation, regexp, regular_expressions_oniguruma, regular_expressions_python, release_notes, remind, restructuredtext, rez, r, ruby_experimental, ruby_on_rails, ruby, s5, scheme, scilab, setext, shell-unix-generic, slate, smarty, sql_rails, sql, ssh-config, standard_ml, strings_file, subversion_commit_message, sweave, swig, tcl, template_toolkit, tex_math, tex, textile, tsv, twiki, txt2tags, vectorscript, xhtml_1.0, xml_strict, xml, xsl, yaml, yui_javascript

Also posted on ELC

Creating wrapper classes for the sandbox

When creating my sandboxed game of Tictactoe (where a user can upload a new algorithm and play tictactoe against it), I wanted to expose only a small part of my application to user uploaded code. In the follow code, for example, I would want to provide user access to only a few methods of the Board class:

class Board < ActiveRecord::Base
  has_many :moves
  belongs_to :algorithm_x, :class_name => "Algorithm", :foreign_key => "algorithm_x_id"
  belongs_to :algorithm_o, :class_name => "Algorithm", :foreign_key => "algorithm_o_id"

  def make_move!(x, y)...
  def move_matrix...
  def log_info(msg)...
  def winner...
  def game_over...
  def make_computer_move!...
  def human_turn?...
end

If I want to allow the user's code to access make_move, moves, move_matrix, log_info only, I'd create a wrapper class as follows:

class BoardWrapper
  def initialize(board); @board = board; end
  def make_move(x,y); @board.make_move(x,y); end
  def moves; @board.moves.collect {|m| MoveWrapper.new(m) }; end
  def move_matrix; @board.move_matrix; end
  def log_info(msg); @board.log_info(msg); end
end

acts_as_wrapped_class

This is pretty cumbersome to build, so I built acts_as_wrapped_class to make creating these wrappers easy. It does the following:

• Automatically generate a wrapper class for each class marked as acts_as_wrapped_class
• Dispatch methods that match (or don't match) a safelist or blacklist
• Finds appropriate wrappers for return results (meaning if Board returns a Move then BoardWrapper returns a MoveWrapper)
• Wrap the contents of arrays and hashes (same as above, but will work with arrays of Move, and Hashes containing Move)
• Dispatch ===, hash, <=> methods directly to the wrapped objects. Compare two wrappers objects and get the same results as the two wrapped objects.

The above example is much shorter when written with acts_as_wrapped_class:

class Board < ActiveRecord::Base
  acts_as_wrapped_class :methods => [:moves, :make_move!, :move_matrix, :log_info]

  def make_move!(x, y)...
  def move_matrix...
  ...
end

class Move < ActiveRecord::Base
  belongs_to :board
    
  acts_as_wrapped_class :methods => [:x_pos, :y_pos, :is_x, :created_at]
end

Simple executing acts_as_wrapped_class inside the definition of Board automatically defines the BoardWrapper class with checks on which methods are called. This is accomplished through undefining all the methods of BoardWrapper and defining a method_missing which checks the safelist/blacklist before dispatching the method call.

Try to access winner on a BoardWrapper and it will throw an exception, because :winner isn't on the list of approved classes. Of course, you can call wrapper._wrapped_class and get access to the original Board object, but if you've set up your sandbox correctly, the class Board will not even be defined in the sandbox and will raise an exception.

View the RDOC for acts_as_wrapped_class for more detail.

acts_as_runnable_code

In order to make sandboxing user code even easier, I created another gem: acts_as_runnable_code. This gem helps you with the creation of the sandbox, the referencing of the wrapper classes, and automatic wrapping/unwrapping of data as it flows in and out of the sandbox. It assumes the following about your application

• you have objects that store user uploaded code in them
• you want to use your classes in the sandbox with reduced functionality provided by acts_as_wrapped_class
• you want to evaluate an instance of user uploaded code within the context of some instance of a wrapped class

When writing tictactoe, I created an Algorithm model which stored user uploaded code in a database TEXT field. I also wanted to evaluate that code using the binding of the Board object on which the game was being played (meaning the user code looks like "make_move!(1,1)" rather than "@board.make_move(1,1)").

class Algorithm < ActiveRecord::Base
  acts_as_runnable_code
end

@board = Board.find(id)
@board.algorithm_x.run_code(@board, :timeout => 1.0)

View the RDOC for acts_as_runnable_code gem.

To see tictactoe in action, create your own algorithm, and test the safety of the sandbox (scary!) visit tictactoe.mapleton.net

I originally posted this on ELC's blog:

A few weeks ago, I decided to make a rails-based game. I wanted to bring the strength of ruby's metaprogramming into the game world, so I investigated sandboxing user uploaded code blocks. The only ruby sandbox was written by Why the Lucky Stiff, and you can find complete details on it here:

The Freaky Freaky Sandbox

The sandbox is an amazing hack on ruby's lookup tables to essentially allow a completely separate execution context with its completely own set of classes. The interesting part is how it interfaces with the outside world (the "Jungle"):

• Classes can be copied in from the Jungle using Sandbox.import, and exist in both places with separate definitions. The sandbox automatically does this with simple essentially classes like String, Object, Hash, Array, etc.
• Classes can be proxied in from the Jungle using Sandbox.ref. In this case, a proxy class is defined in the sandbox with exactly the same name as the outside class, but with only a two methods: const_missing & method_missing. When a method is called on the proxy, the sandbox is disabled and the actual method executes outside the sandbox. The result of the method is Marshalled into the sandbox, and it is enabled again.
• Objects can be copied into the sandbox using Sandbox.set as long as they're defined there. This is accomplished by marshalling
• Objects can be returned from the sandbox at the end of a Sandbox.eval call. This is accomplished by marshalling

Now you're ready to start writing your own applications using the sandbox. It's a pain to install in ruby 1.8.6, because it requires a small patch, but Why says that it works without patch in ruby 1.9.

I posted this originally on ELC's Blog