yonkeltron.com

Temporary exile

Quick Thoughts on Client-side Template Rendering

Very recently, there was a discussion posted about the Google+ team’s usage of client-side template rendering. The discussion continued on the DailyJS blog which asked a few questions about appropriate usages of client-side rendering for performance, design or philosophical reasons. This got me thinking enough to write down some of my thoughts as I have recently experimented with client-side rendering for both personal projects and work. For reference, I have been toying with underscore templates.

For this discussion let’s define performance as having two components:

  • Resource utilization
  • Speed

Naturally, in this simplification both items overlap quite a bit and misrepresent the actual nature of the issue, blah, blah, yadda, yadda. Ok so the computer scientist faculty taught us all that optimization entails either maximizing something or minimizing something. Let’s discuss performance optimization in the general case by optimizing each performance component separately. Explicitly: minimizing resource utilization and maximizing speed.

With respect to resource utilization, consider the example of generating a page containing a large report. The server has to query the database for all required data and instantiate necessary data structures. Thousands of database records could create a very large object graph very quickly, thus consuming lots of memory. Next, the server must perform the required calculations or whatever data manipulation operations required to create the report. This means consuming CPU cycles and (usually) eating up more memory. Already, resource consumption in terms of IO, memory and processing power can clock in at high levels, even after significant optimization. After all this, the server then must render the markup to display the content which, in the case of many reports, means plenty of repetitive markup surrounding the stuff you spent all that time preparing.

In this case, why not just ship the data to the client and generate markup from a few templates in-browser? The more you can farm work out to clients, the fewer resources you have to consume on the server. This means less time your app spends in the HTTP request cycle and the sooner GC can run freeing memory, tempfile file handles can be closed and your app can begin relinquishing all resources, returning them back to the loving arms of the OS.

The same goes for bandwidth and transmission time (even gzip compression which you use currently, right!?) as, even next to large, JSON-serialized data, full-blown pages full of repetitive HTML tags appear huge by comparison. Which brings us to the next item: speed. (Yeah, I did mention the overlap, right?)

The sooner the server can ship the data, the sooner the browser can start rendering. The Google+ guys made it clear that when they do send markup, they frequently flush IO buffers to send ready-made chunks to the browser. This enables browsers to render sooner and begin fetching external resources like CSS, JS and images. With client-side rendering, shipping only data to the browser can get you the same effect. With proper incremental rendering, you render the content at the top of the report (a very good place to start) first thus showing the user some activity.

As another example, take ActiveRecord-powered form validation in Rails. The default behavior seen most commonly has, in cases of validation failure, the entire form being re-rendered and sent to the client.

Re-render the “new” action on validation failure
1
2
3
4
5
6
7
respond_to do |format|
  if @model.save
    format.html { redirect_to model_path(@model) }
  else
    format.html { render :action => :new, :status => :unprocessable_entity }
  end
end

However, as any moderately-experienced Rails dev knows, after failing validation, the ActiveRecord model makes available a hash of errors. Therefore, shipping only the error data offers a significant size reduction during transmission.

Only ship back the errors hash on validation failure
1
2
3
4
5
6
7
respond_to do |format|
  if @model.save
    format.json { render :json => @model }
  else
    format.json { render :json => @model.errors, :status => :unprocessable_entity }
  end
end

As for philosophy, different people have different views on moving business logic to the client side. Some business logic can move easily to the browser while other business logic can not or should not move. Twitter, for example, has made a point of becoming clients of their own API moving significant pieces of functionality to the browser. In our case of report rendering, why not move some of the calculations to the browser? Conversely, if a web app were a game, you might not want to move too much business logic to the browser if that might make it easier for players to cheat. Those who believe in client-side business logic will, by definition, have a much easier time accepting the idea of client-side rendering as well. For others, it may take a little more exploration for people to warm up to the idea.

Google has a reputation for measuring almost everything. Therefore, I imagine that they did not make the decision to implement client-side template rendering in Google+ simply for the lols. They have tremendous experience with all types of content and page fragments so should have copious performance data to look at.

Similarly, we (mere mortal developers) can learn from the ideas of measurement-influenced design decisions. Run some simulations to see which meets your performance requirements: rendering the page on the server and transmitting it to the browser or just sending the data and generating markup on the client. Right tool for the right job and different strokes for different folks. Or whatever.

My Talk at the NewHaven.rb October Lightning Talk Meetup

Exactly one week ago, I had the pleasure of giving a brief talk at NewHaven.rb’s October Lightning Talk meetup. The other presenters totally put me to shame but I did have fun with slides introducing my new RubyGem, CrunchPipe. I hope that CrunchPipe will evolve sufficiently so that it might provide a useful mechanism to help developers organize computation using the pipeline pattern. Furthermore, I find myself very interested in suggestions regarding methods of making CrunchPipe suitable for distributed programming. Checkout the slides for an overview and code examples.

Back in Action

So, I’ve done two things wrong. First, I’ve not blogged anything new in quite some time after obviously finding many awesome things. Second, I’ve been negligent in maintaining the WordPress installation which used to run this site. With the high development speed that’s been going on in the WordPress project, there have been, as there are with any application, some bugs. Some of these are security-related. I’d also like to emphasize that it’s not my host which has done anything wrong here. Nor is it the fault of the WordPress developers! Both gave me the tools to keep my instance running and up to date. I just got lazy, busy with work and careless so things fell into disrepair, blah, blah, blah.

Therefore, I’ve migrated to jekyll and octopress for great justice. Now, I write in markdown, compile the site with a Ruby toolchain and deploy via git, ssh or rsync (still working out what I want to do for deploy).

Hooray!

So yeah, that’s what I’m working on for the moment.

Xz, All the Cool Kids Are Using It

At work, we recently stopped using both bzip2 and gzip for compression. Instead, we’re now using xz which utilizes the LZMA2 algorithm. Why on earth would someone switch compression algorithms? I’ll tell you why. Because, I have used xz to compress all manner of files and it consistently out-performs bzip2! For example, I compressed a 66mb SQL dump with both xz and bzip2. The results were that xz created a file over 1.5mb smaller than the one created by bzip2. original: 67386kb bzip2: 5716.972kb xz: 4117.625kb To the casual user this might not seem like a lot but for large files and archives, the savings add up.

Newschool Git Workflow With Magit

I’ve been on a roll with using Emacs to make my workflow more efficient. So far, so good. The latest major addition is my adoption of the most-excellent Magit mode for integrating Git and Emacs in a wonderfully-efficient way. One of the best features is that you get a magit-status buffer which acts like your Git command center. From there you can perform whatever actions you want and call up any repository information you need with ease. The feature I really wish it had was a git-blame viewer but Magit has so dramatically improved my git workflow that I really don’t care.

Great Post About Emacs Keyboard Macros

I always find great stuff on the emacs-fu blog but today I read a particularly-wonderful post about Emacs keyboard macros. Like the author, I’ve been a long-time Emacs user but never really got into keyboard macros because it’s been quite easy to produce an elisp one-liner in many cases. That being said, this seems like a great time to learn so I look forward to reading more on the topic of keyboard macros as well as learning some new tricks. Thanks for a great post!

newhaven.rb Hackfest Tomorrow!

Tomorrow, in New Haven, CT will be the “Ruby, White and Blue” Hackfest run by newhaven.rb. We’ll be working on a few projects including the group’s site and most likely some Prawn and ScosugBot stuff. Beginners and people interested in learning are more than welcome and we hope to have some excellent projects for people to get started on. Fill out the RSVP form and we’ll meet at 6pm EST at Blue State Coffee on Thursday, July 1st. Bring a computer and let’s write some code.

Creating a Ruby DSL

Tons of people in the Ruby community go on and on about domain-specific languages (abbreviated DSL) and how wonderful they are. In most cases, I agree with them. I began to wonder how I could go about leveraging Ruby’s awesomely-flexible syntax to create my own DSL. To illustrate my quest, I have written this article. It assumes you know Ruby. The example details are completely fictitious and just contrived enough to be interesting. I promise, the code will be the main focus and I will link/point to other resources which might be helpful to budding DSL designers like myself. Problem The Dream Castle Architectural Firm (D-CAF, among friends) wants a tool which it can use for very high-level prototyping of custom homes. They need it to be understandable by both computers and humans. It only needs to keep track of the following things: - Houses - Each house has a name - Floors - Each house has multiple floors. Each floor has a

number
  • Rooms - Each floor has multiple rooms and each room has a type

The only other requirement is that the DSL be translatable to plain English so that they can show it to customers. No problem. Solution The DSL for the high-level specification of custom houses will look like the following:

CustomHouse.build :home do
  floor(1) {
    room :den
    room :kitchen
  }

  floor(2) {
    room :bedroom
    room :bathroom
  }
end

That’s it. You specify that a house should be built, that it has a name, some floors and each floor has some rooms. Simple, easy and 100% pure Ruby. Notice the cunning usage of both do/end and bracket notation for defining blocks. The outer block passed to CustomHouse#build uses do/end syntax while the blocks passed to House#floor use the bracket syntax. These could easily be reversed (or combined or whatever) but it makes it look pretty and helps to visually differentiate things so that you (the developer) and the user (the architect) can things more clearly. Plus, when you print the output (calling to_s on the instance of House which gets returned), you get the following wonderful text:

House named home has 2 floors.
Floor 1 has 2 rooms (den, kitchen)
Floor 2 has 2 rooms (bedroom, bathroom)

It’s a small house, don’t be a wiseacre. So, how can such a DSL be built? Implementation Let’s write the implementation for this together. First, we’re going to start of with a module named CustomHouse. In Ruby, modules are just classes so we’ll define a class method called build which will behave like a factory method.

module CustomHouse
  def self.build(name, &block)
    house = House.new(name)
    house.instance_eval(&block)
    return house
  end
end

As you can see, the method takes two params, a name and a block. The first thing that it does is create an instance of the House class (which we have not yet defined) passing in the name parameter. Second it calls instance_eval on the house passing in the block. This ventures into the territory of metaprogramming which is great fun but beyond the scope of this document, though I am sure others have used it for DSL construction (coincidentally, if you are interested in Ruby metaprogramming, buy this book). Suffice it to say that it executes the supplied block in the context of the House instance. Finally, the CustomHouse#build method returns the instance of House. (There are those who believe that *any* type of eval is fundamentally evil. I have been taught this many times and I try to avoid using eval on an actualy string whenever possible. Still, someone with a better understanding of Ruby internals might be able to better explain if this in any better.) Next, let’s define what the code for the House class looks like. As a note, it will be defined in the CustomHouse module, technically making the fully-namespaced name of the class CustomHouse::House.

class House
  attr_accessor :name, :floors

  def initialize(name = '')
    @name = name.to_s
    @floors = []
  end

  def floor(number, &block)
    fl = Floor.new(number)
    fl.instance_eval(&block)
    @floors < < fl
  end

  def to_s
    str = "House named #{@name} has #{@floors.length} floors.\n"
    @floors.each do |f|
      str << f.to_s
    end
    str
  end
end

The above code should appear relatively simple. The House class has only a few methods. First, it has a constructor function which takes a name and a block, setting the name as an instance variable and another instance variable, @floors, which is just an empty array. Next, it has a method called floor which takes a number and a block. The guts of this method should look familiar to you because it mimics almost exactly the build factory method defined on CustomHouse. Finally, it has a to_s method because of the requirement that the DSL be translatable to plain English for clients to check out. If we can just dwell for a moment on the floor method, notice that, it too, takes a block and uses instance_eval. It then adds the newly-constructed instance of Floor to the array in @floor. Let’s look at the Floor class now.

class Floor
  attr_accessor :number, :rooms

  def initialize(number = 0)
    @number = number
    @rooms = []
  end

  def room(type)
    @rooms < < Room.new(type)
  end

  def to_s
    str = "Floor #{@number} has #{@rooms.length} rooms ("
    @rooms.each do |r|
      str += "#{r.type}, "
    end
    str.chop!.chop!
    str += ")\n"
    str
  end
end

There shouldn’t be anything confusing about the above code as it doesn’t use any sort of block eval. In fact, the only thing left to look at is the class for Room which is even less impressive.

class Room
  attr_reader :type

  def initialize(type)
    @type = type
  end
end

That’s it. Seriously, that’s the entire implementation of the DSL. You pass a block to CustomHouse#build which gets executed in the context of a new instance of House. The block calls the House#floor method with a block which in turn gets executed in the context of a new instance of Floor. The Floor#room method adds new Room instances to the class and that’s basically it. Here’s all the code together with the example:

module CustomHouse

  def self.build(name, &block)
    house = House.new(name)
    house.instance_eval(&block)
    return house
  end

  class House
    attr_accessor :name, :floors

    def initialize(name = '')
      @name = name.to_s
      @floors = []
    end

    def floor(number, &block)
      fl = Floor.new(number)
      fl.instance_eval(&block)
      @floors << fl
    end

    def to_s
      str = "House named #{@name} has #{@floors.length} floors.\n"
      @floors.each do |f|
        str << f.to_s
      end
      str
    end
  end

  class Floor
    attr_accessor :number, :rooms

    def initialize(number = 0)
      @number = number
      @rooms = []
    end

    def room(type)
      @rooms << Room.new(type)
    end

    def to_s
      str = "Floor #{@number} has #{@rooms.length} rooms ("
      @rooms.each do |r|
        str += "#{r.type}, "
      end
      str.chop!.chop!
      str += ")\n"
      str
    end
  end

  class Room
    attr_reader :type

    def initialize(type)
      @type = type
    end
  end
end

h = CustomHouse.build :home do
  floor(1) {
    room :den
    room :kitchen
  }

  floor(2) {
    room :bedroom
    room :bathroom
  }
end

puts h

Try running it and see what happens! Then try writing other definitions for custom houses and experience the theoretical joy of the hypothetical architectural firm. DSL construction techniques For clarification and context, I’d like to share some other, smaller examples which build on this technique and demonstrate one more. Behold, a DSL for feeding Pandas:

Panda.feed {
  nom :bamboo
  nom :chocolate
}

The implementation of this is both simple and straightforward.

class Panda
  def self.feed(&block)
    panda = Panda.new
    panda.instance_eval(&block)
  end

  def nom(food)
    #whatever
  end
end

Since the block is evaluated in the context of the new Panda instance, it has access to the Panda#nom method. For people deathly afraid of eval, there is this alternative syntax:

Panda.feed do |p|
  p.nom :bamboo
  p.nom :chocolate
end

Which is implemented with yield instead of instance_eval like so:

class Panda
  def self.feed
    yield Panda.new
  end

  def nom(food)
    # whatever
  end
end

For a wonderful and inspiring treatment of Ruby DSLs and associated patterns, see the most-excellent talk on the matter given by Jeremy McAnally at the 2009 Mid West Ruby Conference. Conclusion DSLs are a fantastic tool which can help to simplify complicated and repetitive tasks. Ruby is very good for creating DSLs but it is not the only good tool out there. I advise you look into the creation of DSLs with Scala and, the best DSL-creation tool there ever was, Lisp Macros. I am interested in improving this tutorial for the benefit of those programmers who wish to learn about DSL construction but don’t know where to start.

Buy Discount Erectile Dysfunction Pills online Buy 50 mg Levitra 50mg Generic Cialis Buy Cialis Seattle Online Buy Erectile Dysfunction Pills Visa Buy Cheapest ED pills Buy Cheap ED pills uk Cheap 25 mg Levitra