My new server - Part 2

Posted by Nucc

As I’ve mentioned my server provides Ruby On Rails service for companies also, so I’d like to write in short about Rails hosting. Rails is a framework written in Ruby. There’s a ruby application repository called rubygems like apt-get. So when you want to upgrade your Rails, as in apt-get, you can do it by the gem management application. The current rails version is over the 2.0 release and rubygems is over the 1.3. When we were working on the sites (those are being hosted currently), we used Rails 1.2.3 and rubygems 0.7. Today these versions are unsupported, and the upgrade is not so trivial.

Rails application opens ports by its own server engine (you should only call script/server in the project to start a process on the 3000 port), or you can use other software, for example Mongrel (there is another solution with fastcgi, but the restart is complicated). You can specify the number of the listening ports and the number of the parallel threads to response to the requests. You need to bunch these ports to port 80, so you need a server that proxies the requests coming on the port 80 to the Rails processes. So briefly, you need a web server to proxy, and Mongrel processes which run the Rails applications.

Currently the Passenger module to Nginx and Apache skips the application server part of the previous process (the Mongrel processes), and creates Rails instances directly from the web server, so you should specify only your application’s public directory in the web server’s configuration file, and it works out of the box. So, I have an old active system with the 3-step-process, and there is a better solution that I would like to support and would be comfortable in the future.

I was sitting on a fence how I should solve this problem, how I could separate similar environments to different scopes. Building a virtual server park could be an alternative, but I don’t want to run a lot of apache web servers and ssh daemons concurrently. My other problem was ssh protocol doesn’t not support virtual hosts, and since I haven’t got different IP addresses to each virtual server, I should support ssh access on different ports, which I don’t want.

I tried to make a solution for this hosting problem, and finally the result was different chrooted environments for the old and the new version of Rails. I decided to use that kind of separation for all bigger components, like database services, name server, mail server, developer part, so I would call these bigger environments as racks. Backuping files is very important on servers, therefore I needed to found a solution for that problem also, so I’d like to show some tricks which is very comfortable later.

Receipt:

You will need squashfs and aufs first. Aufs is an union filesystem in kernel space (using postfix caused some aufs ooops in kernel, so I recommend to use fuse-unionfs in this case, the problem it’s running in user space, so the context switches cause performance decrease)

First create a minimal chroot environment that will be extended to service oriented rack. Using mksquashfs, we pack it to an image file that will be the hibernated service. Using aufs, make a merge of the read only image file’s content with a writable directory on the file system. All changes in this rack appear in this writable directory, so when you want to make a backup, you have to save only the writable partition. So the goal is each read only image (created by mksquashfs) contains (only) the whole software environment for a service (like mail-server, database-server), and each more file like configurations, created and modified files will be on the writable partition.

My directory structure is the following:

/racks - all the data of the racks
/racks/iso - iso files which contain the services
/racks/readonly - readonly directories mounted by loopback
/racks/readwrite - modified files
/racks/active - merged readonly and a readwrite directories
/var/share - shared files like unix sockets, common log files...
/racks/share/dev - dev directory into chroot environments

Building steps:Let webserver-1.0.iso is the web server environment. Building racks will be in the next part of this thread, till then you can create chroot environment with debootstrap in Debian or Ubuntu (of course it won’t contain webserver components), or download a Gentoo stage 3 archive.

debootstrap --variant=minbase lenny /tmp/chroot_base
mksquashfs /tmp/chroot_base /racks/iso/webserver-1.0.iso

webserver-1.0.iso is in the /racks/iso directory, and you can burn it to a disc for backup. According to the previous directory roles, you need to create the corresponding directories in the rack structure, so you need

/racks/readonly/webserver
/racks/readwrite/webserver
/racks/active/webserver

Mount the image to the readonly directory:

mount -o loop,ro /racks/iso/webserver-1.0.iso /racks/readonly/webserver

Merge it with the readwrite part to the active directory

mount -t aufs -o br=/racks/readwrite/webserver=rw:/racks/readonly/webserver=ro none /racks/active/webserver

After merge is ready, we have to mount the default directories:

Mount /proc

mount -t proc none /racks/active/webserver/proc

Mount /devmount -o bind /racks/share/dev /racks/active/webserver/dev

Mount /var/share

mount -o bind /var/share /racks/active/webserver/dev

Mount /home

mount -o bind /home /racks/active/webserver/home

And finally, jump to the rack with

chroot /racks/active/webserver

I have a script that runs these steps, starts daemon applications after the mount process, when you want to stop the rack it stops all daemons and processes and unmount the directories in the right order.When you have a new upgrade from a software (for example when a new Nginx or Apache is released), just extract the current image to somewhere, upgrade the software in the new one, test it by chroot into it, when everything seems to be good, make a new iso with squashfs and edit your script to use webserver-2.0.iso instead of 1.0. The writable layer is the same, and if something goes wrong, you just back up to 1.0.

In the next part I’m going to show how you can create environments from scratch, which contain only that software you need.

Posted in: English, rails, server Comments(0) November 2009


Erubis

Posted by Nucc

I’ve been looking for memcache solutions, when I found merb. Merb is similar to Rails, it’s developed by Ezra Zygmuntowicz (author of Deploying Rails). Merb uses Erubis instead of ERB. Erubis is refinement of ERuby, three times faster than ERB, it has auto HTML escaping support, but the most important for me is the preprocessing ability.

I tried to install Erubis to Rails 2.0.2 for testing, but I had problems. I would like to share how to setup in Rails 2.0.2, because it’s not the same in 2.0.1 and 2.0.2

First, setup from gem

sudo gem install erubis --include-dependencies

We create /config/initializers/erubis.rb for loading on boot.

  1.  
  2. class ActionView::Base
  3.   private
  4.     def convert_template_into_ruby_code
  5.       # dummy
  6.     end
  7.  
  8.     def delegate_compile_with_preprocessing(handler, template)
  9.  
  10.       if ::Erubis::Helpers::RailsHelper.preprocessing
  11.         preprocessor = ::Erubis::Helpers::RailsHelper::PreprocessingEruby.new(template)
  12.         template = self.instance_eval(preprocessor.src)
  13.         if ::Erubis::Helpers::RailsHelper.show_src
  14.           logger.debug "** Erubis: preprocessed==<<’END’\n#{template}END\n"
  15.         end
  16.       end
  17.  
  18.       delegate_compile_without_preprocessing(handler, template)
  19.     end
  20.  
  21.     alias_method_chain :delegate_compile, :preprocessing
  22. end
  23.  
  24. require ‘erubis’
  25. require ‘erubis/helpers/rails_helper’
  26.  
  27. module ActionView
  28.   module TemplateHandlers
  29.     class Erubis < TemplateHandler
  30.  
  31.       def compile(template)
  32.         klass = ::Erubis::Helpers::RailsHelper.engine_class
  33.         properties = ::Erubis::Helpers::RailsHelper.init_properties
  34.         klass.new(template, properties).src
  35.       end
  36.     end
  37.   end
  38. end
  39.  
  40. ActionView::Base.register_default_template_handler :erb, ActionView::TemplateHandlers::Erubis
  41. ActionView::Base.register_template_handler :rhtml, ActionView::TemplateHandlers::Erubis
  42.  
  43. # settings
  44. Erubis::Helpers::RailsHelper.engine_class = Erubis::FastEruby
  45. Erubis::Helpers::RailsHelper.show_src = true
  46. Erubis::Helpers::RailsHelper.preprocessing = true

In settings, we have three options. I use FastEruby, and I turned on show_src to see precompiled html code in console. If you want to use preprocessing, need to set preprocessing true. Preprocessing is useful for loop, but the syntax changes, when you use it. Instead of <% %> you have to use [% %], and if you have variable, you have to use _?(’variable’). I tried to test in a view,

  1. <% 1.upto 1000 do %><%= link_to "My post", _?(’@post link’) %>

with ERB it’s generate 49-61 req/sec, erbius without preprocessing is 51-74 req/sec, and with preprocessor 90-120 req/sec. If I have more time, I try to make test cases, and publish them.

One comment: If you use html escaping function, change every <%= %> tags to <%== %> ;) I’ve need some time to realize my mistake.

Posted in: English, rails Comments(0) February 2008


Slots plugin

Posted by Nucc

I’d like to share my last Rails plugin which add comfortable management for remote_calls. I have a work for developing a photo manager. It needs a lot of remote call for change for example the name of the album, upload photo, remove uploaded photos from the div tree, rotate image without reload the whole page. The currently solution in rails like this:

/app/view/photo/index.rhtml

  1. <div id="photo_1"></div>
  2. <div id="photo_2"></div>
  3. <%= link_to_remote "Rotate Photo 1",
  4.         :url => {:action => "rotate", :id => "1"} %>

/app/controllers/photo_controller.rb:

  1. class PhotoController < ApplicationController
  2.   def index
  3.   end
  4.  
  5.   def rotate
  6.     @photo = Photo.find_by_id(params[:id])
  7.     @photo.rotate(90)
  8.     @photo.save!
  9.     render :update do |page|
  10.       page["photo_1"].replace_html  :partial => "photo",
  11.                                     :object => @photo
  12.     end
  13.   rescue
  14.   end
  15. end

If we have a lot of link_to_remote for different methods, we have to write render :update .. for each one. The other problem, what happens if one of my action try to call another action. For example, always when I rotate an image, I’d like to share with the user what happens (flash a div element, with “Photo rotated” message). If I write another method for flashing message, it would be double rendering. If I write a helper method, I should always pass the page parameter, and if the message use my params[], I should pass it too. I have a solution for this problem.

Install this plugin in your Rails framework.

script/plugin install https://svn.bteam.hu/plugins/slots/trunk/

Let’s look the changes

/app/view/photo/index.rhtml

  1. <div id="message"></div>
  2. <div id="photo_1"></div>
  3. <div id="photo_2"></div>
  4. <%= link_to_remote "Rotate Photo 1",
  5.         :url => {:action => "rotate", :id => "1"} %>

/app/controllers/photo_controller.rb:

  1. class PhotoController < ApplicationController
  2.   slots :rotate, :message
  3.  
  4.   def index
  5.   end
  6.  
  7. end

/app/helpers/photo_helper.rb

  1. class PhotoHelper
  2.   def rotate
  3.     @photo = Photo.find_by_id(params[:id])
  4.     @photo.rotate(90)
  5.     @photo.save!
  6.  
  7.     page["photo_1"].replace_html  :partial => "photo",
  8.                                   :object => @photo
  9.     message("Photo was rotated by 90 degree")
  10.     #Or we can use
  11.     # message("Photo #{params[:id]} was rotated by 90 degree")
  12.   rescue
  13.   end
  14.  
  15.   def message(msg)
  16.     page["message"].replace_html  :partial => "message",
  17.                                   :object => msg
  18.   end
  19. end

As you see, I put our remote called methods into helper. It’s necessary because only helpers can change our views. If we want to use the params hash in the message method, we can do this.

In controllers, when we call slots method, it generates a method with the same name as the slot, so we can use before_filter for this methods.

Enjoy it!

Posted in: English, rails Comments(0) February 2008