Ruby on Rails
Last edited August 24, 2008
More by 無名 »
Installation

在 Ubuntu 下安裝 Rails « The Magic DIrk « http://www.dirkye.net
www.dirkye.net/diary/install_rails_on_ubuntu_and_r...

在 Ubuntu 下安裝 Rails

DIrk 發表於 2006-07-29 Saturday

Ubuntu 的官方源中,有 rails 的包,但沒有 RubyGems 的包,用過 Rails 的人應該知道,通過 RubyGems 來安裝、升級 Rails 是最方便的,gem 會自動處理相關的包依賴關係。想要在 Ubuntu 下使用 RubyGems,需要手動安裝:

1、到http://rubyforge.org/projects/rubygems/下載最新的 RubyGems 源代碼;
2、使用以下命令進行安裝,完成後安裝相應的 Rails 包(以 RubyGems 0.9.0 為例):

[dirk@ubuntop:~] wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
[dirk@ubuntop:~] tar -zxvf rubygems-0.9.0.tgz
[dirk@ubuntop:~] cd rubygems-0.9.0
[dirk@ubuntop:~] sudo ruby setup.rb
[dirk@ubuntop:~] sudo gem update --system
[dirk@ubuntop:~] sudo gem install rails --remote --include-dependencies
[dirk@ubuntop:~] sudo gem install ajax_scaffold_generator
[dirk@ubuntop:~] sudo gem install redcloth
[dirk@ubuntop:~] sudo gem install BlueCloth

類似的,可以通過 RubyGems 安裝你想要的其他軟件包。另外,如果想要使用 RMagick,需要預先安裝 ImageMagick:

[dirk@ubuntop:~] sudo apt-get install imagemagick
[dirk@ubuntop:~] sudo apt-get install libmagick9-dev ruby1.8-dev
[dirk@ubuntop:~] sudo gem install rmagick --remote --include-dependencies
最後,可以使用以下命令檢查 RMagick 是否安裝成功(命令後面幾行是我機器上的執行結果):
[dirk@ubuntop:~] ruby -rrubygems -e "require 'RMagick'; puts Magick::Long_version;" 
This is RMagick 1.13.0 ($Date: 2006/06/26 23:32:37 $) Copyright (C) 2006 by Timothy P. Hunter
Built with ImageMagick 6.2.4 01/28/06 Q16 http://www.imagemagick.org
Built for ruby 1.8.4 (2005-12-24) [x86_64-linux]
Web page: http://rmagick.rubyforge.org
Email: rmagick@rubyforge.org
Hivelogic: Articles: Building Ruby, Rails, LightTPD, and MySQL on Tiger
hivelogic.com/articles/2005/12/01/ruby_rails_light...

Building Ruby, Rails, LightTPD, and MySQL on Tiger

What follows are instructions for manually building and installing the following applications on Mac OS X 10.4 (Tiger).

  • Ruby on Rails 1.1
  • Ruby 1.8.4
  • LightTPD 1.4.11
  • MySQL 4.1

These instructions also cover the installation of the following supportive applications as well:

  • FastCGI 2.4.0
  • RubyGems 0.9.0
  • Readline 5.1
  • PCRE 6.6
  • FastCGI and MySQL bindings

Compiling and installing these tools this way is well worth the effort, as the end result delivers an easy-to-upgrade, system-independent, stand-alone development platform that is impervious to potential problems that can be caused by system updates, operating system upgrades, etc. These issues and additional background information about why one might roll their own tools in this fashion is detailed in the article, Using /usr/local/, which could be considered a prerequisite for this task.

A Favor

Please don't email me asking for help. I feel bad writing that, it feels pretty crummy to say, and I truly wish that I could answer specific questions about these instructions and help figure out why a certain step might have gone wrong for you, but I just can't.

Just make sure to follow each step (Set your path! Install Xcode!) and these instructions should 「just work」 for you. Hopefully, the comment system I'm implementing here soon will help fill this need and allow for more community interaction to help with problem solving.

The Concept

Basically, what we're going to do here is download a bunch of open-source tools (some of which rely upon each other to work), configure them, compile them, and install them, one by one, until we have everything we need for a Mac OS X machine to run pretty much any Ruby on Rails application.

What's Needed

A few things are needed to get this going:

  1. Mac OS X 10.4 (Tiger)
  2. The Developer Tools – Xcode 2.0 or newer (not installed by default!)
  3. Willingness to brave the rough underbelly of OS X and type commands into the Terminal application exactly as they appear here
  4. A tasty beverage to enjoy while things compile

A Quick Warning

While it's unlikely anything we do here might do any kind of damage to the system, it's good advice to have a current backup of everything, just in case. The Narrator doesn't take any responsibility for anything that results from following these instructions. We're following these instructions at our own risk.

Setting Up

Open the Terminal application. It can be found in the /Applications/Utilities folder.

Each of the lines below appearing in monospaced type should be entered into Terminal, and be followed by the Return key. But everybody knew that already.

Create a folder to contain all of the downloaded files and their respective build folders. For these examples, we'll create a folder called src in the root of our home folder, and change directories into that folder. It will be our workspace:

mkdir src
cd src

It doesn't really matter where this folder actually lives. We've created it in the home folder, but it could be on the Desktop, or in /usr/local/src for example. All operations should take place there.

You'll download and compile everything from right here.

Paths

Do not skip this step! Most everything else will fail if you do.

We need to make sure that our path is set to look for files in /usr/local (also the place where we'll be installing the tools) before looking anywhere else.

To see if the path has been set properly, we can check the contents of the .bash_login file (a special, hidden file in the root of our home folder) for a PATH line using a text editor. TextMate, TextWrangler, BBEdit, and vi are all perfectly good options. To open the file with TextMate, for example, we can type:

mate ~/.bash_login

This will open the file if it already exists, or open a blank file if it doesn't. In either case, add the following line at the very end of the file:

export PATH="/usr/local/bin:/usr/local/sbin:$PATH" 

Now save and close the file.

It's critical that /usr/local/bin and /usr/local/sbin come first in the path. Just having them in the path isn't enough. They have to be first.

To make sure the changes are picked up correctly, we now need to execute the file with the following command:

. ~/.bash_login

It's likely there will be no response from the shell here, just the prompt, but that's OK, the changes have been picked up and we're ready to move on.

Ruby

We're ready to start the process. Just type (or cut and paste) each one of the following lines into Terminal, one by one. When one line finishes (some will take a while and dump a lot of information to the screen), enter the next one.

The first time we run the sudo command, and possible again later, we may be prompted for a password. We'll just enter our own password here, and the process will continue.

We'll start by installing Readline, a prerequisite for Ruby on OS X systems:

curl -O ftp://ftp.gnu.org/gnu/readline/readline-5.1.tar.gz
tar xzvf readline-5.1.tar.gz
cd readline-5.1
./configure --prefix=/usr/local
make
sudo make install
cd ..

If you get an error here like this:

checking whether make sets $(MAKE)... no
checking for gcc... no
checking for cc... no
checking for cc... no
checking for cl... no
configure: error: no acceptable C compiler found in $PATH

This means that you did not follow the instructions and don't have Xcode installed. Apple provides Xcode on the install CDs/DVDs, and also for free on the Apple Developer Connection website. Run through the install, and you should be just fine.

Next up, we'll download and install Ruby itself:

curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.4.tar.gz
tar xzvf ruby-1.8.4.tar.gz 
cd ruby-1.8.4
./configure --prefix=/usr/local --enable-pthread --with-readline-dir=/usr/local
make
sudo make install
sudo make install-doc
cd ..

RubyGems

RubyGems is a handy command-line tool for managing the installation of Ruby packages, like Rails.

Did you remember to set your path correctly? If not, this step will fail.

curl -O http://rubyforge.iasi.roedu.net/files/rubygems/rubygems-0.9.0.tgz
tar xzvf rubygems-0.9.0.tgz
cd rubygems-0.9.0
sudo /usr/local/bin/ruby setup.rb
cd ..

Ruby on Rails

With RubyGems installed, Rails is a simple, one-line install:

sudo gem install rails --include-dependencies

Some people following these instructions report one of two errors after performing this step. They may see a message like this:

/usr/local/bin/gem:3:in `require': No such file to load -- rubygems (LoadError)
from /usr/local/bin/gem:3

If you see this, it means you did not set your path as instructed in the very first step.

Go back to the beginning and run that step again, then retry this step.

Some readers also report seeing an RDoc failure error. This one is actually nothing to worry about. Just re-run the original command above, or don't. Things should be fine either way (really).

FastCGI

FastCGI is an extension to CGI (Ruby on Rails is essentially a collection of CGI scripts) that provides high performance without the limitations of server specific APIs. Don't worry if that doesn't make perfect sense. Just so long as it's installed on the system:

curl -O http://www.fastcgi.com/dist/fcgi-2.4.0.tar.gz
tar xzvf fcgi-2.4.0.tar.gz
cd fcgi-2.4.0
./configure --prefix=/usr/local
make
sudo make install
cd ..

We'll also need to add the Ruby-FastCGI bindings

curl -O http://sugi.nemui.org/pub/ruby/fcgi/ruby-fcgi-0.8.6.tar.gz
tar xzvf ruby-fcgi-0.8.6.tar.gz
cd ruby-fcgi-0.8.6
/usr/local/bin/ruby install.rb config --prefix=/usr/local
/usr/local/bin/ruby install.rb setup
sudo /usr/local/bin/ruby install.rb install
cd ..

We're almost done with FastCGI. We just need the Ruby-FCGI Gem—another nice one-line installation:

sudo gem install fcgi

PCRE

But before we can compile LightTPD, we need to build its prerequisite, the PCRE library, a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5:

curl -O ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-6.6.tar.gz
tar xzvf pcre-6.6.tar.gz
cd pcre-6.6
./configure --prefix=/usr/local CFLAGS=-O1
make
sudo make install
cd ..

Note for UNIX Geeks: The CFLAGS=-O1 setting on the configure line above is there to make PCRE work with Intel Macs (such as the iMac Core Duo and the MacBook Pro). The setting has to do with GCC optimization levels, and we're taking it down one level from the default (2) for the PCRE compile. This shouldn't affect performance or even compile time, but if you're on a non-Intel Mac and care about such things, feel free to leave the CFLAGS=-O1 parameter off.

LightTPD

LightTPD is an open-source webserver designed with security, speed, compliance, and flexibility in mind. It's great for serving up Rails applications, and the latest versions of Rails are already setup to use it as the development webserver if it's on the system.

curl -O http://lighttpd.net/download/lighttpd-1.4.11.tar.gz
tar xzvf lighttpd-1.4.11.tar.gz
cd lighttpd-1.4.11
./configure --prefix=/usr/local --with-pcre=/usr/local
make
sudo make install
cd ..

MySQL

While it's possible to compile and install MySQL ourselves, using the OS X MySQL package is actually advantageous. Not only is the install much faster and easier, but the package includes a handy startup item and a preference panel, and the binary is tuned by the MySQL team for OS X.

Even better, the package installs MySQL right into the /usr/local/ folder, just like it should.

The install still requires a few steps:

  1. Download the MySQL 4.1 package for OS X PPC or the MySQL 4.1 package for OS X Intel
  2. Double-click the drive image to mount it
  3. Locate the MySQL installer (a file named something like mysql-standard-4.1.20-apple-darwin8.6.0-powerpc.pkg) and run it, authenticating as needed
  4. Double-click MySQLStartupItem.pkg, authenticate, and let it install
  5. Double-click MySQL.prefPane and install it, deciding whether to make it available to just the current user, or for all system users

Once the install is complete, start the MySQL server using the newly-installed control panel.

Note: MySQL installs with a default user of root which has no password. Please read this page about MySQL usernames and passwords and set a good one.

Add MySQL to Your Path

This is an important step because it allows us to run MySQL commands right from the command line, without having to preface them with the long, tedious path to the MySQL binaries folder.

As described in the directions above, edit your .bash_login file once more (perhaps using the mate ~/.bash_login command), and make your PATH statement look like this:

export PATH="/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin:$PATH" 

Close and then open a new Terminal window, and you'll be all set.

MySQL Native Bindings (Optional)

This step is an optional one, but the performance gains seem to make it worth the extra step. Thanks to RubyGems, it's another one-line install:

sudo gem install mysql -- --with-mysql-dir=/usr/local/mysql

You might see a prompt asking you 「which gem to install for your platform.」 Because the mysql gem is updated independently of this article, the versions may change, but your goal will be to pick the very latest 「ruby」 version listed. So for example, you might see the following list:

Select which gem to install for your platform (i686-darwin8.7.1)
1. mysql 2.7.1 (mswin32)
2. mysql 2.7 (ruby)
3. mysql 2.6 (ruby)
4. mysql 2.5.1 (ruby)
5. Cancel installation

You wouldn't want to select option one because it's for Windows (「mswin32」). Instead, you'd want to select the second option, 「mysql 2.7 (ruby)」 which will install the latest standard version for Ruby. It probably wouldn't hurt to select any of the non 「mswin32」 versions, actually, but you usually want the latest/greatest option.

We're Done

Phew! It's over. We now have a properly located installation of Ruby, Ruby on Rails, MySQL, and LightTPD.

So … what's the hold-up? Start writing those Web 2.0 applications already!

 Installation of the Libxml2 in the MacOS

1) Remove the existing libxml2 which built in with the OSX
 

cd /usr/lib  
sudo mv libxml2.2.dylib libxml2.2.dylib.old
sudo mv libxml2.dylib libxml2.dylib.old
sudo mv libxml2.la libxml2.la.old

cd ~

2)  Download and install the update version of libxml , libxslt , and libxml-ruby

curl -O ftp://xmlsoft.org/libxml2/libxml2-2.6.27.tar.gz
 tar zxvf libxml2-2.6.27.tar.gz
 cd libxml2-2.6.27/
 ./configure --without-python
 make && make check
 sudo make install

 curl -O ftp://xmlsoft.org/libxml2/libxslt-1.1.18.tar.gz
 tar zxvf  libxslt-1.1.18.tar.gz
 cd libxslt-1.1.18/
 ./configure --without-python
 make && make check
 sudo make install

 curl -O http://rubyforge.org/frs/download.php/10118/libxml-ruby-0.3.8.tar.gz
 tar zxvf libxml-ruby-0.3.8.tar.gz
 cd libxml-ruby-0.3.8/ext/xml/
 ruby extconf.rb
 make
 sudo make install 

3) Test if the installation is ok 

$ irb
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'xml/libxml'
=> true

4) if you also want to install the libxml2 for python , 
can reference to this link 
http://jamesclarke.info/notes/libxml2

http://72.14.235.104/search?q=cache:h8r3_gZP8rQJ:quruli.ivory.ne.jp/diary/%3F10311421+mac+libxml_so.bundle&hl=zh-TW&gl=hk&ct=clnk&cd=3
 

InternationalizationComparison in Ruby on Rails
wiki.rubyonrails.com/rails/pages/Internationalizat...


InternationalizationComparison

Home Page | All Pages | Recently Revised | Feed

Options available August 2006

Most probably dead

“Multilingual is a project by Per Wigren that does not seem to have been in active development for some time. Globalize took many ideas from Multilingual and was at one early stage called “Multilingual”, but the two projects have little in common now.” (Josh Harvey)

http://www.digitale-wertschoepfung.de/artikel/gettext/gettext.html
www.digitale-wertschoepfung.de/artikel/gettext/get...
Rails i18n interface
www.digitale-wertschoepfung.de/artikel/gettext/gettext.html
www.digitale-wertschoepfung.de/artikel/gettext/get...

Using Gettext To Translate Your Rails Application

DRAFT VERSION. FINAL VERSION TO BE RELEASED SOMETIME IN 1st quarter of 2006.

The version of this document is optimized for Ruby-GetText 1.1.0. Make sure you use it.

Introduction

Stand on the shoulders of Giants

Gettext is a great tool for translating user interfaces of applications into different languages. It has been around for a long time and is very well established for this task. Gettext helps you to open up your application for a much wider user base than you would normaly reach with only one language. Since it is used in many open source projects it has a lot of useful tools you can use to your own advantage. It would be possible to “roll your own” solution, but this would consume a large amount of time. Time you would lose for the development of your app. And this is not something you would want, right?

So in this tutorial I am going to show you how to effectively use Gettext for all your translation needs in your Ruby on Rails application.

What can Gettext do for you?

Here is a quick explanation of Gettext for those of you who haven’t worked with it, yet. Gettext is a set of tools that provide help when translating strings in your software. It gives you (straight from the gettext manual):

  • A set of conventions about how programs should be written to support message catalogs.
  • A directory and file naming organization for the message catalogs themselves.
  • A runtime library supporting the retrieval of translated messages.
  • A few stand-alone programs to massage in various ways the sets of translatable strings, or already translated strings.

The only thing you have to do to your program sources is wrap all translatable strings with a method call: gettext(text) or shorter _(text). Example:

Without gettext:

notice = 'Thank you for buying our product.'

With gettext:

notice = _('Thank you for buying our product.')

If you have wrapped everything between the method call Gettext will provide you with the tools to extract these messages (harvester) from your source code and save the results to a portable file format. A runtime library will allow you to display a translated message whenever _() is called. In essence this is what Gettext does. It can do a couple of more things (like update / merge message catalogs, handling plurals, ...) but essentially it tries to make it as easy and as unobtrusive as possible to translate the language strings in your software and then get out of your way.

Preperation is everything—use UTF-8 everywhere

Edit your files with UTF-8 only

The W3C has an awesome page with all you need to know about character sets (choosing, declaring, serving, editing, ...) and other important stuff concerning internationalization on their site called W3C I18N Topic Index. There you will find a lot of good help and suggestions on everything you need to know about general internationalization topics. You really want to spend at least a couple of hours reading through the resources they offer or link to.

If you happen to use VIM like I do you can put these two lines in your .vimrc and it will from then on treat your files as UTF-8 and convert everything to UTF-8 whenever you save a file:

set encoding=utf8
set fileencoding=utf8

If your editor of choice doesn’t provide support for UTF-8 it is probably time to get a new one ;)

Feed your database with UTF-8

If I need a database I usually go for MySQL. At least most of the time. Since version 4.1.x MySQL has very good support for different character sets. Please read the extensive MySQL Character Set Support article. It will give you a very good idea about the kind of support MySQL has to offer.

Here is a table definition I use in a current project:

CREATE TABLE `pages` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `title` varchar(255) default NULL,
  `keywords` varchar(255) default NULL,
  `description` varchar(255) default NULL,
  `block1` text,
  `block2` text,
  `block3` text,
  `block4` text,
  `block5` text,
  `lang` varchar(10) NOT NULL default 'en_EN',
  `category` varchar(255) default NULL,
  `path` varchar(255) default '/',
  `updated_at` datetime default NULL,
  `created_at` datetime default NULL,
  `published_at` datetime default NULL,
  `layout` varchar(255) default 'main.rhtml',
  `template` varchar(255) default NULL,
  `access` tinyint(3) unsigned default '3',
  `version` int(10) unsigned default '1',
  `is_published` tinyint(3) unsigned default '0',
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;

There is nothing really special about it except the DEFAULT CHARSET=utf8 in the last line. It will make MySQL use UTF-8 for every field that holds text like varchar and text.

If you use another maybe more mature and feature rich database like PostgreSQL UTF-8 support should also be available. Just make sure you configure everything upfront, because it might save you time later on.

If you still think you are going to tack on support for UTF-8 later you might reconsider after reading David’s and Jamis’ experiences: Forty-four grueling hours (or Welcome to 37s!) by David Heinemeier Hansson and On the job by Jamis Buck.

If for any reason you need to use anything other than UTF-8 you might want to know that you can use the Iconv module to convert between different character sets. This module should be installed by default on most distributions. If it is not it is usually easy to get. Just be sure to get it if you need charcter set conversion. On Debian you might need a apt-get install libiconv-ruby to get started. Here is a little example stolen from Pickaxe 2 page 686:

Example: Convert olé from UTF-8 to ISO-8859-1

require 'iconv'
result = Iconv.conv("ISO-8859-1", "UTF-8", "ol\303\251")
result #-> "ol\351"

Make Ruby understand UTF-8

Ruby also has to be told that we will use UTF-8 since Unicode strings will be processed. The best place to do this is at the end of $RAILS_ROOT/config/environment.rb. Or if you are like me and want to quickly update your environment.rb after one of those frequent Rails updates you only have one line there:

# Include your app's configuration here:
require 'custom_environment.rb'

and in $RAILS_ROOT/config/custom_environment.rb we add to the beginning:

$KCODE = 'u'
require 'jcode'

Now we have UTF-8 support switched on in Ruby.

Installing Ruby-GetText

After everything in your setup will accept UTF-8 we can go ahead and install Gettext support in Ruby. We will use the Ruby-GetText-Package which is a Ruby implementation of the Gettext interface that also has some c-bindings for the Locale.

If you have rubygems, you can very easily install Ruby-GetText:

$ gem install gettext
Attempting local installation of 'gettext'
Local gem file not found: gettext*.gem
Attempting remote installation of 'gettext'
Select which gem to install for your platform (i386-cygwin)
 1. gettext 1.1.0 (ruby)
 2. gettext 1.1.0 (mswin32)
 3. Cancel installation
> 1
Building native extensions.  This could take a while...

Be sure to choose option 2 if you want Ruby-GetText to work on native Windows(with One Click Ruby Installer) as you are most likely not able to compile otherwise. For all other platforms use 1.

Alternative: If you are using Debian Testing/Unstable (Debian Stable doesn’t come with Ruby-GetText 1.1.0) like I do you can also do a apt-get libgettext-ruby1.8 libgettext-ruby-util, to get everything installed. If you need the source you can get it from the author’s site: Ruby-GetText-Package. Besides installation instructions you will find a small HOWTO, API Reference and documentation on how to use provided tools.

Also: Be sure to check the sample Rails application that is delivered with Ruby-GetText. If you installed via RubyGems it is most likely here: /usr/lib/ruby/gems/1.8/gems/gettext-1.1.0/samples/rails/, but it could be somewhere else depending on where your Ruby is installed. Later I will call that path $RUBYGETTEXT_RAILS_SAMPLE. So keep this in mind when you read.

Create the “po” directory structure

Create a po dir right in your $RAILS_ROOT. In it you will create a subdir for every language/dialect combination (actually the second part of this abbreviation stands for “geographic region”, but I will just call it dialect throughout this document). If you don’t know the right code for your language you can seek help at the Language section of the W3C I18N Topic Index.

Ultimately your directory structure is going to look like this:

simplepages@colinux: /home/simplepages/rails/po:
$ d -T
/home/simplepages/rails/po/:
  |-myapp.pot
  |-de_DE/:
  |    `-myapp.po
  |-en_GB/:
  |    `-myapp.po
  |-en_US/:
  |    `-myapp.po
  |-fr_FR/:
  |    `-myapp.po
  |-fr_CH/:
  |    `-myapp.po
  `-ja/:
       `-myapp.po

The myapp.pot file is the original file created by the rgettext script introduced later. The po files will contain the translation messages that your application is going to use depending on the language that is requested.

Convert pofiles to mofiles

After creating pofiles, you need to convert them to mofiles. If you haven’t yet, please read the the GNU Gettext manual with the explanation of what mo and po stands for.

Add the code below to Rakefile:

simplepages@colinux: /home/simplepages/rails/Rakefile
require 'gettext/utils'
desc "Create mo-files for L10n" 
task :makemo do
  GetText.create_mofiles(true, "po", "locale")
end

Then,

$ rake makemo

It will create locale directories and subdirectroies such as:

simplepages@colinux: /home/simplepages/rails/locale:
$ d -T
/home/simplepages/rails/locale/:
  |-de_DE/:
  |   `-LC_MESSAGES/:
  |       |-myapp.mo
  |-en_GB/:
  |   `-LC_MESSAGES/:
  |       |-myapp.mo
  |-en_US/:
  |   `-LC_MESSAGES/:
  |       |-myapp.mo
  |-fr_FR/:
  |   `-LC_MESSAGES/:
  |       |-myapp.mo
  |-fr_CH/:
  |   `-LC_MESSAGES/:
  |       |-myapp.mo
  `-ja/:
      `-LC_MESSAGES/:
          |-myapp.mo

These files are used by Ruby-GetText. You don’t need to touch these files.

Tools of trade

Gettext

You will need to install Gettext. On Debian I would do apt-get install gettext to do that. It contains a couple of tools that will be handy later on, most importantly msgmerge which can merge different po files so updating your message files will be a snap and msginitwhich can set default values to the header of pofile in your locale.

Ruby-GetText and rgettext

After you have installed Ruby-GetText and tools you should able to call rgettext on the command line. rgettext is a replacement for xgettext which comes with the main Gettext application. Beyond xgettext, rgettext supports not only ruby scripts(.rb) but also ERB files (.rhtml), ActiveRecord(.rb) directly.

h4. ActiveRecord support rgettext extracts all of the table names and field names which the subclass of ActiveRecord::Base has.

Notice: You need to run your database server and configure the config/database.xml correctly before executing rgettext.

poEdit

poEdit is a great tool to manage and edit your translations. It gives you a nice graphical frontend to translate all your messages. It is available for many different platforms. A nice side effect about using an easy to use GUI tool is that you can tell non-programmers to install it, open the po file and just start translating. They might have nothing to do with the coding in your project but will be able to help you translate your software. So if your grandma can translate English to Chinese she can maybe help you with your software project :)

Collecting messages

Now that you have all the tools installed we can start collecting messages.

Add the code below to Rakefile:

simplepages@colinux: /home/simplepages/rails/Rakefile
desc "Update pot/po files to match new version." 
task :updatepo do
  MY_APP_TEXT_DOMAIN = "myapp" 
  MY_APP_VERSION     = "myapp 1.1.0" 
  GetText.update_pofiles(YOUR_APP_TEXT_DOMAIN, 
                         Dir.glob("{app,lib}/**/*.{rb,rhtml}"), 
                         MY_APP_VERSION)
end

Running this task will either create or update your po/myapp.pot and po/*/myapp.po files in the respective po directories. It will go through all the important directories of your rails app and harvest all the Gettext strings in files ending in *.rb, *.rhtml.

$ rake updatepo

Translating and compiling with and without poEdit

After you have successfully harvested your files you should have a myapp.po file in every locale dir. Now you need to translate them. Since the myapp.po files are mere text files you could just use your favourite text editors to translate them. Given that your text editor can edit in UTF-8 mode and you know the escaping rules of Gettext this is all you actually need. Open the file, translate the text and save it. After you have saved the file compile it. Gettext doesn’t work with the text files (myapp.po) directly. It wants a compiled version of it (myapp.mo). Use the rake makemo command to compile:

simplepages@colinux: /home/simplepages/rails/po/de_DE:
$ ls
myapp.po

simplepages@colinux: /home/simplepages/rails:
$ rake makemo

simplepages@colinux: /home/simplepages/rails/locale/de_DE/LC_MESSAGES:
$ ls
myapp.mo

However, it is way more comfortable to use an application like poEdit for this. With poEdit you can also easily open up the myapp.po file. It will give you a nice side by side view your original strings and the translated version, telling what is already translated and what is not. You click on a message and start translating it in a special field. When you are ready just hit the save button and poEdit will automatically compile the myapp.mo file for you (check the preferences if it doesn’t do it by default). That’s it. With a compiled myapp.mo you can start to teach your rails app how to translate your user interface.

See Documents for Translators for more details to translate the po-file.

Implementing Ruby-GetText into your rails app

By now you should have a translated and compiled myapp.mo in your locale dir. For example my German translation of SimplePages is at $RAILS_ROOT/locale/de_DE/LC_MESSAGES/myapp.mo.

Including Ruby-GetText

Edit application.rb to bind textdomain to your application.

simplepages@colinux: /home/simplepages/rails/app/controllers/application.rb:
require 'gettext/rails'

class ApplicationController < ActionController::Base
  init_gettext "myapp" 
  #init_gettext "myapp", "UTF-8", "text/html" # <= Also you can set charset and content_type.
end

In this sample, the textdomain name is “myapp”. Replace it as you like to fit your application. Maybe you want to have different textdomains for your site and the admin section.

And you can select the scope of the textdomain.

  1. If you call bindtextdomain in ApplicationControler, the textdomain applies to the entire application.
  2. If you call bindtextdomain in any other controller with a different textdomain, this textdomain only applies to this specific controller. For example if you call a different textdomain in myapp_controller.rb it will only be used in myapp_controller.rb.

The textdomains are applied to each controller/view/model.

Choosing the right language on every request

Since we are developing a web application we want to be able to choose the current language by request. Additionally we might want to offer the user the possibilty to choose the language from a menu.

Ruby-GetText chooses the current language by following these rules in the given order:

  1. the first value passed the ‘locale’ parameter of GetText.bindtextdomain method call
  2. ‘lang’ value of QUERY_STRING
  3. ‘lang’ value of the Cookie
  4. the value of HTTP_ACCEPT_LANGUAGE
  5. or default ‘en’ (English).

The script $RUBYGETTEXT_RAILS_SAMPLE/vendor/plugins/lang_helper.rb is a sample that selects locale using the cookie value of the user.

Using the Locale in your templates

Depending on the locale that gets selected you will want to customize the language and character set in your templates.

File: $RAILS_ROOT/app/view/layouts/main.rhtml

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="<%= Locale.get %>" lang="<%= Locale.get %>">
<head>
<title><%= @page["title"] %></title>
<meta http-equiv="content-type" content="text/html; charset=<%= Locale.charset %>" />
...

Have your own textdomain for plugin applications

If you are a plugin developer and want to have your own textdomain, you need to separate the Class/Module from ActionView::Base/ApplicationController.

simplepages@colinux: /home/simplepages/rails/vendor/plugins/gettext/lib/gettext_plugin.rb:
require 'gettext/rails'

module LangHelper
  # If you need to bind yet another textdomain to your plugin.
  # Separate the name space from ActionView::Base/ApplicationController.
  class YetanotherTextDomain
    include GetText::Rails

    def initialize
      # You need to call bindtextdomain in an instance of ActionView::Base.
      # The locale is used a same values which define ApplicationController#init_gettext instead of
      # the textdomain.
      bindtextdomain("gettext_plugin") 
    end

    def show_language(actionview)
      langs = ["en"] + Dir.glob(File.join(RAILS_ROOT,"locale/*")).collect{|item| File.basename(item)}
      langs.delete("CVS")
      langs.uniq!
      ret = "<h4>" + _("Select locale") + "</h4>" 
      langs.sort.each do |lang|
    ret << actionview.link_to("[#{lang}]", :action => "cookie_locale", :lang => lang)
      end
      ret
    end

    def cookie_locale(cookies, flash, params)
      cookies["lang"] = params["lang"]
      flash[:notice] = _('Cookie &quot;lang&quot; is set: %s') % params["lang"]
    end
  end

  # This function shows supported languages with link to set cookie 
  # action (cookie_locale).
  def show_language
    YetanotherTextDomain.new.show_language(self)
  end

  # This function is called when the language link is set.
  def cookie_locale
    YetanotherTextDomain.new.cookie_locale(cookies, flash, params)
    redirect_to :action => 'list'
  end
end

Simply put gettext_plugin.po into the po directory.

Conclusion

That should be “it”

That’s it. If you have your translated message catalogs (myapp.mo) in all the right places your application should show your message strings in your favourite language.

You (and your users) can now easily start to translate your application into all the different languages you (they) want. I hope this guide helps you to get started. There are certainly many more aspects of internationalization that you will have to learn and apply. Remember that this is only one of many possible ways to do it. If you find any mistakes, shortcomings or have good suggestions on how to improve this guide I would be more than happy to hear from you.

If you want you can download the original Textile document, make modifications and send them to me. I will be sure to include you in the Thanks section.

Sascha Ebach se at digitale-wertschoepfung dot de

Thanks

  • David Heinemeier Hansson for creating the best web application framework that has ever existed P-E-R-I-O-D
  • Johan Srensen for suggesting that I really should explain the use of instance variables in the ApplicationController#localize.
  • Masao Mutoh – The Author of Ruby-GetText: mutoh at highway dot ne dot jp. Thanks to Masao for extending the old version and making it more flexible.

Author

Sascha Ebach is the owner and lead developer of a small web design and development shop Digitale Wertschöpfung in Cologne, Germany. Together with his two partners he develops and designs complete online solutions for small to medium sized businesses. Up until the surfacing of Rails he used to develop everything in PHP although he has already fallen deeply in love with Ruby since version 1.6.2 came out. For him it is very clear that Rails and Ruby will be The Future Way of developing web applications and he already looks forward to the day when he has ported his last line of PHP to Ruby.

Appendix A: Downloads

Used files

Download the archive with files and scripts I use and talk about in this guide. The file includes the complete skeleton of files that you need to get started.

/home/simplepages/using-gettext-with-rails/:
  |-app/:
  |   `-controllers/:
  |       `-application.rb (the ApplicationController with the .init_gettext method)
  `-po/: (sample directory structure)
      |-de_DE/:
      |-en_GB/:
      `-en_US/:

Download using-gettext-with-rails.tgz

jvoorhis | Globalize Ready for Testing on Rails 1.1
www.jvoorhis.org/articles/2006/04/19/globalize-rea...

Globalize Ready for Testing on Rails 1.1

Posted by Jeremy Voorhis Tue, 18 Apr 2006 22:35:00 GMT

When The Great Destroyer – aka Rails 1.1 – was released, many developers were baffled when their favorite plugins no longer functioned. Found in the wreckage were Globalize and Rails Engines. Applications were forced to remain on older revisions of Rails, and developer happiness was stifled.

No more.

Practitioners of internationalization, rejoice! Like the mighty Phoenix, Globalize has been… err… branched. If you would like to test the new branch which supports Rails 1.1 and onwards (and breaks compatibility with Rails 1.0), then run this command in your plugins directory:

svn export http://svn.globalize-rails.org/svn/globalize/globalize/branches/for-1.1 globalize

and give it a go. With adequate testing, this branch will be merged into trunk and will put us on track for a proper Globalize 1.0 release candidate. As always, please submit tickets to our Trac, located at http://www.globalize-rails.org/trac/.

And for those of you who are interested, yes I am using this branch myself at PLANET ARGON.

Acknowledgements go to Joshua Harvey for his excellent work on the branch.

Tugela Cache

http://www.digitale-wertschoepfung.de/artikel/gettext/gettext.html
www.digitale-wertschoepfung.de/artikel/gettext/get...

and in $RAILS_ROOT/config/custom_environment.rb we add to the beginning:

$KCODE = 'u'
require 'jcode'
Turn Rails to support Unicode

Fixing internals

To tune standard input and output, as well as to enter Unicode literals in scripts add the following code to your application configurations at the start of config/environment.rb (and don’t forget to restart the server for the change to take effect):

$KCODE = 'u'
require 'jcode'
 
 

When using string functions use the versions from the String class in jcode.rb (which is part of your ruby installations). There are some functions where the name differs from the original version. E.g. instead of foo.length() you now have to use foo.jlength().

Note though that most of the methods return wrong results. Among the methods which are not covered by jcode.rb (every use of these methods in your application introduces a bug, sometimes including irreversible damage to strings):


 String#reverse
 String#size
 String#index
 String#[]
 String#downcase
 String#capitalize
 String#downcase
 String#strip, String#rstrip and String#lstrip
 String#slice

Note that all of these are heavily used internally by Rails.

You can partially heal this by using the routines from the Unicode library by Yoshida Masato, available as a gem

But before you do this, don’t forget to have the dev-libraries for that.
(e.g. Debian needs apt-get install ruby1.8-dev)

Windows people might need unicode-0.1-mswin32.gem

then install your gem

gem install unicode

After that the following functions will be available:

Unicode::downcase(string)
Unicode::upcase(string)
Unicode::normalize etc.

You can also try to install the unicode_hacks gem which is available at http://julik.textdriven.com/svn/tools/rails_plugins/unicode_hacks/. It gives you an accessor to address strings as characters explicitly:


  "Some string".chars[0..2] # => "Som" but works for UTF8 too
  "Cafe".chars.reverse # => "efaC" - works for UTF8 too

If you have the Unicode gem installed it will also help you with tasks such as normalizing all the incoming data to KC (so that you don’t strore ligatures in your database and all the Unicode whitespace is stripped).

Configuring the database connection

In Rails before 1.0
If you are using MySQL 4.1 or above, it is indispensable that you tell Rails to tell MySQL to SET NAMES UTF8. You should therefore use a modified version of the before_filter above, as follows:

class ApplicationController < ActionController::Base
    before_filter :configure_charsets

    def configure_charsets
        @headers["Content-Type"] = "text/html; charset=utf-8" 
        suppress(ActiveRecord::StatementInvalid) do
            ActiveRecord::Base.connection.execute 'SET NAMES UTF8'
        end
    end
end

For PostgreSQL this will be:

class ApplicationController < ActionController::Base
    before_filter :configure_charsets

    def configure_charsets
        @headers["Content-Type"] = "text/html; charset=utf-8" 
        ActiveRecord::Base.connection.execute 'SET CLIENT_ENCODING TO UNICODE'
    end
end

In Rails 1.0 and above
You have a special setting in your database.yml at your disposal, made for setting the charset used by your database connection, it is called encoding. Example:

development:
  adapter: mysql
  database: example_development
  encoding: utf8
  username: root
  password:

This will be “utf8” for MySQL or “unicode” for PostgreSQL respectively (for names of character sets that you need to use see MySQL and PostgreSQL manuals – Rails just forwards this name to the database driver).

To use SQLite with UTF8 you need to compile your SQLite with UTF8 support burned-in, after that you don’t need any special settings in database.yml

Beng careful with the schema
When using [Migrations] and schema.rb be careful and set up your database to create Unicode tables by default! The Ruby dump format does not preserve any charset-specifics.

ALTER DATABASE foo_development CHARSET=utf8; ALTER DATABASE foo_test CHARSET=utf8;
You have been hit by this problem if you are running your tests and all characters from the database come as ”????”, while ind evelopment mode they look and work OK.
 

Using helpers and ActionMailer

Some helper methods that work with text (such as truncate) will use Unicode-aware truncations if you have your $KCODE set to “UTF8”. Others will just damage your strings or give no result. Please file a big on the TRAC if you have problems with helpers. ActionMailer should be set up using the charset option.

Configuring output correctly
Further you have to set the charset of your pages to UTF-8. You can do this by adding a before_filter to your ApplicationController:

class ApplicationController < ActionController::Base
  after_filter :set_charset

  def set_charset
    content_type = @headers["Content-Type"] || 'text/html'
    if /^text\//.match(content_type)
      @headers["Content-Type"] = "#{content_type}; charset=utf-8" 
    end
  end
end

To make Unicode support work with the Ajax helpers and versions of Safari prior to 2.0 (which has a bug) you have to take care of that in a after_filter in your ApplicationController:


class ApplicationController < ActionController::Base
after_filter :fix_unicode_for_safari

# automatically and transparently fixes utf-8 bug 
# with Safari when using xmlhttp
  def fix_unicode_for_safari 
    if @headers["Content-Type"] == "text/html; charset=utf-8" and
    @request.env['HTTP_USER_AGENT'].to_s.include? 'AppleWebKit' and request.xhr?
    @response.body = @response.body.gsub(/([^\x00-\xa0])/u) { |s| "&#x%x;" % $1.unpack('U')[0] }
    end
  end

This will convert all Unicode glyphs in the response body to UTF-8 decimal entities. Please note that of you send Javascript encoded like this it won’t be evaluated, so your best bet would be to advise your Safari users to upgrade.


I’m using Safari 2.0.3, and I’m facing this UTF-8/AJAX helper problem. I tried the solution of def fix_unicode_for_safari, but it didn’t worked. The solution I found was to change the test

if @headers["Content-Type"]  "text/html; charset=utf-8"
by
if @headers["Content-Type"]  “text/javascript”

Hope this helps.
Thomas

Another way to fix UTF-8 handling for AJAX in Safari is to prepend an XML prolog to your AJAX output (this will also prevent all the aJS code that was sent from being run):


 def add_item
   render_text "<?xml version='1.0' encoding='utf-8'?>" + "<li>" + params[:newitem] + "</li>" 
 end

Configuring input

If you send your headers properly (and let the browser know that your site outputs pages in UTF8) all modern browsers will send you forms in UTF-8 automatically, so you don’t have to do any input conversions or define “accept-charset” on forms.

Converting between charsets

Use iconv.


require 'iconv'
# will convert from UTF8 to UTF16
Iconv.new('utf-16', 'utf-8').iconv(person.name)

Configuring ActionWebService

Just let AWS know that you have iconv (or the character detection in SOAP won’t work and it will just presume it’s a dirty string and BASE64 your strings)


require 'iconv'

and ensure $KCODE is properly set to “UTF8”! OK!



How to let Time.now() use utf8 encoding?

This peratins to locales, irrelevant here—Julik

If all String functions are broken, how reasonable is it to use unicode?

Just as reasonable as it is to work for someone who doesn’t have the luck of writing in Latin1—Julik


I just encountered a problem with a String containinung German umlauts (aou?) and the scan method of the String class. I wanted to tokenize the string on word boundaries and tried the following:

firstname, lastname = s.scan(/\w+/)

This does not work; scan generated a new token each time it came across an umlaut. So I changed the call slightly:

firstname, lastname = s.scan(/[^ ]+/)

This works as expected.

String#scan is Umlaut-aware for me on Ruby 1.8.4 when $KCODE is properly set to “u”.


Additional community links

For Nuby on Unicode: Unicode video presentation by Julik

If you want to know what exactly you can expect.

Read this for a partial solution

Quick up and running notes about Unicode in Rails

 
compiling imagemagick on mac os x (v10.2+)
written by: takashi okamoto january 4, 2004
NOTE: start by compiling libjpeg, libpng, libtiff and freetype2. if you already have them, just skip all the way down to compile imagemagick. YOU WILL NEED THE DEVELOPER TOOLS INSTALLED!

libjpeg - for jpeg support
 

curl -O http://www.ijg.org/files/jpegsrc.v6b.tar.gz
tar zxvf jpegsrc.v6b.tar.gz
cd jpeg-6b
cp /usr/share/libtool/config.sub .
cp /usr/share/libtool/config.guess .
./configure --enable-shared --enable-static
make
sudo make install
sudo ranlib /usr/local/lib/libjpeg.a

libpng - for png support
curl -O ftp://swrinde.nde.swri.edu/pub/png/src/libpng-1.2.5.tar.bz2
bzcat libpng-1.2.5.tar.bz2 | tar xf -
cd libpng-1.2.5
sed 's/LDFLAGS=-L/LDFLAGS=-dynamiclib -L/' < scripts/makefile.macosx > Makefile
make ZLIBLIB=/usr/lib ZLIBINC=/usr/include
sudo make install
sudo ranlib /usr/local/lib/libpng.a

libtiff - for tiff support
curl -O ftp://ftp.remotesensing.org/pub/libtiff/tiff-v3.5.7.tar.gz
tar zxvf tiff-v3.5.7.tar.gz
cd tiff-v3.5.7
./configure
make
sudo make install
sudo ranlib /usr/local/lib/libtiff.a

freetype2 - for type support
curl -O http://umn.dl.sourceforge.net/sourceforge/freetype/freetype-2.1.5.tar.gz
tar zxvf freetype-2.1.5.tar.gz
cd freetype-2.1.5
./configure
make
sudo make install
sudo ranlib /usr/local/lib/libfreetype.a

imagemagick
curl -O ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-5.5.7-15.tar.gz
tar zxvf ImageMagick-5.5.7-14.tar.gz
cd ImageMagick-5.5.7
./configure --enable-shared
sudo make -k -i install
sudo ranlib /usr/local/lib/libMagick*

 
SVN

HowtoUseRailsWithSubversion Home Page | All Pages | Recently Revised | Feed

This assumes that you have already created a subversion repository, and you are just wondering how to setup your rails project.

If you have WEBrick or lighttpd running after running “script/server” in your rails directory, shutdown the server.

  1. Import your rails application.
    • navigate to the root of your rails app.
    • svn import . repository_url -m "Import" --username user
  2. Change to the directory containing your rails application, and rename it for safe keeping.
    • mv your_rails_app your_rails_app-backup
  3. Check out your rails application
    • svn checkout svn_url_to_your_repository your_rails_app
    • navigate to the root of this freshly checked out working copy.
  4. Remove the log files.
    • svn remove log/*
    • svn commit -m 'removing all log files from subversion'
  5. Ignore the log files when they are re-created (Note for Capistrano/Switchtower users: the log directory is “shared” so it needs to be ignored itself, not just it’s contents)
    • svn propset svn:ignore "*.log" log/
    • svn update log/
    • svn commit -m 'Ignoring all files in /log/ ending in .log'
  6. Ignore the tmp directory with all its content (cache, sessions, sockets)
    • svn propset svn:ignore "*" tmp/sessions tmp/cache tmp/sockets
    • svn commit -m 'Ignoring all files in /tmp/'
  7. Don’t version control database.yml
    • svn move config/database.yml config/database.example
    • svn commit -m 'Moving database.yml to database.example to provide a template for anyone who checks out the code'
  8. Ignore database.yml when it’s re-created
    • svn propset svn:ignore "database.yml" config/
    • svn update config/
    • svn commit -m 'Ignoring database.yml'
  9. Remove any sqlite databases from version control and ignore .sqlite(3) files.
  10. Ignore temporary files (for Rails >=1.1)
    
    svn remove tmp/*
    svn propset svn:ignore "*" tmp/
    svn update tmp/
    svn commit -m "ignore tmp/ content from now" 
    

You’re done. Now your log files will not be checked into subversion on a commit, and you don’t need to worry about the database settings on your development machine and your production machine and things breaking as database.yml moves between them.

Feel free to skip the steps where you remove database.yml, but if you are working with several other programmers who may have different database.yml files you may want to keep it ignored.

to remove all the .svn from the directory

find . -type d -name ".svn" -exec rm -rf \{\} \;

to remove the .svn when pack the directory

tar xzf myproject.tgz myproject/ --exclude "*.svn" 
 

I'm managing a website in my repository. How can I make the live site automatically update after every commit?

This is done all the time, and is easily accomplished by adding a post-commit hook script to your repository. Read about hook scripts in Chapter 5 of the book. The basic idea is to make the "live site" just an ordinary working copy, and then have your post-commit hook script run 'svn update' on it.

In practice, there are a couple of things to watch out for. The server program performing the commit (svnserve or apache) is the same program that will be running the post-commit hook script. That means that this program must have proper permissions to update the working copy. In other words, the working copy must be owned by the same user that svnserve or apache runs as -- or at least the working copy must have appropriate permissions set.

If the server needs to update a working copy that it doesn't own (for example, user joe's ~/public_html/ area), one technique is create a +s binary program to run the update, since Unix won't allow scripts to run +s. Compile a tiny C program:

#include <stddef.h>
#include <stdlib.h>
#include <unistd.h>
int main(void)
{
  execl("/usr/local/bin/svn", "svn", "update", "/home/joe/public_html/",
        (const char *) NULL);
  return(EXIT_FAILURE);
}

... and then chmod +s the binary, and make sure it's owned by user 'joe'. Then in the post-commit hook, add a line to run the binary.

Also, you'll probably want to prevent apache from exporting the .svn/ directories in the live working copy. Add this to your httpd.conf:

# Disallow browsing of Subversion working copy administrative dirs.
<DirectoryMatch "^/.*/\.svn/">
    Order deny,allow
    Deny from all
</DirectoryMatch>
Show up the SVN version in the rails web application

<% svn_info = YAML.parse(`svn info #{RAILS_ROOT}`)%>
Current SVN version :<%= svn_info['Revision'].value%>
Last Changed Date :<%= svn_info['Last Changed Date'].value%> by
<%=svn_info['Last Changed Author'].value%></div>
Deployment

Sketchpad » Blog Archive » Capistrano, Mongrel, and Mongrel_cluster
www.dansketcher.com/2006/04/26/capistrano-mongrel-...

Capistrano, Mongrel, and Mongrel_cluster

April 26th, 2006

After my hiatus on posting it seems appropriate to get back into the meaty stuff…

Ever since I started using Debian (instead of FreeBSD) I've been having weird problems with my rails dispatch.fcgi processes multiplying in the night. Nothing shows up in the logs, but I spawn 2 externally of Lighttpd, and in the morning i've got 6 of the little blighters. Of course, at 20-30 meg a pop, the poor little Xen VPS isn't too happy about that, so I have a nightly job that kills them all and restarts them. And then, 12 hours later, 4 more than I asked for are there. It seems to be a load issue, but I digress.

So ever since I noticed that I've been wanting to use Mongrel to run my Rails apps. For those not familiar with mongrel, it's a Tomcat-style application host for rails apps that avoids (huzzah) the FCGI palaver that we ordinarily have to deal with. The problem was that according to mongrel, a USR2 signal should fully restart the daemon, and it does, but it's a tiny little bit funny in a way that makes it totally painful to use with Capistrano for automated deployment.

You see, on restart, it doesn't re-evaluate what the ./current symlink is pointing to. So when it's restarted over a ./current that points to ./releases/2006xxxxxxxxx1 that's fine, but on a redeployment through Cap when this symlink is repointed to ./releases/2006xxxxxxxxx2, the mongrel instance still points to ./releases/2006xxxxxxxxx1

So that's not so good. Btw, Zed, you're an awesome coder and I in no way mean you disrespect here. I'm just telling the problem I faced.

Along comes mongrel_cluster. Totally fixed the issue for me, and here's how.

1. Install mongrel_cluster, and then (in ./current) run 「mongrel_rails cluster::configure」

2. open the new ./config/mongrel_cluster.yml and edit the line that starts with 「cwd:」 like so:

  • change 「cwd: /path/to/app/releases/2006xxxxxxxxx1〞 to 「cwd: /path/to/app/current」
  • change the port to the first one you want for this cluster, and select how many you want (defauts to 2 which should be 『enough' for most cases)
  • change from 「development」 to 「production」

3. Add the following tasks to your capistrano deploy.rb file

desc 「The spinner task is used by :cold_deploy to start the application up」
task :spinner, :roles => :app do
send(run_method, 「cd #{deploy_to}/#{current_dir} && mongrel_rails cluster::start」)
end

desc 「Restart the mongrel cluster」
task :restart, :roles => :app do
send(run_method, 「cd #{deploy_to}/#{current_dir} && mongrel_rails cluster::restart」)
end

4. Rejoice!

5. Before you do anything else, add to your crontab an @restart task to start the mongrel instances when your server comes up! Very important!

Now 「rake remote:cold_deploy」 and 「rake remote:deploy」 work for the mongrel cluster! I used the instructions on the mongrel site on how to integrate with lighttpd at http://mongrel.rubyforge.org/docs/lighttpd.html but there are equally good Apache 2.2 docs out there (why Apache 2.2 isn't available for Debian is anoher question entirely…)
Much kudos and thanks to Zed Shaw for the excellent mongrel server and to Bradley Taylor for mongrel_cluster. Oh, and Jamis Buck for Capistrano!

Photo Matt » Cross-Datacenter File Replication
photomatt.net/2006/04/26/cross-datacenter-file-rep...
Photo Matt » Cross-Datacenter File Replication 
Easy Mongrel Clustering with mongrel_cluster
fluxura.com/articles/2006/04/24/easy-mongrel-clust...

Easy Mongrel Clustering with mongrel_cluster

Posted by Bradley Taylor on 2006年4月25日

A quick note to announce the release of mongrel_cluster, a gem_plugin for the Mongrel web/application server. This plugin makes it easy to manage multiple Mongrel processes behind a reverse-proxy server and load balancer such as Pound, Balance, Lighttpd, or Apache.

This is the first of many release to support simplified Ruby on Rails application deployment using Rails Machine.

Setup

To get started, install the gem:

sudo gem install mongrel_cluster
Configure the cluster and generate a configuration file. This will run 2 mongrel processes on port 8000 and 8001 using 「production」 environment :
cd /path/to/my/rad/app
mongrel_rails cluster::configure -p 8000 -e production -a 127.0.0.1

With the configuration file written, you can easily start, stop, and restart the cluster. By default the file is written to config/mongrel_cluster.yml, but you can change the path with the 「-C」 option.

Start the cluster:
mongrel_rails cluster::start
Restart the cluster:
mongrel_rails cluster::restart
Stop the cluster:
mongrel_rails cluster::stop

Reverse proxy and load balancing

Pound is a reverse proxy, load balancer and HTTPS front-end. It is very easy to configure for use in front of a mongrel cluster.

Install Pound from source or packaging system of choice then edit your configuraton (most likely in /etc/pound/pound.conf).

Sample configuration for use with mongrel_cluster:
User "nobody"
Group "nobody"

ListenHTTP
  Address ip.address.goes.here
  Port 80

  Service
    Backend
      Address 127.0.0.1
      Port 8000
    End
    Backend
      Address 127.0.0.1
      Port 8001
    End
  End 
End
Start Pound:
sudo pound -f /etc/pound/pound.conf

So far in our testing for Rails Machine, this setup works well and its dead simple to setup and maintain. If you don't want Mongrel to handle static files you could put lighttpd behind Pound and route requests for /images, /stylesheets, /javascript, and other specific paths. This will give you a performance boost on static files, but increase the complexity of your deployment. Your application may not need faster static file serving.

For information on using Mongrel behind Apache, read Scaling with Rails with Apache 2.2, mod_proxy_balancer and Mongrel.

Good luck!

Gea-Suan Lin's BLOG
blog.gslin.org/page/8/

把放很久的文章整理出來。

Photo Matt 這裡看到 MySQL 4.1 之後的版本可以有 INSERT ... ON DUPLICATE KEY UPDATE 這種用法:MySQL Counters。引用的原文在 INSERT ON DUPLICATE KEY UPDATE and summary counters. 這篇。

如果你對於 race condition 有瞭解,你可以在文章裡看到這種用法將本來要自己做的檢查交給 MySQL 處理:

INSERT INTO ipstat VALUES(inet_aton('192.168.0.1'), 1, now()) ON duplicate KEY UPDATE hits = hits + 1;

這個功能在 MySQL 4.1 以及之後的版本有提供。

Gea-Suan Lin's BLOG
blog.gslin.org/page/18/

Matt 問了 Cross-Datacenter File Replication,看起來是為了 WordPress.com 問的。

下面的 comments 裡面有些蠻值得看一看,像是 DRBD 可以玩看看。

Debug

Misc

TREASURED BLOG : Universal Encoding Detector in Ruby
blog.treasured.cn/articles/2006/03/28/universal-en...

Universal Encoding Detector in Ruby

Posted by hui on 2006年3月28日

download:

http://rubyforge.org/frs/?group_id=1506

install:

gem install chardet-0.9.0.gem

usage:

require 'rubygems'
require 'UniversalDetector'
require 'net/http'
Net::HTTP.version_1_2 
Net::HTTP.start( 'yahoo.co.jp' ) {|http|
    data = http.get("/").body
    p UniversalDetector::chardet(data)
    #=> {"encoding"=>"EUC-JP", "confidence"=>0.99}
}

base on Mark Pilgrim's python port.

Jserv's blog: charset-detector:自動偵測文件編碼的小程式
blog.linux.org.tw/~jserv/archives/001672.html

charset-detector:自動偵測文件編碼的小程式

發展程式前,通常會有個動機,而就我剛剛做的這個小程式來說,就是為了透過 [PCManX] 連線到對岸的 BBS 站台,可惜我遇到很麻煩的問題,就是得自己指定編碼,偏偏上週騎腳踏車時,把手握太大力造成輕微受傷,所以一直打錯字... Anyway,我決定要替 [PCManX] 加上自動偵測 BBS 編碼的功能。

自動猜測文件編碼的演算法,在 Mozilla 中已經有不錯的實做,而 Mozilla 官方網頁也提供論文 [A composite approach to language/encoding detection] 作參考,對岸的網友提供了簡體中文翻譯 [一種語言/編碼檢測的復合方法],相關的實做可參考 Mozilla cvs tree [extensions/universalchardet],而之前的 blog [Mozilla Re-licensing 完畢] 也提到 Mozilla Foundation 日前宣佈,Mozilla codebase 由原本的 MPL (Mozilla Public License) 轉換為 MPL / GPL / LGPL 三重授權模式,這與 [PCManX] 的授權相容,所以當務之急就是如何整合。

我初步將 NSPR (Mozilla Runtime) 一類的包袱去掉,並且用 G++ 的 -fno-rtti、-fno-exceptions,以及 -nostdinc++ compilation flags 來編譯 ,如果將 -lstdc++ 換成 -lsupc++,還可進一步得到 C-only library,目標是作成一個 add-on,讓 [PCManX] 可透過 dlopen 來操控內部實做,初步完成自動偵測文件編碼與測試程式,名為 [charset-detector] (bzip2 tarball)。

以下以測試程式 (放在 test 目錄下) 作範例,看看運作情況,initcall.txt 是個用 Big5 編碼的文件:
    charset-detector/test$ file initcall.txt
    initcall.txt: ISO-8859 English text, with CRLF line terminators
    charset-detector/test$ ./test-chardetect ./initcall.txt
    File ./initcall.txt ...
    Charset = Big5
    
UNIX 的工具 file 就誤判了,還好咱們的 charset-detector 正確識別編碼,而 charset-detect library 的 API 只有六個,很容易操作。下一步就是 hack [PCManX],使其建立 BBS connection 後,將 buffer 傳遞給 charset-detect APIs 作編碼的判斷,稍後作適度的畫面重繪動作。
由 jserv 發表於 May 22, 2006 05:40 PM
involution.com » RSS Parser in 10 lines of Ruby or in 153 lines of C++
involution.com/2006/03/20/rss-parser-in-10-lines-o...

I know there's like 10 libraries to do this already, but they all seemed a bit heavy to me. I just wanted to grab the items from an RSS feed. Plain and simple. What's amazing to me is that it could be done in 10 lines of Ruby. This parser reads an RSS feed, and stores the descriptions, items, and links into a hash of arrays.


#!/usr/bin/env ruby
require 'http-access2'
	
h = HTTPAccess2::Client.new()
element = Hash.new {|h,k| h[k] = []}
	
while urlstr = ARGV.shift
  response = h.get(urlstr) { |data|
    data.gsub(/<(description|title|link)>(.*?)<\/(description|title|link)>/m) {
      element[$1].push($2.gsub(/\s\s+/m,' '))
    }
  }
end
	
site_description = element['description'].shift
site_title = element['title'].shift

Stupidly, I wrote the same code in C++ first because I thought it would run faster. After starting down that path, I found myself wanting to use ActiveRecord. So, I ported over to Ruby.

The C++ version is here if you really want to see it. It's 121 lines of C++ with a 32 line h-file. Truth be told, it actually took me less time to code the C++ because I made a couple of mistakes in my Ruby version at first. All said and done, the Ruby version is a helluva lot easier to maintain than the C++ which I think is why I spent the time to do it in the first place.

Rails Migration Cheat Sheet
garrettsnider.backpackit.com/pub/367902

Rails Migration Cheat Sheet

Make sure you view the (short) screencast by DHH

Valid as of Rails 1.1.2

VERIFY: To enable full use of ruby schema support, uncomment 『config.active_record.schema_format = :ruby' in your /config/environment. (Update: The rails schema mechanism is the default as of Rails 1.1)

  • rake db_schema_dump: run after you create a model to capture the schema.rb
  • rake db_schema_import: import the schema file into the current database (on error, check if your schema.rb has 」:force => true」 on the create table statements
  • ./script/generate migration MigrationName: generate a new migration with a new 『highest' version (run 』./script/generate migration' for this info at your fingertips)
  • rake migrate: migrate your current database to the most recent version
  • rake migrate VERSION=5: migrate your current database to a specific version (in this case, version 5)

(run rake -T for most of this information as rake usage information)

Example schema.rb:

ActiveRecord::Schema.define(:version => 2) do

  create_table "comments", :force => true do |t|
    t.column "body", :text
    t.column "post_id", :integer
  end

  create_table "posts", :force => true do |t|
    t.column "title", :string
    t.column "body", :text
    t.column "created_at", :datetime
    t.column "author_name", :string
    t.column "comments_count", :integer, :default => 0
  end

end

What can I do?

  • create_table(name, options)
  • drop_table(name)
  • rename_table(old_name, new_name)
  • add_column(table_name, column_name, type, options)
  • rename_column(table_name, column_name, new_column_name)
  • change_column(table_name, column_name, type, options)
  • remove_column(table_name, column_name)
  • add_index(table_name, column_name, index_type)
  • remove_index(table_name, column_name)

See the Rails API for details on these.

Example migration:

class UpdateUsersAndCreateProducts &lt; ActiveRecord::Migration
  def self.up
    rename_column "users", "password", "hashed_password" 
    remove_column "users", "email" 

    create_table "products", :force => true do |t|
        t.column "name", :text
        t.column "description", :text
    end
  end

  def self.down
    rename_column "users", "hashed_password", "password" 
    add_column "users", "email" 
    drop_table "products" 
  end
end

Problems? When the migration fails

Tip: if you need to manually override the the schema version that in the DB:
ruby script/runner 'ActiveRecord::Base.connection.execute( 
"INSERT INTO schema_info (version) VALUES(0)")'

Snippets

On OS X, I'm using TextMate with the syncPEOPLE on Rails v.0.9
TextMate Bundle. This provides the following…
(snipped from the release notes)

Snippets are small capsules of code that are activated by a key sequence followed by the [tab] key. For example, mcdt[tab] will activate the Migration Create and Drop Table snippet.

  • mcdt: Migration Create and Drop Table
  • mcc: Migration Create Column
  • marc: Migration Add and Remove Column
  • mct: Migration Create Table
  • mdt: Migration Drop Table
  • mac: Migration Add Column
  • mrc: Migration Remove Column

See Also Sami Samhuri's information for a more complete description of these snippets

Additional places to look:

Steve Eichert: Migrations Explained

Links to other public Backpack pages

 

Published with Backpack. This page is subject to the terms of service.

Digest 支援 MD5 和 SHA1 兩種編碼, 你若有儲存密碼的需求就要用到, 一般是用 SHA1.

MD5 計算


require 'digest/md5'

puts Digest::MD5.hexdigest("Hello World!")
計算檔案的 MD5, 可以確保檔案未曾被修改


require 'digest/md5'

#method 1
puts Digest::MD5.hexdigest(File.read("o.rb"))

#method 2
class Digest::MD5
  def self.open(path)
    o = new
    File.open(path) { |f|
      buf = ""
      while f.read(256, buf)
        o << buf
      end
    }
    o
  end
end
puts Digest::MD5.open("o.rb").hexdigest
SHA1 計算


require 'digest/sha1'

puts Digest::SHA1.hexdigest("Hello World!")

Plugin

Mongrel Upload Progress Plugin

One of the really nice things about Mongrel is its simplicity. It’s very easy for someone to take and extend for their own needs. The Mongrel Upload Progress plugin is an example of how I’m able to extend the Mongrel HTTP Request object and provide near-realtime progress updates.

The reason why this is a challenge, is because web servers usually gather the HTTP request, send it on to the web framework, and wait on a response. This is fine for most requests, because they’re too small to cause an issue. For large file uploads this is a usability nightmare. The user is left wondering what whether their upload is going through or not.

To do this, I’ve written a Mongrel handler that hooks into some basic Request callbacks. To use it, you need to install the gem, and create a small config file for it:

gem install mongrel_upload_progress

# config/mongrel_upload_progress.conf
uri "/", 
  :handler => plugin("/handlers/upload", :path_info => '/files/upload'), 
  :in_front => true

# start mongrel
mongrel_rails -d -p 3000 -S config/mongrel_upload_progress.conf

That config file tells mongrel to load the Upload handler in front of all other handlers. :path_info is passed to it, telling upload_progress to only watch the /files/upload action. There are two more parameters that I’ll get into later: :frequency and :drb. I’m using Rails as an example, but this should work with any Ruby framework, such as Camping or Nitro.

Now that Mongrel is set up, let’s create a basic upload form. If you look closely you’ll notice a few things:

  • A unique :upload_id parameter must be sent to the upload_progress handler. This is so requests don’t get mixed up, and the client page has an ID to query with.
  • The <form> tag is targetted to an iFrame to do the uploading. Certain browsers (like Safari) won’t execute javascript while a request is taken place, so this step is necessary.
  • There is a little javascript library being used. This handles the polling and status bar updates.
  • Notice the form’s action is file/upload, just like the upload_progress handler.

The Rails controller actions for this are very simple. The upload form itself needs no custom code. The upload action only renders javascript to be executed in the iFrame, to modify the contents of the parent page. The progress action is a basic RJS action that updates the current status. Most of the guts of this are implemented in the javascript library.

Here’s what happens when you submit the form:

  • The UploadProgress class creates a PeriodicalExecuter and gets ready to poll.
  • The browser initiates the upload.
  • Every 3 seconds, the PeriodicalExecuter calls the RJS #progress action and gets back the current status of the file.
  • Once finished, the iFrame calls window.parent.UploadProgress.finish(), which removes the status bar and performs any other finishing actions.

How’s this work with a single Mongrel process if Mongrel synchronizes Rails requests? It’s actually very careful about locking, synchronizing only the bare minimum. The whole time that Mongrel is receiving the request and updating the progress is spent in Mongrel, so it can happily serve other requests. This is how the RJS action is able poll while it’s uploading.

This is fine and dandy, but not too many sites run on a single Mongrel. You’ll quickly run into problems with multiple mongrels since only one Mongrel process knows about the upload. You’ll either have to specify a specific mongrel port to communicate with, or set up a dedicated mongrel upload process. The third option, is use DRb.

# config/mongrel_upload_progress.conf
uri "/", 
  :handler => plugin("/handlers/upload", 
    :path_info => '/files/upload', 
    :drb => 'druby://0.0.0.0:2999'), 
  :in_front => true

# lib/upload.rb, the upload drb server
require 'rubygems'
require 'drb'
require 'gem_plugin'
GemPlugin::Manager.instance.load 'mongrel' => GemPlugin::INCLUDE
DRb.start_service 'druby://0.0.0.0:2999', Mongrel::UploadProgress.new
DRb.thread.join

Now in addition to starting mongrel, you’ll need to start the DRb service too:

ruby lib/upload.rb

The Rails app should work the same as before, but now it is using a shared DRb instance to store the updates. This gives us one other advantage: a console interface to the current uploads.

# lib/upload_client.rb, a simple upload drb client
require 'drb'
DRb.start_service

def get_status
  DRbObject.new nil, 'druby://0.0.0.0:2999'
end

# typical console session
$ irb -r lib/upload_client.rb
>> uploads = get_status
>> uploads.list
=> []
# start uploading in the browser
>> uploads.list
=> ["1157399821"]
>> uploads.check "1157399821" 
=> {:size=>863467686, :received=>0}

Using DRb gives you a simple way to monitor the status of current uploads in progress. You could also write a simple web frontend for this too, accessing the DRb client with Mongrel::Uploads.

One final note is the use of the :frequency option. By default, the upload progress is marked every three seconds. This can be modified through the mongrel config file:

uri "/", 
  :handler => plugin("/handlers/upload", 
    :path_info => '/files/upload', 
    :frequency => 1,
    :drb => 'druby://0.0.0.0:2999'), 
  :in_front => true
Unit Testing

Luke Redpath—Ruby, Rails and other Musings
lukeredpath.co.uk/2006/8/29/developing-a-rails-mod...

Developing a Rails model using BDD and RSpec, Part 1

Writing Rails testing articles seems to be quite popular at the moment; seeing as I’m often quite vocal about testing on the #caboose and #rubyonrails IRC rooms I felt it was about time I posted one of my own. I have a large series of articles on testing with Rails in the pipeline, but until that is done, here is a nice and simple tutorial for newcomers to BDD and RSpec – the first in a two-part article exploring the development of a typical Rails model, using BDD techniques and the RSpec framework. If you are interested in BDD and RSpec, or new to testing in general and want to learn how to iteratively develop a model test/spec-first, this is the article for you.

Anatomy of a typical Rails-style test

Will Rails developers please raise their hands: how many times have you written a test that looks like this:


class UserTest < Test::Unit::TestCase
  def test_create
    user = User.create(:some => 'params')
    assert user.save
  end
end

Now ask yourself how many times have you sat back and asked yourself why you are writing the above test?

If the concept of unit testing is new to you, then writing tests at all is a great first step. But its also important to have useful tests. Are your tests valuable? Are your tests acceptable?

Avoid meaningless tests

The above test is a good example of a meaningless test. Why is it meaningless? Because you aren’t testing your own code; you are testing the ActiveRecord library, which is pretty well tested already. Let’s take a look at a default Rails model:


class User < ActiveRecord::Base
end

Just those two lines of code give us a whole load of functionality, all of which is provided by the ActiveRecord library. Its fair to assume that the functionality given by those two lines of code will work. If it doesn’t then there is either something wrong with your local setup or something fundamentally wrong with ActiveRecord; in either case, your own tests are the last of your problems.

Test your own code

So if we can safely assume that the built-in ActiveRecord functionality works as advertised, what should you be testing? The simple answer: test any code that you write. Anything that gets added to your model needs test coverage. The aim of this tutorial is to place an emphasis on testing the behaviour of your code in different situations (or contexts). This is the basis of Behaviour Driven Development, the methodology that I will use in this tutorial to iteratively develop a Rails model, test-first spec-first.

For this tutorial, I will be using the excellent RSpec framework, but you could easily apply these principles to TDD using Test::Unit. Before we get started, you’ll need to install RSpec and the RSpec On Rails plugin for your current app:


~/mygreatapp/ $ sudo gem install rspec
~/mygreatapp/ $ ./script/plugin install svn://rubyforge.org/var/svn/rspec/tags/REL_X_Y_Z/vendor/rspec_on_rails/vendor/plugins/rspec
~/mygreatapp/ $ ./script/generate rspec

Replace X, Y and Z in the above with the version of RSpec that you are using. If you have any problems, refer to the full instructions.

Going into RSpec in full detail is outside of the scope of this article, but it should be pretty clear what is going on – this is one of RSpec’s strengths. If you would like to read a more generic RSpec tutorial, the RSpec website has a great tutorial to get you started.

The problem

We’re in the process of writing our fantastic new Web 2.0 application, and we’ve decided that we need people to be able to create accounts and log in to the application. We don’t want to use any of the available Rails authentication plugins; we want to develop our own User model. After a quick whiteboard/CRC session, we come up with a few basic specs for our User model:

  • A user should have a username that they can log in with
  • A user should have a password between 6 and 12 characters in length
  • A user’s password should always be encrypted in the database
  • A user should have an email address
  • A user can optionally have a first name, surname and profile/description

With this in mind, we fire up our favourite text editor and start work on a new Specification. We use the generator that comes with RSpec on Rails to generate a new model, with an accompanying spec file.


$ ./script/generate rspec_model User

This will create a new user.rb file for our model, just like the normal Rails script/generate model command, but it will also create an accompanying spec file in the spec/ directory. If you open up the created user_spec.rb file, you will see a stub context ready and waiting.

Behaviour Driven Development favours the breaking up of specifications into individual “contexts”. A context is an object (or collection of objects, but generally object being specced) in a certain state. As we are going to start our specs from scratch, you can safely remove the stub context in the user_spec.rb file (don’t remove the require line at the top though!).

So what is a good starting point? I tend to favour a more generic starting context: “A user”. We can use this to specify the behaviour of a user in general.

Specifying your model in code

Let start with our first specification: a user should have a username that they can log in with. Its fair to assume that the username is required (otherwise they won’t be able to log in). So what could we specify? How about this:


context "A user (in general)" do
  def setup
    @user = User.new
  end


  specify "must have a username" do


  end
end

That’s not bad, but it could be better. We’ve expressed a requirement in our code but we haven’t said anything about the behaviour of a User object. What about this instead:


context "A user (in general)" do
  def setup
    @user = User.new
  end


  specify "should be invalid without a username" do


  end
end

That’s better. Not only have we expressed that our user must have a username, but we’ve also expressed what behaviour should be expected from the User model if it doesn’t have one; it should be invalid. Let’s fill this spec in, so we have a failing spec:


context "A user (in general)" do
  def setup
    @user = User.new
  end


  specify "should be invalid without a username" do
    @user.should_not_be_valid
    @user.username = 'someusername'
    @user.should_be_valid
  end
end

Now we need to make this pass. The first things we need is our actual User model, and a table in our database. BDD (and TDD) emphasise taking small steps, following the red, green, refactor mantra. However, due to our coupling to the database as a result of using the ActiveRecord pattern, we are going to have to make a slightly larger leap: our users table schema.

We need to write a migration for our users table, but at this stage we aren’t certain exactly what columns we need. We could write a migration every time we want to add a column but that would quickly become tedious. Instead, we’ll make a reasonable guess at our schema based on our written specs – if we get it wrong at this stage it doesn’t matter. Migrations make it easy to modify our schema in the future. Something like this should do the trick:


class AddUsersTable < ActiveRecord::Migration
  def self.up
    create_table :users do |t|
      t.column :first_name, :string
      t.column :last_name, :string
      t.column :email, :string
      t.column :description, :string
      t.column :username, :string
      t.column :encrypted_password, :string
      t.column :salt, :string
    end
  end


  def self.down
    drop_table :users
  end
end

You’ll note that we’ve made a few assumptions regarding our password columns. We already have an idea in mind about how we want to store the password – as a salted hash – so we’ve created columns for the encrypted password and salt. Now we’ve written and run our migration, and created our User model, its time to get the spec to pass:


class User < ActiveRecord::Base
  validates_presence_of :username
end

You’ll notice that we’ve not added a should statement for the error message itself. That is because we know Rails will happily provide us with the default “can’t be blank” message. Remember: only test the code that you write. In this case, we decide we do want a custom message, so lets add a spec and make it pass:


context "A user (in general)" do
  def setup
    @user = User.new
  end


  specify "should be invalid without a username" do
    @user.should_not_be_valid
    @user.errors.on(:username).should_equal "is required"
    @user.username = 'someusername'
    @user.should_be_valid
  end
end


class User < ActiveRecord::Base
  validates_presence_of :username, :message => 'is required'
end

We’ve also specified that our user must have an email address, so lets add a spec for that:


context "A user (in general)" do
  def setup
    @user = User.new
  end


  specify "should be invalid without a username" do
    @user.should_not_be_valid
    @user.errors.on(:username).should_equal "is required"
    @user.username = 'someusername'
    @user.should_be_valid
  end


  specify "should be invalid without an email" do
    @user.should_not_be_valid
    @user.errors.on(:email).should_equal "is required"
    @user.email = 'joe@bloggs.com'
    @user.should_be_valid
  end
end

That’s simple enough to implement:


class User < ActiveRecord::Base
  validates_presence_of :username, :message => 'is required'
  validates_presence_of :email, :message => 'is required'
end

Great, we’re on a roll. But wait a minute, both of our specs are now failing. What gives? Of course, because we’ve now added two validation requirements, we need to add an email address in the first spec to make it pass and a username in the second spec to make that one pass. Hmm, it doesn’t sound very DRY, but lets go with it for now – we want our specs to pass after all!


context "A user (in general)" do
  def setup
    @user = User.new
  end


  specify "should be invalid without a username" do
    @user.email = 'joe@bloggs.com'
    @user.should_not_be_valid
    @user.errors.on(:username).should_equal "is required"
    @user.username = 'someusername'
    @user.should_be_valid
  end


  specify "should be invalid without an email" do
    @user.username = 'joebloggs'
    @user.should_not_be_valid
    @user.errors.on(:email).should_equal "is required"
    @user.email = 'joe@bloggs.com'
    @user.should_be_valid
  end
end

Phew, that was a close one. Finally, lets add the specs for the password. We know a password is required and that it has to be between 6 and 12 characters in length. Because that is actually two specifications, we’ll write two separate specs in our code. Lets start with the required field specification, as that will look similar to our above specs:


context "A user (in general)" do
  def setup
    @user = User.new
  end


  specify "should be invalid without a username" do
    @user.email = 'joe@bloggs.com'
    @user.password = 'abcdefg'
    @user.should_not_be_valid
    @user.errors.on(:username).should_equal "is required"
    @user.username = 'someusername'
    @user.should_be_valid
  end


  specify "should be invalid without an email" do
    @user.username = 'joebloggs'
    @user.password = 'abcdefg'
    @user.should_not_be_valid
    @user.errors.on(:email).should_equal "is required"
    @user.email = 'joe@bloggs.com'
    @user.should_be_valid
  end


  specify "should be invalid without a password" do
    @user.email = 'joe@bloggs.com'
    @user.username = 'joebloggs'
    @user.should_not_be_valid
    @user.password = 'abcdefg'
    @user.should_be_valid
  end
end

Now, we don’t actually have a password column in our users table, but we need somewhere to store the cleartext password before it gets encrypted. A standard Ruby instance variable will do. Here’s the code to make it pass:


class User < ActiveRecord::Base
  attr_accessor :password


  validates_presence_of :username, :message => 'is required'
  validates_presence_of :email, :message => 'is required'
  validates_presence_of :password
end

Refactoring towards cleaner, clearer specifications

Before moving on to the password length specification, lets address our duplication issue here. Its already getting tedious adding all the other required fields in each spec in order to make them pass. It is making our specs bloated, ugly and it will be a nightmare to maintain in the future if our specification changes. Let’s solve this by introducing a small helper module and a neat Hash extension:


module UserSpecHelper
  def valid_user_attributes
    { :email => 'joe@bloggs.com',
      :username => 'joebloggs',
      :password => 'abcdefg' }
  end
end


context "A user (in general)" do
  include UserSpecHelper


  def setup
    @user = User.new
  end


  specify "should be invalid without a username" do
    @user.attributes = valid_user_attributes.except(:username)
    @user.should_not_be_valid
    @user.errors.on(:username).should_equal "is required"
    @user.username = 'someusername'
    @user.should_be_valid
  end


  specify "should be invalid without an email" do
    @user.attributes = valid_user_attributes.except(:email)
    @user.should_not_be_valid
    @user.errors.on(:email).should_equal "is required"
    @user.email = 'joe@bloggs.com'
    @user.should_be_valid
  end


  specify "should be invalid without a password" do
    @user.attributes = valid_user_attributes.except(:password)
    @user.should_not_be_valid
    @user.password = 'abcdefg'
    @user.should_be_valid
  end
end

There, thats much DRYer, more expressive and easier to maintain. If our valid attributes ever change, we only need to change them in one place. However, we haven’t sacrificed readability in the name of DRY, which is very important with any tests/specs.

Finally, lets add a spec for our password length:


specify "should be invalid if password is not between 6 and 12 characters in length" do
  @user.attributes = valid_user_attributes.except(:password)
  @user.password = 'abcdefghijklm'
  @user.should_not_be_valid
  @user.password = 'abcde'
  @user.should_not_be_valid
  @user.password = 'abcdefg'
  @user.should_be_valid
end

And to make it pass:


class User < ActiveRecord::Base
  attr_accessor :password


  validates_presence_of :username, :message => 'is required'
  validates_presence_of :email, :message => 'is required'
  validates_presence_of :password
  validates_length_of :password, :in => 6..12, :allow_nil => :true
end

You’ll notice we’ve added the :allow_nil option to the length validation. This is to avoid a double validation error if we haven’t set a password – the validates_presence_of validation will already handle this and we don’t want an extra error message complaining about the length of the password as well.

There is one last refactoring that we can do at this stage. In each of our validation specs, we’ve checked that the model is invalid, then set the required value and checked that it is now valid, to ensure that the validation is working end to end. We can extract all of these checks into a single specification:


  specify "should be valid with a full set of valid attributes" do
    @user.attributes = valid_user_attributes
    @user.should_be_valid
  end

Download the full specification.

Whats next?

So far we’ve written a basic User model, with an initial schema and a validation of required attributes. We’ve covered the basics of RSpec syntax and we’ve learnt how to DRY up our specs by extracting common code into a helper module.

In the second part of this tutorial, we’ll look at password encryption and authentication. If you have any questions or feedback, do not hesitate to leave a comment; I’ll be happy to answer any queries you may have.

RSpec on Rails

A Rails plugin that brings RSpec to Rails.

Features

  • Use RSpec to verify models and controllers
  • Integrated fixture loading
  • Special generators for models and controllers that generate specs instead of tests.

Pre-Installation You’ll have to install the RSpec core gem first:

gem install rspec

Take note of what rspec version you’re installing – it’s very important that you install a compatible RSpec on Rails plugin.

Installation

RSpec on Rails is a regular Rails plugin. It can be installed via your Rails app’s script/plugin tool. In the examples below – make sure you replace TAG with the version of the plugin you want to install. It’s important that the REL_X_Y_Z corresponds to the version of RSpec core you have installed – for example REL_0_5_16.

ruby script/plugin install svn://rubyforge.org/var/svn/rspec/tags/REL_X_Y_Z/vendor/rspec_on_rails/vendor/plugins/rspec

Or if you like to live dangerously you can get the HEAD of the trunk:

script/plugin install svn://rubyforge.org/var/svn/rspec/trunk/vendor/rspec_on_rails/vendor/plugins/rspec

Once the plugin is installed, you must bootstrap your Rails app with RSpec. Stand in the root of your Rails app and run:

ruby script/generate rspec

This will generate the various files needed to use RSpec with Rails.

Using the generators

RSpec on Rails contains generators that you can use to generate models and controllers in a similar fashion to Rails’ builtin generators, but will generate specs instead of tests and will put fixtures under the spec folder. Example:

ruby script/generate rspec_model person

or

ruby script/generate rspec_controller person

For more information on each generator, just run them without any arguments.

Running specs

All specs can be run with

rake spec

Model specs can be run with

rake spec:models

Controller specs can be run with

rake spec:controllers

To see all the RSpec related tasks, run

rake --tasks spec

You can also run specs ‘directly’ with

spec path/to/my/spec.rb

But we recommend you use the faster rails_spec tool when working with Rails.

Running specs faster with rails_spec

Loading the entire Rails environment every time a spec is executed is quite slow. We have therefore made a little tool that allows you to load the rails environment once for all. In a separate shell, run:

script/rails_spec_server

-And then in a separate shell:

script/rails_spec REGULAR_SPEC_OPTIONS

The script/rails_spec script has the same command line interface as the familiar spec command. But it’s MUCH faster.

Naming conventions

When you use Rails without RSpec (with Test::Unit), tests for models end up in tests/unit and tests for controllers end up in tests/functional.

In order to make things more consistent, RSpec chooses a slightly different naming convention for direcotries and Rake tasks. So you will find model specs under specs/models, and controller specs under specs/controllers. The Rake tasks are named accordingly.

Examples

RSpec on Rails adds several methods to your specs with a look and feel similar to Test::Unit. Example:

Model:

require File.dirname(__FILE__) + '/../spec_helper'

context "The Person model" do
  fixtures :people, :animals

  setup do
    # fixtures are setup before this
  end

  specify "should find an existing person" do
    person = Person.find(1)

    person.should_equal people(:lachie)
    person.name.should_equal 'Lachie'
  end

  specify "should have animals" do
    people(:lachie).should_have(2).animals
  end

  # http://rubyforge.org/tracker/index.php?func=detail&aid=5539&group_id=797&atid=3149
  #specify "should include animals" do
  #  people(:lachie).animals.should_include animals(:horse)
  #end

  teardown do
    # fixtures are torn down after this
  end
end

context "A new Person" do
  fixtures :people

  specify "should have no name (this finally passes with underscores)" do
    Person.new.name.should_be nil
  end

  specify "should have no name (this passes using dots)" do
    Person.new.name.should_be nil
  end
end

Controller:

require File.dirname(__FILE__) + '/../spec_helper'

context "The PersonController" do
  inherit Test::Unit::TestCase
  fixtures :people
  controller_name :person

  specify "should be a PersonController" do
    controller.should_be_instance_of PersonController
  end

  specify "should create an unsaved person record on GET to create" do
    get 'create'
    response.should_be_success
    response.should_not_be_redirect
    assigns('person').should_be_new_record
  end

  specify "should persist a new person and redirect to index on POST to create" do
    post 'create', {:person => {:name => 'Aslak'}}
    Person.find_by_name('Aslak').should_not_be_nil
    response.should_be_redirect
    response.redirect_url.should_equal 'http://test.host/person'
  end
end

context "Rendering /person" do
  inherit Test::Unit::TestCase
  fixtures :people
  controller_name :person

  setup do
    get 'index'
  end

  specify "should render 'list'" do
    response.should_render :list
  end

  specify "should not render 'index'" do
    lambda {
      response.should_render :index
    }.should_raise
  end

  specify "should find all people on GET to index" do
    get 'index'
    response.should_be_success
    assigns('people').should_equal [people(:lachie)]
  end

  specify "should display the list of people" do
    response.body.should_have_tag :tag => 'p'
  end

  specify "should display the list of people using better api" do
    should_have_tag('p', :content => 'Finds me in app/views/person/list.rhtml')
  end

  specify "should not have any <div> tags" do
    lambda {
      response.body.should_have_tag :tag => 'div'
    }.should_raise
  end

end

Translating existing Test::Unit tests

The test2spec tool that ships with RSpec translates existing tests into RSpec specs. Translating tests to specs in a Rails environment requires some manual steps…

Install the rspec_generator

How to do this is described above.

Modify your test/test_helper.rb

In order to be able to translate any Rails tests, you must modify your test/test_helper.rb file:

# This line must be commented out in order for test2spec to work.
# require 'test_help'
require 'test2spec_help'

The reason for this is that the test_help mixin confuses test2spec to the point where it’s unable to perform the translation. The test2spec_help addresses this shortcoming.

Perform the translations

Now you can translate your unit tests (model tests) with test2spec:

test2spec --template spec/test2spec.erb --specdir spec/models test/unit

and your functional tests (controller tests) with:

test2spec --template spec/test2spec.erb --specdir spec/controllers test/functional

Edit your translated specs

test2spec currently doesn’t translate class-level statements such as fixtures, so you have to add those statements manually. Copy all the fixtures statements in your tests to the corresponding contexts. Example:

context "The Foo Model" do
  fixtures :foo
end

Make sure fixtures are found.

By default, RSpec on Rails expects to find fixtures under spec/fixtures. You should either move your existing test/fixtures/*.yml files to spec/fixtures or edit your spec/spec_helper.rb to point to the old test/fixtures location. Beware that every time you do a script/generate rspec_model, new fixtures will always be written to spec/fixtures.

Prototype class: FastInit() * Dexagogo
tetlaw.id.au/view/blog/prototype-class-fastinit/

Prototype class: FastInit()

The awsome Dean Edwards in "Faster DOM Queries" describes a faster version of the window.onload handler for Mozilla browsers and Internet Explorer. It's an ingenius little hack and offers a real javascript DHTML initialisation boost for those browsers. The explanation of the technique is in the article, basically it allows you to initialise when the browser DOM is loaded and not have to wait for all the other resources to load (like images). Here's a neat little class for Prototype I knocked up. [Updated!] Version 1.1 based on new code from Dean Edwards.

This class implements a kind of onload function queue. This will ensure that your initialisation function(s) are called once the browser DOM is ready and not have to wait for all the images. Hopefully easing that bugbear of DHTML interfaces: slow initialisation.

Instructions

To use it you need Prototype and the fastinit.js file. In your page you instantiate the Fastinit class with a reference or 2 to your onload functions like this:

new FastInit(yourFunction, anotherFunction, yet AnotherFunction);

You can also add more functions like this:

FastInit.addOnLoad(aFunction);  

All functions added via the initialisation method or addOnLoad are called in the order in which they were added.

It will also default to the standard Event.observe(window, 'load' ...) handler if the browser doesn't support either of the faster methods.

Demo

Dorky demo.

Download

Download here: fastinit.js under the standard Creative Commons license.

Or download here under an MIT-style license the same as Prototype: fastinit.js.

Change Log

Version 1.11

Released under an MIT-Style license, same as Prototype.

Version 1.1

  • New code from Dean Edwards adds support for saffari and removes need for seperate IE file
  • Added ability to pass multiple functions on initialisation
  • Added addOnLoad function
  • Fixed bug for onload handler for non supported browsers
  • Fixed bug so onload is only called once

Continuous Integration w/ Rails

Posted by Ryan on 2006年5月25日


Rails does many things well, but one of the best is providing a framework for easy and complete testing. However, continuous integration, theoretically a by-product of easy testability, doesn’t seem to be a major player in the Rails landscape yet. One good reason is that there’s not yet a continuous integration application like CruiseControl or AntHill out for ruby. Sure, there have been sightings of such tools like CIA (seems to have been deprecated) and Damage Control, and there is the handy Autotest which kind of gives you a continuous integration feel – but what about real continuous integration?

Well, we do have an option in the form of the continuous_builder plugin. Is it a slick web-app that lets you configure your build process within the comforts of a GUI? No. But, it does automatically run all your tests on every subversion commit and send a nasty-gram calling out the person who broke it, and that’s pretty much what continuous integration is right?

On a side note, I would love to see an open-source CruiseControl for Rails developed and suspect one will be out shortly as the community grows even further

So how does one get the continuous builder plugin up and running? Read on…

Install continuous_builder Plugin

Installing the continuous_builder plugin is a cinch since it’s part of the rails source tree and is a recognized plugin repository.


cd app
script/plugin install continuous_builder

You can also manually install it if the plugin install doesn’t work for you – though I don’t know why it wouldn’t.


cd app
cd vendor/plugins
svn export http://dev.rubyonrails.com/svn/rails/plugins/continuous_builder

Configure Environment & Mailer

The builder plugin runs as a rake task triggered by subversion’s post-commit trigger. However, it needs a few things before it can just work.

The first thing to note about the plugin is that it’s a bit of a dichotomy. When it runs the rake task to perform the tests (all unit, functional and integration tests) it runs under the test environment. That part is easy and expected. However, what you should also know is that the builder runs inside the development environment by default (which is normal for all other rails commands but slightly unintuitive in this particular situation). This means that, if left as is, the builder will pull the action mailer config from the config/environments/development.rb configuration (or config/environment.rb if there’s one configuration for all environments). So to summarize, the builder configuration wil pull from the development environment config and the rake testing task operates in the test environment.

This could be an issue if your development config doesn’t send real emails (your test config probably wouldn’t either since you’d usually be using ActionMailer’s mock delivery to unit test email functionality). What I would suggest doing to keep your builder configuration seperate from your other environments is to create a new build environment config file. This involves

  1. Creating an config/environments/build.rb config file with a working ActionMailer config
  2. Updating config/database.yml to point to a build database (this can be an empty db as all it will do is create an initial connection (though I wouldn’t suggest pointing to the test or production db)

Now we can safely tell the builder to use the “build” environment to get its mailer configuration, and we can leave the other environments untouched.

Test Configuration

Once you have your build configuration setup (or even if you haven’t chosen to do that), you’re going to want to test the builder. This can be done on your local machine, though it requires some little hacks to make the mail function trigger.

The mail functionality will trigger when the build or tests fail and there’s code to update from the repository. To simulate this I created a unit test that always failed, and reverted one small portion of the code to a previous version (using svn update -r _previous revision#_ ). Once you’ve got this simulated failure setup, run the following:


# Omit "RAILS_ENV=build" if you did not set up a seperate environment
# Include the BIN_PATH if your rake executable is not located
# at /usr/local/bin (and include the trailing "/"!)
RAILS_ENV=build BIN_PATH=/usr/bin/ rake -t test_latest_revision \
NAME=app_name RECIPIENTS="dev-list@yourcompany.com" \
SENDER="build@yourcompany.com" 

If all is setup correctly, you will see the standard test output and should see the build status output to the log/last_build.log file. If anything is wrong it should be recognizeable as a build or test error. Fix it and try again :) (and don’t forget to re-setup the fake failure situation described before)

Setup Environment on Build Server

Now that you know your plugin is working, you have to duplicate that setup on the build server – which has to be the same server that subversion is running. Here’s the process I went through:

  1. SSH to the subversion machine as the user that subversion runs as
  2. Make a directory where the builder can check out your source to (I’ve used /tmp/rails/app)
  3. Checkout the code to that directory from the command line – just to make sure you can
  4. Edit the hooks/post-commit file located in your subversion repository directory to look like this (for me this was ~svn/repo/hooks/post-commit)

#!/bin/sh

# Set vars (if necessary)
BUILD_DIR=/tmp/rails/app
BIN_PATH=/usr/bin

# Execute builder (use the same command that worked on your local
machine in the previous test scenario)
cd $BUILD_DIR && \

# Make sure we have the latest and greatest db structure
$BIN_PATH/rake migrate VERSION=0 && \
$BIN_PATH/rake migrate && \

# Run continuous build task!
RAILS_ENV=build BIN_PATH=$BIN_PATH $BIN_PATH/rake -t test_latest_revision \
NAME="App" \
RECIPIENTS="dev-list@yourcompany.com" \
SENDER="build@yourcompany.com" 

Once the post-commit file is in place, you need to do make sure of a few things:

  1. That your post-commit file is owned by the user that the subversion process runs as
  2. That your post-commit file is executable by that user
  3. That the subversion user can checkout code to the build directory (easy as setting that user to be the owner of the build dir)

An easy way to test this setup is to run the post-commit file from the command line. Since it’s executable you should be able to just run ./post-commit. If you’ve recreated the fake failure scenario on the build server as you did on your local machine you should get the nasty-gram email sent to the dev-list@yourcompany.com email address. If not you should at least see the process output.

Test in Production

Now the only thing left to do is to test the builder on an actual commit. This is pretty easy, all you have to do is commit a test that fails. You should see an email pop up in your inbox saying that you wreaked havoc on the build, bringing the scorn of the developers upon you.

Damn – that was a lot. Hopefully you’re now living in continuous build nirvana…

Resources:

http://dev.rubyonrails.org/browser/plugins/continuous_builder

CIA Stuff:

http://wiki.rubyonrails.org/rails/pages/How+To+Use+CIA+For+Continuous+Integration http://article.gmane.org/gmane.comp.lang.ruby.rails.core/790 http://blog.innerewut.de/articles/2005/09/18/setting-up-cia-with-rails-and-subversion

tags: rubyonrails, continuous integration, agile

InfoQ: Introduction to BackgrounDRb
www.infoq.com/articles/BackgrounDRb

Introduction to BackgrounDRb

Posted by Ezra Zygmuntowicz on Jun 27, 2006 03:09 AM

Community
Ruby
Topics
Architecture,
Ruby on Rails,
Programming

Ruby on Rails is a great framework for developing many diverse types of web applications. As the problem domain of these web applications expands, you may need to run computationally intensive or long running background tasks. This poses a problem in that you are constrained to work within the request/response cycle of HTTP. So how can you run these long background tasks without your web server timing out? And how do you display the progress to your users?

Enter BackgrounDRb. This is a Rails plugin I wrote recently as one way to solve this problem. Ruby includes DRb (Distributed Ruby) as part of the standard library. DRb provides a simple API for publishing and consuming Ruby objects over TCP/IP networks or unix domain sockets. BackgrounDRb is a small framework that facilitates running background tasks in a separate process from Rails, thereby decoupling them from the request/response cycle. With DRb you can manage your tasks from Rails using hooks for progress bars or status updates to your users.

The BackgrounDRb server works by publishing a MiddleMan object. This object is the manager for your worker classes. It holds a @jobs hash composed of { job_key => running_worker_object } pairs and a @timestamps hash composed of { job_key => timestamp } pairs. The MiddleMan object straddles the interface between the DRb server and your Rails application. Here is a simple diagram to show the architecture.

This is a generic worker class as created by the worker generator provided by the plugin.

$ script/generate worker Foo
class FooWorker < BackgrounDRb::Rails
def do_work(args)
# This method is called in its own new thread when you
# call new worker. args is set to :args
end

end

When your FooWorker object is instantiated from rails via MiddleMan, the do_work method is automatically run in its own thread. We use a thread here so rails does not wait for the do_work method to finish before it continues on.

With BackgrounDRb, you usually create a new worker object with an AJAX request. Your view can then use periodically_call_remote to fetch the progress of your job and display it however you like. Let's flesh out the FooWorker class and show how you would create a new FooWorker object and retrieve its progress from within a rails controller.

class FooWorker < BackgrounDRb::Rails
attr_reader :progress
def do_work(args)
@progress = 0
calculate_the_meaning_of_life(args)
end
def calculate_the_meaning_of_life(args)
while @progress < 100
# calculations here
@progress += 1
end
end
end

Now in the controller:

class MyController < ApplicationController
def start_background_task
session[:job_key] =
MiddleMan.new_worker(:class => :foo_worker,
:args => "Arguments used to instantiate a new FooWorker object")
end
def get_progress
if request.xhr?
progress_percent = MiddleMan.get_worker(session[:job_key]).progress
render :update do |page|
page.call('progressPercent', 'progressbar', progress_percent)
page.redirect_to( :action => 'done') if progress_percent >= 100
end
else
redirect_to :action => 'index'
end
end
def done
render :text => "

Your FooWorker task has completed

"
MiddleMan.delete_worker(session[:job_key])
end
end

And in your start_background_task.rhtml view file you could use something like this:

    <html>
	      <head>
	       <style type="text/css">
	         .progress{
	           width: 1px;
	           height: 16px;
	           color: white;
	           font-size: 12px;
	           overflow: hidden;
	           background-color: #287B7E;
	           padding-left: 5px;
	         }
	       </style>
	       <script type="text/javascript">
	         function progressPercent(bar, percentage) {
	           document.getElementById(bar).style.width =  parseInt(percentage*2)+"px";
	           document.getElementById(bar).innerHTML= "<div align='center'>"+percentage+"%</div>"
	         }
	       </script>
	      </head>
	       <body>
	        <div id='progressbar' class="progress"></div>
	        <%= periodically_call_remote(:url => {:action => 'get_progress'}, :frequency => 1) %>
	       </body>
    </html>

MiddleMan.new_worker returns a randomly generated job_key that you can store in the session for later retrieval. If you want to specify a named key instead of using the generated key you can do so like this:

 # This will throw a BackgrounDRbDuplicateKeyError if the :job_key already exists.
 MiddleMan.new_worker(:class => :foo_worker,
                           :job_key => :my_worker,
                           :args => "Arguments used to instantiate a new FooWorker object")

      MiddleMan.get_worker :my_worker

Upon instalation, the plugin writes a config file into RAILS_ROOT/config/backgroundrb.yml. In this file there is a load_rails config option. If this is set to true then you will be able to use your ActiveRecord objects in your worker classes. When you start the server it will use your already existing database.yml file for database connection details.

This plugin can also be used for caching large or compute-intensive objects including ActiveRecord objects. You can store rendered views or large queries in the cache. In fact you can store any text or object that can be marshalled. Here is how you would use the cache:

# Fill the cache
@posts = Post.find(:all, :include => :comments)
MiddleMan.cache_as(:post_cache, @posts)
# OR
@posts = MiddleMan.cache_as :post_cache do
Post.find(:all, :include => :comments)
end

# Retrieve the cache
@posts = MiddleMan.cache_get(:post_cache)
# OR
@posts = MiddleMan.cache_get(:post_cache) { Post.find(:all, :include => :comments) }

MiddleMan.cache_get takes an optional block argument. If the cache located at the :post_cache key is empty, the results of evaluating the block are placed in the cache and assigned to @posts. If you don't supply a block and the cache is empty it will return nil.

In the current implementation, you are responsible for expiring your own caches and deleting your own workers from the main pool. This works two ways. You can either explicitly call MiddleMan.delete_worker(:job_key) or MiddleMan.delete_cache(:cache_key). There is also a MiddleMan.gc! method that takes a Time object and deletes all jobs with a time-stamp older than the one specified. Here is a script that can be run from cron to expire jobs older than 30 minutes:

#!/usr/bin/env ruby
require "drb"
DRb.start_service
MiddleMan = DRbObject.new(nil, "druby://localhost:22222")
MiddleMan.gc!(Time.now - 60*30)

In the near future there will be a timing mechanism built into BackgrounDRb. This will allow for jobs and garbage collection to be run at scheduled times and for specifying a time-to-live parameter when you create new jobs or caches.

There are Rake tasks as well as plain Ruby command line scripts to start and stop the daemon. On OS X, linux or BSD you can use the Rake tasks to start and stop the server:

$ rake backgroundrb:start
$ rake backgroundrb:stop

On Windows you currently have to keep a console window open while you run the backgroundrb server (Hopefully this will change in the near future). So on Windows, to start the daemon you would open a console and run the command like this:

> ruby script\backgroundrb\start
# ctrl-break to stop

So what are a few real world use cases, you ask? Here is a small list of things I am currently using BackgrounDRb for:

  • Downloading and caching RSS feeds for a feed aggregator.
  • Screen scraping automation using watir to drive a web browser that navigates to other websites in the background to collect information.
  • Automating Xen VPS creation and sysadmin tasks.
  • Creating indexes in the background for Hyper Estraier and ferret search technologies.
  • Bridging Rails and IRC bots.

Plans for the future include the ability to fork new processes to handle larger jobs that require their own Ruby interpreter instance. Also work needs to be done to let BackgrounDRb run as a Windows service. Anyone who is familiar with Windows services that can offer some help here would be greatly appreciated. Suggestions and patches are also welcome.

  • rubyforge project
  • Blog
  • install as plugin: script/plugin install svn://rubyforge.org//var/svn/backgroundrb
Hivelogic: Articles: Building RMagick on Mac OS X
hivelogic.com/articles/2006/06/10/rmagick_os_x

Building RMagick on Mac OS X

Jun. 11, 2006 | 214 words
Development | Apple | Mac OS X | Ruby on Rails

There are many cool things you can do with the RMagick gem. Unfortunately, it has a handful of pre-requisites that are hard to track down. Worse, some packages have “issues” with Mac OS X, so finding just the right combination of versions can be tricky.

Because there’s nothing fun or interesting about building and installing the prerequisites, I’m providing the instructions for you below in one big block. You shouldn’t see any errors, and you can ignore most of the output. If all goes well, after about an hour, you’ll have a working installation of RMagick on OS X installed correctly into /usr/local (why this is important).

curl -O http://download.savannah.gnu.org/releases/freetype/freetype-2.1.10.tar.gz
tar xzvf freetype-2.1.10.tar.gz
cd freetype-2.1.10
./configure --prefix=/usr/local
make
sudo make install
cd ..

curl -O http://superb-west.dl.sourceforge.net/sourceforge/libpng/libpng-1.2.10.tar.bz2
bzip2 -dc libpng-1.2.10.tar.bz2 | tar xv
cd libpng-1.2.10
./configure --prefix=/usr/local
make
sudo make install
cd ..

curl -O ftp://ftp.uu.net/graphics/jpeg/jpegsrc.v6b.tar.gz
tar xzvf jpegsrc.v6b.tar.gz
cd jpeg-6b
ln -s `which glibtool` ./libtool
export MACOSX_DEPLOYMENT_TARGET=10.4
./configure --enable-shared --prefix=/usr/local
make
sudo make install
cd ..

curl -O ftp://ftp.remotesensing.org/libtiff/tiff-3.8.2.tar.gz
tar xzvf tiff-3.8.2.tar.gz
cd tiff-3.8.2
./configure --prefix=/usr/local
make
sudo make install
cd ..
curl http://superb-east.dl.sourceforge.net/sourceforge/ghostscript/ghostscript-8.54-gpl.tar.gz | tar xfz -
cd ghostscript-8.54-gpl/
./configure --prefix=/usr/local
make
sudo make install
cd ..
 
curl ftp://mirror.cs.wisc.edu/pub/mirrors/ghost/GPL/current/ghostscript-fonts-std-8.11.tar.gz | tar xfz -
sudo mv fonts /usr/local/share/ghostscript
cd ..


curl -O http://easynews.dl.sourceforge.net/\
sourceforge/imagemagick/ImageMagick-6.2.8-0.tar.gz
tar xzvf ImageMagick-6.2.8-0.tar.gz
cd ImageMagick-6.2.8
./configure --prefix=/usr/local
make
sudo make install
cd ..

sudo gem install RMagick

That’s it, you should have RMagick installed!

Demo :
full_filename = RAILS_ROOT + '/file_system/test.pdf'
image = Magick::Image.read("#{full_filename}[0]").first
thumb =  image.crop_resized(50,50)
image.format = "GIF"
thumb.write(full_filename_thumb.gif)
 
Installing RMagick on OS X
rmagick.rubyforge.org/install-osx.html
Installing RMagick on OS X

This page describes a method for installing RMagick, ImageMagick or GraphicsMagick, and the delegate libraries used by ImageMagick and GraphicsMagick. You only need to install one of ImageMagick or GraphicsMagick. The procedure is the same for either library. Throughout the remainder of this document I will use the word ×Magick to refer to either of these two libraries. I developed this procedure using a Powerbook G4 and Mac OS X 10.3.8, and I've tested it with Tiger. If you are using a different version of Mac OS X some of the details may be different. In particular these instructions assume you are using bash as your shell.

You will need to have your Mac OS X installation disks, a connection to the Internet, and at least an hour of free time (assuming you have a broadband connection).

Step 1. Install X11, the Xcode Tools, and the X11SDK.

×Magick needs the X11 fonts and a X server to display images, so you'll need to install X11. You also need the Xcode tools and the X11SDK.

Install X11, the Xcode Tools and the X11SDK. from the OS X installation disk. See http://developer.apple.com/darwin/runningX11.html for more information.

Step 2. Install DarwinPorts

Go to http://darwinports.opendarwin.org/getdp/ and follow the instructions to download and install DarwinPorts. The remainder of this document assumes that you take all the defaults during the installation. In particular, the default location for installing the delegate libraries is the /opt/local directory.

Step 3: Install the delegate libraries

Here we'll use DarwinPorts to install delegates for popular image formats and that are needed to run the RMagick example programs. Enter the following port commands:

sudo port install jpeg
sudo port install libpng
sudo port install libwmf
sudo port install tiff (see note below)
sudo port install lcms
sudo port install freetype
sudo port install ghostscript

(Our correspondent Al E. reports that as of this writing libtiff 3.8.0, the most recent version of libtiff available from DarwinPorts, is completely broken. Al recommends installing libtiff 3.8.1 or later. 23Mar2006)

Note that some of these libraries have prerequisites which will be automatically installed.

Before proceeding, you need to make sure you're using the correct version of the FreeType library. The X11 files you installed in Step 1 include a version of the FreeType library, and of course you just installed another version in /opt/local using DarwinPorts. You need to use the DarwinPorts version when you're building ×Magick. To make sure you have the right version, enter the command:

freetype-config --cflags
You should see this output:
-I/opt/local/include/freetype2 -I/opt/local/include
If you see the following output instead,
-I/usr/X11R6/include -I/usr/X11R6/include/freetype2

edit your $PATH to make sure that /opt/local/bin preceeds /usr/X11R6/bin. Do not try to install ×Magick until you get the correct output from the freetype-config program.

Step 4: Install ImageMagick or GraphicsMagick

Go to http://www.imagemagick.org or http://www.graphicsmagick.org and download the latest version of the software to a temporary directory. Unroll the tarball and make the new directory current. For example, if you downloaded the ImageMagick.tar.gz file, use these commands (where X.Y.Z is the ImageMagick version number):

  tar xvzf ImageMagick.tar.gz
  cd ImageMagick-X.Y.Z

Similarly, if you downloaded the GraphicsMagick-LATEST.tar.gz file, use these commands (where X.Y.Z is the GraphicsMagick version number):

  tar xvzf GraphicsMagick-LATEST.tar.gz
  cd GraphicsMagick-X.Y.Z

To configure ×Magick, enter these commands:

export CPPFLAGS=-I/opt/local/include
export LDFLAGS=-L/opt/local/lib
./configure --prefix=/opt/local --disable-static --with-modules --without-perl \
   --without-magick-plus-plus --with-quantum-depth=8 \
   --with-gs-font-dir=/opt/local/share/ghostscript/fonts

The ./configure command should be entered on a single line. The --prefix=/opt/local option will cause ×Magick to be installed in the same directory as the libraries we installed with DarwinPorts. If you want to install ×Magick somewhere else, specify a different directory. If you do not specify the --prefix option ×Magick will be installed in /usr/local. The --disable-static and --with-modules options cause ×Magick to be built with dynamically loaded modules. Since you're installing ×Magick for use with Ruby, I've included the --without-perl and --without-magick-plus-plus options to suppress the Perl and C++ support. The --with-quantum-depth=8 option configures ×Magick to use a bit depth of 8. If you need to build with a different bit depth (and if you need to you'll already know it) you can specify 16 or 32. Finally, the --with-gs-font-dir option tells ×Magick where the Ghostscript fonts are installed.

For more information about all these options see ×Magick's README.txt file.

./configure will produce quite a bit of output. The last page is the most interesting. If you've successfully performed all the steps so far and used all the deafults, the output from configure should end with a page like this:

ImageMagick is configured as follows. Please verify that this configuration matches your expectations.

Host system type : powerpc-apple-darwin7.8.0

Option Value
-------------------------------------------------------------------------
Shared libraries --enable-shared=yes yes
Static libraries --enable-static=no no
Module support --with-modules=yes yes
GNU ld --with-gnu-ld=no no
Quantum depth --with-quantum-depth=8 8

Delegate Configuration:
BZLIB       --with-bzlib=yes           yes
DPS         --with-dps=yes             yes
FlashPIX    --with-fpx=no              no
FreeType 2.0 --with-ttf=yes            yes
Ghostscript None                       gs (8.14)
Ghostscript fonts --with-gs-font-dir=/opt/local/share/ghostscript/fonts /opt/local/share/ghostscript/fonts/
Ghostscript lib   --with-gslib=no      no
Graphviz    --with-dot=yes             no
JBIG        --with-jbig=yes            no
JPEG v1     --with-jpeg=yes            yes
JPEG-2000   --with-jp2=yes             no
LCMS        --with-lcms=yes            yes
Magick++    --with-magick-plus-plus=no no
PERL        --with-perl=no             no
PNG         --with-png=yes             yes
TIFF        --with-tiff=yes            yes
Windows fonts --with-windows-font-dir=none
WMF         --with-wmf=yes             yes
X11         --with-x=                  yes
XML         --with-xml=yes             yes
ZLIB        --with-zlib=yes            yes

X11 Configuration:
X_CFLAGS = -I/usr/X11R6/include

X_PRE_LIBS = -lSM -lICE
X_LIBS = -L/usr/X11R6/lib
X_EXTRA_LIBS =

Options used to compile and link:
PREFIX = /opt/local
EXEC-PREFIX = /opt/local
VERSION = X.Y.Z
CC = gcc
CFLAGS = -g -O2 -Wall
CPPFLAGS = -I/opt/local/include
PCFLAGS =
DEFS = -DHAVE_CONFIG_H
LDFLAGS = -L/opt/local/lib -L/opt/local/lib -L/usr/X11R6/lib -L/opt/local/lib -lfreetype -lz -L/usr/lib
LIBS = -lMagick -llcms -ltiff -lfreetype -ljpeg -lXext -lSM -lICE -lX11 -lXt -lbz2 -lz -lpthread -lm -lpthread
CXX = g++
CXXFLAGS =

Of course, instead of VERSION X.Y.Z you will see the version number of the version of ×Magick that you downloaded. Check your output to make sure that ×Magick located all the delegate libraries. You should see "yes" in the Value column for bzlib, FreeType 2.0, JPEG v1, LCMS, PNG, TIFF, WMF, X11, XML, and ZLIB.

If you get this output from ./configure you're ready to proceed. If you are missing some delegates you should resolve those issues before continuing. Re-run ./configure, being very careful to enter the commands correctly.

Once you're satisfied that you've configured ×Magick the way you want it, enter these two commands:

make
sudo make install

Where to go for more information

Check this page for in-depth information about installing ImageMagick. Check this page for more information about installing GraphicsMagick.

Step 5: Install RMagick

The hard part is done. All we have to do now is install RMagick. If you haven't already done so, download the RMagick tarball from Rubyforge and unroll it into a temporary directory. Make that directory current.

  tar xvzf RMagick-X.Y.Z.tar.gz
  cd RMagick-X.Y.Z

Enter these commands

./configure
make
sudo make install

The make step will take a few minutes to run since it builds all of the RMagick examples. That's it. You should have a complete install of ×Magick and RMagick.

Back to the FAQ.

Ubuntu Linux: error while loading shared libraries: libstdc++.so.5 - RealPlayer - Real Support Commu
real.lithium.com/real/board/message?board.id=realp...
sudo apt-get install libstdc++5

Postfix Enabler for Mac OS X

What is Postfix Enabler?

Postfix Enabler helps Mac users set up a totally-functional buzzword-compliant mail server in less than a minute, the Mac Way. It sets up SMTP, POP3 and IMAP services, with or without SSL support. It even sets up SSL test certs so that you can test the SSL connection. It enables SASL Authentication so you can connect to ISPs who require the SMTP connection to be authenticated. Or, the other way around, it allows you to turn on SMTP-AUTH on the server, so that you can authorise remote users who need to send mail through it.

Plus, it has a few other features, including the ability to set up a roving SMTP server for PowerBook users to send mail wherever they are, whenever they want, so long as they have an Internet connection.

Now it also works with Tiger and Macintels

10.4.8 Update : Works with OS X 10.4.8. The "WorkAround Bonjour" stall that Apple introduced with 10.4.7 is still there (please read the 10.4.7 notes in my weblog). However this doesn't affect the Postfix, IMAP, and POP services which are all still enabled OK and continue to work in 10.4.8.

Release Notes 1.2 15th March 2006. This is a Universal Binary release.
If you're upgrading from a previous version of OS X to Tiger, use the Red Cross (at the top left hand corner of the Postfix Enabler window) to re-enable the Enable Postfix button. Clicking Enable Postfix now will force Postfix Enabler to check through your system and make sure you've got everything needed to turn on Postfix.

Note 1. Do not use this Mac OS X Hint about editing the /etc/sudoers file. It'll really mess up things up when you're using Postfix Enabler.

Note : Some mail servers may reject mail coming from a dynamic IP address, on the (flawed) assumption that such mail must be from a spammer. But this is how I've set things up so that I can continue to send mail from anywhere I happen to be.

The One-Click Road Warrior's Guide

Start up Postfix Enabler. Hit the Enable Postfix button. And that's it.

This turns on the built-in SMTP server in Mac OS X and, in most cases, this would be it. You don't have to type in anything else and you can move on to the Setting Up Mail.app section.

Problems?

However, you may find yourself faced with a few more obstacles before you can send mail successfully. For example, you may find that the ISP of the network (that you're currently on) is blocking your ability to send mail, e.g., by blocking port 25, the smtp port.

You can test this using Terminal. First, type in this command:

telnet localhost 25

If you see a :

Connected to localhost.
Escape character is '^]'.
220 iBook.local ESMTP Postfix

it means Postfix Enabler has successfully set up your Mac to send mail. You can type 'quit' at this point to get out of the telnet session.

Now, do a :

telnet cutedgesystems.com 25

If you do not see the "ESMTP Postfix response" from the remote server, cutedgesystems.com, but instead the session times out, then you can deduce that the outgoing port is being blocked by the ISP.

The first panel of Postfix Enabler gives you a few options to get around that.

Using Panel 1 of Postfix Enabler - The Send Mail Tab

If your ISP requires you to go through their mail server, enter their server name into the Smart Host field (otherwise leave it blank). Your built-in SMTP server will then contact this Smart Host and relay mail through it.

In addition, if your ISP requires you authenticate against its SMTP server, you can use the Enable SASL Authentication check-box to turn on SMTP authentication. Enter the ISP's mail server address and your userID:password combination, as shown in the example below :

Finally, if you want your message to look like it's being sent from a particular domain (and avoid the "May be forged" headers that some ISPs' servers tag onto it), enter that doman name into the Masquerade As field.

The Masquerade As field is particularly important for PHP programmers using PHP's built-in mail function. Enter a doman name into the Masquerade As field that corresponds to to the e-mail address that you want all replies to come back to, and you will find that your messages will get to their destination safely from the PHP scripts. Without this, the messages will get rejected.

Please note that the Masquerade As field and the Domain Name field in Panel 2 are mutually exclusive. If you are running a full-fledged mail server, do not use the Masquerade As field.

The last field on Panel 1 is the Message Size Limit. Set to 0 for no limit.

After any of these changes, hit the Restart Postfix button for the changes to take affect.

Benefits of having your own SMTP server

For roving PowerBook users, this could sometimes be the only way you get to send mail. E.g., this is from someone who does the reality TV show, Survivor:

"Thanx for a great program. I’m constantly traveling to different parts of the world doing TV productions. Sometimes we get an Internet  connection at the location we are shooting at but no SMTP server to send mail. So Postfix Enabler has saved me many times by making it possible to send mail from remote locations."

For PHP programmers, web designers, and other software developers, it's often useful to set up a local SMTP server on the development machine and communicate with it through "localhost". This is because you can let the local SMTP server do the job of communicating with a Smart Host, or set up the SSL connections, if required, or work with the DNS System, without your having to figure out what to do to effect these in your code. In your code, you simply talk to "localhost" and leave it to the Postfix Enabled-SMTP server to do the rest.

Setting Up Mail.app

This is how you set up Mail.app to talk to the local SMTP server.

Set up the POP or IMAP account information the usual way, for the Incoming Mail Server. Set them to point to whichever mail server is providing the POP or IMAP services.

You can use Postfix Enabler set up POP and IMAP services so you can become your own ISP - see the section on Panel 2 and 3 of Postfix Enabler, below.

To use use your own built-in SMTP server, set the Outgoing Mail Server to localhost or 127.0.0.1, as shown above, and that's it. Make sure that the Authentication pop-up menu is set to none because you don't need to authenticate with your own built-in mail server.

(... though the built-in mail server could, in turn, be made to authenticate with other mail servers, so it can relay mail out through them, as I had mentioned earlier.)

If you use Eudora or Entourage, you can set them up in a similar way.

In summary, this is what you're doing. You set up your POP or IMAP accounts so that replies coming back to you will reach you on your mail client. But the messages sent out your Mac via localhost will be despatched directly to the recipients.

Note for PowerBook users : You may like to know if a large attachment has been sent out your machine, so you can close your PowerBook lid. Look into the Log Panel and check for a mail log entry that indicates Status=Sent for that particular message.

Warning: If you're only going to send mail out and not trying to set up a full mail server (see next section), do not use the Mail Server Panel because the settings for the two situations are slightly different. Specifically, do not enter a domain name into the Mail Server Panel because it will cause Postfix to hold on to mail that are addressed to people on that domain, rather than sending them out.

The One-Click Mail Administrator's Guide

Postfix Enabler can be used to set up a fully functioning mail server, complete with POP3 and IMAP services. Workstations (which include PCs) on the local network can use this server to relay mail to each other, as well as to send them out to the rest of the world. This section describes how you would set this up.

First, make sure that you have used the first panel (the "Send Mail" tab) to Enable Postfix and you have tried to send mail out successfully to another mail server. If not, please review the first section.

Then, go to the second panel, the "Mail Server" tab. Make sure you have a valid domain name and that it is pointing correctly to your server machine. If it is, enter it into the domain name field. In the example above, my domain name is cutedgesystems.com.

Once I've done this, I can click on Restart Postfix, and I've set up a mail server for the domain cutedgesystems.com that all machines on the same local network as the server can send mail through.

Please note : When you're setting up a mail server that is accessible by the rest of the world, you must have a valid domain name. Check out this tutorial if you want to try this out using a free domain name.

You need to check that the domain name works. The simplest way to do this is to turn on the web server on the same machine you are using to run your mail server (using OS X's Sharing Preferences). Then, fire up a web browser, like Safari, and see if you can hit the web pages that you know you have on this machine.

Setting up POP3 and IMAP Services

It's important to realise at this point that you need to set up user accounts on the mail server to collect (and act as diistribution points) for the in-coming mail.

To create an account for a mail user, simply create a New User on that server machine using the System Preferences -> Accounts panel.

Once you've created your user accounts on the server, you can choose between two different mechanisms that will allow your mail users to download their in-coming mail to whatever machine they happen to be using as their workstation.

POP3 is a simple mechanism for transferring mail to a mail client software like Eudora, Mail.app, or Entourage. IMAP is a "smarter" system because you can use more than one machine to read your mail and the state of your mail box is synchronised across all these machines (in terms of the messages last read, state of drafts, etc.)

You can set up both POP3 and IMAP services using Postfix Enabler and have both running at the same time, allowing your users to choose which service they prefer.

So, next, you will need to enable either POP3 or IMAP services (or both) so that all the machines and users on your network can retrieve their incoming mail.

Leave all the other settings alone, for the moment, and click on the Enable POP3 button or the Enable IMAP button, depending on which mode of mail service you prefer to run.

Hit the Restart Postfix button.

Check that it works

Assuming that my domain name is cutedgesystems.com, this is how I'll set up the mail client, OS X's Mail.app, on each user's machine. Test it first on the local machine, i.e., the same machine you're using to run your server.

The User Name and Password fields will correspond with the name and password of a user you had created using the Systems Preferences - Accounts Panel on the server machine. (If you've enabled the IMAP server, you can also use the Account Type: IMAP).

When you are ready, use Mail to send mail out to anybody you know and see if you can get a reply. The replies will come back to the same server. You can pick them up using Mail because Postfix Enabler has equipped your server with POP3 services.

The next step is to share the mail server with all the other machines on your network.

Share the Mail Server

Via an Airport Base Station

Mac users typically share an Internet connection in three ways. One way is to use an Airport Base Station to connect to the Internet and then share its connection. There's a tutorial (OS X, Broadband, and the Airport Base Station - but pay special attention to the section covering DNS) which will show you how to get a server running behind an Airport Base Station. In this case, if you've set up Mail for the other machines in the way shown above, you've really got nothing else to do. So long as you've got your DNS settings right (so that your other machines know where your mail server is), the other machines can now use your mail server to relay mail.

Via Internet Sharing

The second way to share an Internet connection is to turn on Internet Sharing on the mail server machine. If your mail server is equipped with an Airport card, this is really easy. The Airport card allows the server to create a secondary internal IP network which the rest of your machines can get up on, provided they're also equipped with Airport.

In this case, besides setting Mail in the way shown above, you've also got one more thing to do on your server. By default, the Airport network created by the mail server will use a network in the range 10.0.2.x (please confirm that this is true before proceeding).

Use Postfix Enabler, look for the Access field, and enter the following into a new line in the Access field :

10.0.2 OK

This tells the mail server to allow all machines on the internal 10.0.2.x network to relay mail through the server.

Via a Router

The third way to share an Internet connection is via a router. The things you have to do here are a combination of steps from the first two methods described above. You have to enable port mapping on your router to make sure that ports 25 and 110 are mapped to the specific internal IP address you have reserved for your server (say, 192.168.2.18).

Then, you have to ask Postfix to relay mail for your internal network, which should be 192.168.2, in the example above. Use Postfix Enabler, look for the Access field, and enter the following into a new line in the Access field:

192.168.2 OK

This tells the mail server to allow all machines on the internal 192.168.2.x network to relay mail through the server.

Please note : between the three ways, described above, for sharing an Internet connection, the ones with the router or Airport Base Station are the safer options. This is because you're situating the server on a private network behind the router or Base Station. Postfix is programmed, by default, to reject all attempts to relay mail through it by machines sitting outside its local network. In an Airport network, this network spans the private 10.0.1.x range. The mail server will relay mail only from its own 10.0.1.x network, rejecting all other attempts.

The other way of sharing an Internet connection, through OS X Internet Sharing, though cheaper and more convenient, exposes your server to attempts to relay mail through it by other machines sitting on your ISP's network (because it is sitting directly on your ISP's network). In this latter case, you should use Postfix Enabler's Custom Postfix Settings field and add this one line :

mynetworks_style = host

To allow your other machines and users to route mail through this server, then, you should set up SMTP Authentication on the server - see Panel 3 of Postfix Enabler.

Other uses for the Access field

The Access field can be used to blacklist individual mail senders from sending mail to your site, or even entire domains.

spammer@yahoo.com REJECT
spamUnlimited.com REJECT

The Aliases Field

Some required entries for Aliases are already created for you. Each site needs to have a Postmaster and a Root user so that other ISPs and you own system processes can contact a responsible person when they find problems with your system. MAILER-DAEMON is the conventional name attached to bounced messages. When senders find that their messages have bounced, they may need to contact someone for clarification. Their replies to their bounced messages will go to MAILER-DAEMON, so you need someone to pick these up.

The first line in the example, below, shows that you can create e-mail groups quickly by entering a group name on the left-hand side of an Alias entry, and entering a series of user names, separated by commas, on the right-hand side, which can include users from other domains.

nightrunner: haihwee,beekhim,brendan@sky.com
postmaster: bernard
root: bernard
MAILER-DAEMON: bernard
mailist: :include:/full/path/name/to/mailinglist.txt

The last line in the example, above, shows another way of creating e-mail groups - by pointiing the mail server to a file that contains a list of e-mail addresses, with one address on each line.

Forward mail without valid recipients to - the Catch-All mailbox

You can choose who, among your users, gets to be swamped by mail that has been sent to no one with that name on your server. If you elect not to nominate anyone, i.e., leave the pop-up menu for the catch-all mailbox blank, all messages for which there is no valid recipient will be bounced back to the sender. Actually, this is the suggested option, if you don't want to be swamped by junk mail.

The Additional Domain Names Field

If your server hosts more than one domain, you can list the additional domains in this field (separated by commas) so that Postfix knows that it has to accept messages sent to these domains. Make sure that these domain names work first and that they're also pointing correctly to your server machine.

There is no separation between users into particular domains. A user may get mail addressed to any of the domains. E.g., on my server, mail sent to bernard@cutedgesystems.com and mail sent to bernard@roadstead.com will all reach me in my single mail box on the server.

Relay Mail From - the server machine only or all machines on subnet

This option allows you to prevent your Mac acting as an open relay if you've placed it directly on a broadband line. The default setting is to allow all machines on the same subnet as the server to relay mail through it without needing to authenticate, which is convenient for getting a shared server up quickly. But if you've placed the server directly on a broadband or dial-up line, then you will have all machines sitting on your ISP's network becoming your local network, inadvertently creating an open relay.

Clicking on the "Relay Mail From : This server machine only" choice will close up the open relay. If you need to still allow mail relay from known users, turn on authentication. This will be the safest option.

The Custom Postfix Settings field

This is meant to allow experienced Postfix users to add their own modifications to the Postfix configuration that have not been taken care of by the Postfix Enabler user interface.

These will stick in the Postfix config file at /etc/postfix/main.cf and will not be over-written when you do a Restart Postfix from Postfix Enabler. (In this way, Postfix Enabler works a little better than OS X Server's Mail Admin tool).

Addtional Note for Outbound Mail

If you're running a mail server and your ISP requires you to go through their mail server for outbound mail, enter their server name into the Smart Host field (otherwise leave it blank) on the Send Mail panel.

In addition, if your ISP requires that you authenticate against its SMTP server, turn on SASL authentication and enter the ISP's mail server address and your userID:password combination into the relevant fields on the Send Mail panel.

Also, if your ISP requires that the authentication be done in SSL, you're all set to go by turning on SSL mode in Panel 3 of Postfix Enabler, below.

The Postfix Enabler Advanced Tab

This allows the administrator to turn on SMTP-AUTH for the mail server. It allows the mail server to be accessed remotely by authorised users, whose name:password combinations have been registered with the server. The Advanced tab also allows the mail administrator to quickly create self-signed SSL certs for testing secured connections to and from the mail server.

If you need to turn on SMTP Authentication, you have two choices - use the built-in OS X user accounts or SASLDB.

The first method is so simple to use. It authenticates against the Mac's built-in user account management - so you maintain just one set of passwords, using System Preferences. Turn it on and you're done. (But the downside is that passwords are sent in the clear, unless you turn on SSL encryption, as shown below and explained in the SSL section.)

In Mail.app, under Outgoing Mail Server, click on Server Settings, and set up the SMTP Server Options, as shown below. You need to make sure you enter the same User Name and Password combination that you gave to this user, using the server's OS X System Preferences panel :

SASLDB is considered to be more secure because passwords are never sent down the wire, only tokens. If you choose to turn on SMTP Authentication via SASLDB, you will need to provide the server with a list of username:password combinations, for each user who will be needing to send mail remotely through the server.

Please note that SASLDB support in Panther broke when 10.3.9 came out. This will now only work on Tiger. Please upgrade to TIger if you need this feature.

Then, in Mail.app, under Outgoing Mail Server, click on Server Settings, and set up the SMTP Server Options, as shown below. That is, set Authentication to "MD5 Challenge-Response".Then enter the username:password combination that was registered for this user on the server, using Postfix Enabler's Advanced Pane.

SSL (Secure Sockets Layer)

You can use the Advanced Panel to turn on or off SSL mode to encrypt the communications between client and server, over SMTP, POP, and IMAP. However, you will need to have the appropriate SSL certs in /System/Library/OpenSSL/certs before you can enable SSL.

You can use this panel to create test (self-signed) certs to test the SSL connection to and from the mail server. You can always replace them with "real" certs, of the same name, in the future.

If you're testing the SSL connection, make sure you quit Mail.app and come back in again, when you switch the server from non-SSL to SSL mode. This is important, and had been the source of quite a few support calls. Mail.app seems to cache the information it keeps about a connection. If you switch modes, in mid-stream, it may get confused and you will see a connection error until you quit Mail.app and come back in.

Also, if you're using the self-signed test certs, you will see the following dialog box thrown up by Mail.app, when you first send mail over SSL :

This is OK. It shows that the SSL mode is working. The cert used is a self-signed cert that hasn't been verified by any of the known certification services, e.g., Verisign. The cert can still be used to enable SSL encryption between client/server communications. If you click on "Show Certificate", it will show you the data you have set for this certificate (if you've updated the Country/State/Locality fields before clicking on the "Create SSL Test Certs" button. You can always replace the test certs with "real" certs of the same name. They are stored in /System//Library/OpenSSL/certs.

The Postfix Enabler Log Panel

You can use this panel to monitor the mail log. The Get button retrieves the last 30 (or so) records from the tail-end of the mail log, in reverse order. Because the table does not have enough width to show all the details of the connection, you can click on any line and the information will be re-displayed in the detail-fields below the table.

The mail log can be used to check if a large attachment has been sent out the mail server, as in the case of a PowerBook user. Look into the mail log for a Status=Sent indicator for the specific message and destination.

There is also a Postfix Config Summary button at the bottom of the panel. When clicked it will show a summary of the active Postfix Configuration Parameters. If you know enough Postfix, this is useful for checking if the system is set up the way the GUI says it has been set up.

Note that you can print out both the mail log and the Postfix configuration summary. (Actually, you can print any piece of information by just clicking on it, to give it the focus, before doing a Print from the File menu.)

The Mail Server and the OS X Firewall

You should check your mail server machine to see if you have OS X's built-in firewall turned on. If so, you should learn how to set it so that information could still pass through to your mail server. Look here to see how this is done.

In summary, you should open, at least, ports 25 (smtp), 110 (pop3) and 143 (imap) in the firewall. If you've turned on SSL, you should also make sure ports 995 (pop3 over SSL) and 993 (imap over SSL) are opened.

Release Log

1.0. 27th October 2003. Postfix Enabler 1.0 released without POP server.

1.0.1 2nd November 2003. Released with a POP server.

1.0.2 4th November 2003. Added ability to re-enable the Enabler, in case new system updates overwrites the current configuration.

1.0.3 6th November 2003. Made sending mail and administering a mail server into two distinct pieces, so it's clearer you can use just the first part without using the other. Also, the configuration for a mail server is slightly different from one that would only support outgoing mail.Made the changes to reflect that. Finally, the system should now work for Macs that have been upgraded to Panther, rather than via a clean install.

1.0.4 16th November 2003. Includes both an IMAP and a POP3 server from the UW-IMAP project, with permission. Both modes of operation support SSL. Added the UW-IMAP license agreement into the user interface, so that the user has to agree with the disclaimers before installing the POP3 and IMAP binaries.

1.0.5 30th December 2003. An Admin Password is requested only once, on startup. Added ability to enable SASL Authentication for the Postfix SMTP client and server. Users can now create SSL test certs from within Postifx Enabler with just 1 click. Added ability to set Message Size Limit.

1.0.6 1st January 2004. There is one bug fix. The "auxprop_plugin" line in /usr/lib/sasl2/smtpd.conf should read "auxprop_plugin: sasldb" instead of sasldb2. This prevented SASL Authentication for the server from working properly.

1.0.7 6th January 2004. Added the ability to authenticate the in-coming SMTP connection against the built-in OS X user accounts which, unlike SASLDB, does not require the user to maintain a separate password database. This solution was contributed by Andy Black. Also, thanks to Eric Kuo, we now also have a Traditional Chinese interface.

1.0.8 9th January 2004. Added the ability to turn on or off SSL mode.

1.0.9 15th January 2004. Added the ability to look into the mail log, get a summary of the active Postfix configuration, and append custom Postfix parameters to that provided by the user interface

1.0.10 2nd November 2004. Updated the POP3 and IMAP binaries with the latest from 2004 UW/IMAP release. Made one important oft-requested change to where IMAP stores its mailboxes, so that it will work nicely with Mail.app. They're now stored at each user's ~/Library/Mail/IMAP folder on the server.

1.1 29th April 2005. The first version compatible with OS X Tiger 10.4. Rewrote everything in Objective-C. Window is now re-sizeable.

1.1.1 30th April 2005. The release version of OS X Tiger is missing some of the things essential for running Postfix. This version helps to put them back.

1.1.2 12th May 2005. SMTP Authentication via SASLDB broke in Tiger. This release fixes it. It also contains an updated French translation from Michel Pansanel from http://www.carpo.org. Thanks, Michel.

1.1.3 13th May 2005. This version will work also on systems that have been formatted case-sensitive - i.e., as Mac OS X Extended (case sensitive).

1.1.4 15th May 2005. This version accepts admin passwords containing diacriticals like accents and umlauts.

1.1.5 17th May 2005. Compatibility fix. An interim solution to the problem of POP and IMAP not starting up after a reboot on systems that have the Tiger 10.4.1 update.

1.1.6 24th May 2005. Conforms to Tiger's new way of launching system services, using launchd. Use the Red Cross (in the top left hand corner of the Postfix Enabler window) to re-enable the Enable Postfix button. Clicking Enable Postfix now will shift you over to launchd. POP and IMAP services launch more reliably now even on 10.4.1.(Will continue to launch services in the "old" way on Panther). This release also fixes the problem where a PowerBook refuses to go to sleep when running Postfix. Also, the serial number field is now more forgiving of leading and trailing spaces.

Version 1.2 Release Notes

1.2 15th March 2006. This is a Universal Binary release. The application and the included POP and IMAP binaries, as well as the saslpasswd2 tool needed for sasldb password authentication, are now all Universal Binaries. (If you are running Potfix Enabler on an Intel Mac, just use the Red Cross to re-enable the Enable buttons and you will be able to replace the current POP and IMAP executables with Universal Binaries.) Also, new in 1.2, is support for using the Keychain to store the admin password and log into Postfix Enabler automatically.

1.2.1 20th April 2006. This release adds a Japanese localisation, originally undertaken by Chiang Hai Hwee, with a lot of help from Takashi Yoshida (thanks Takashi, always in your debt), and also to Makoto Imai for the suggestions for improvement and encouragement.

1.2.2 5th May 2006. Added a radio button (the Relay Mail button in the Mail Server panel) for the user to make sure that the server is not acting as an Open Relay when it's placed directly on a broadband line, as opposed to being behind a router or Airport Base Station.

Running Hadoop on Amazon EC2

Amazon EC2 (Elastic Compute Cloud) is a computing service. One allocates a set of hosts, and runs ones's application on them, then, when done, de-allocates the hosts. Billing is hourly per host. Thus EC2 permits one to deploy Hadoop on a cluster without having to own and operate that cluster, but rather renting it on an hourly basis.

This document assumes that you have already followed the steps in Amazon's Getting Started Guide.

Concepts

  • Abstract Machine Image (AMI), or image. A bootable Linux image, with software pre-installed.

  • instance. A host running an AMI.

Conventions

In this document, commands lines that start with '#' are executed on an Amazon instance, while command lines starting with a '%' are executed on your workstation.

Building an Image

To use Hadoop it is easiest to build an image with most of the software you require already installed. Amazon provides good documentation for building images. Follow the "Getting Started" guide there to learn how to install the EC2 command-line tools, etc.

To build an image for Hadoop:

  1. Run an instance of the fedora base image.

  2. Login to this instance (using ssh).

  3. Install Java. Copy the link of the "Linux self-extracting file" from Sun's download page, then use wget to retrieve the JVM. Unpack this in a well-known location, like /usr/local.

    # cd /usr/local
    # wget -O java.bin http://.../jdk-1_5_0_09-linux-i586.bin
    # sh java.bin
    # rm java.bin
    
  4. Install rsync.

    # yum install rsync
    
  5. (Optional) install other tools you might need.

    # yum install emacs
    # yum install subversion
    

    To install Ant, copy the download URL from the website, and then:

    # cd /usr/local
    # wget http://.../apache-ant-1.6.5-bin.tar.gz ( wget http://people.apache.org/dist/ant/v1.7.0Beta3/src/apache-ant-1.7.0Beta3-src.tar.gz )
    # tar xzf apache-ant-1.6.5-bin.tar.gz
    # rm apache-ant-1.6.5-bin.tar.gz
    
  6. Install Hadoop.

    # cd /usr/local
    # wget http://.../hadoop-X.X.X.tar.gz
    # tar xzf hadoop-X.X.X.tar.gz
    # rm hadoop-X.X.X.tar.gz
    
  7. Configure Hadoop (described below).

  8. Edit shell config files to, e.g., add executables to your PATH, and perform other configurations that will make this system easy for you to use. For example, you may wish to add a non-root user account to run Hadoop's daemons.

  9. Start Hadoop, and test your configuration minimally.

    # cd /usr/local/hadoop-X.X.X
    # bin/start-all.sh
    # bin/hadoop jar hadoop-0.7.2-examples.jar pi 10 10000000
    # bin/hadoop dfs -ls
    # bin/stop-all.sh
    
  10. Create a new image, using Amazon's instructions (bundle, upload & register).

Configuring Hadoop

Hadoop is configured with a single master node and many slave nodes. To facilliate re-deployment without re-configuration, one may register a name in DNS for the master host, then reset the address for this name each time the cluster is re-deployed. Services such as DynDNS make this fairly easy. In the following, we refer to the master as master.mydomain.com. Please replace this with your actual master node's name.

In EC2, the local data volume is mounted as /mnt.

hadoop-env.sh

Specify the JVM location, the log directory and that rsync should be used to update slaves from the master.

# Set Hadoop-specific environment variables here.

# The java implementation to use.  Required.
export JAVA_HOME=/usr/local/jdk1.5.0_09

# Where log files are stored.  $HADOOP_HOME/logs by default.
export HADOOP_LOG_DIR=/mnt/hadoop/logs

# host:path where hadoop code should be rsync'd from.  Unset by default.
export HADOOP_MASTER=master.mydomain.com:/usr/local/hadoop-X.X.X

# Seconds to sleep between slave commands.  Unset by default.  This
# can be useful in large clusters, where, e.g., slave rsyncs can
# otherwise arrive faster than the master can service them.
export HADOOP_SLAVE_SLEEP=1

You must also create the log directory.

% mkdir -p /mnt/hadoop/logs

hadoop-site.xml

All of Hadoop's local data is stored relative to hadoop.tmp.dir, so we only need specify this, plus the name of the master node for DFS (the NameNode) and MapReduce (the JobTracker).

<configuration>

<property>
  <name>hadoop.tmp.dir</name>
  <value>/mnt/hadoop</value>
</property>

<property>
  <name>fs.default.name</name>
  <value>master.mydomain.com:50001</value>
</property>

<property>
  <name>mapred.job.tracker</name>
  <value>master.mydomain.com:50002</value>
</property>

</configuration>

mapred-default.xml

This should vary with the size of your cluster. Typically mapred.map.tasks should be 10x the number of instances, and mapred.reduce.tasks should be 3x the number of instances. The following is thus configured for a 19-instance cluster.

<configuration>

<property>
  <name>mapred.map.tasks</name>
  <value>190</value>
</property>

<property>
  <name>mapred.reduce.tasks</name>
  <value>57</value>
</property>

</configuration>

Security

To access your cluster, you must enable access from at least port 22, for ssh. Generally it is also useful to open a few other ports, to view job progress. Port 50030 is used for the JobTracker's web interface, permitting one to view job status, and port 50060 is used by the TaskTracker's web interface, for more detailed debugging.

% ec2-add-group my-group
% ec2-authorize my-group -p 22
% ec2-authorize my-group -p 50030
% ec2-authorize my-group -p 50060

Instances within the cluster should have unfettered access to one another. This is enabled by the following, replacing the XXXXXXXXXXX with your Amazon web services user id.

% ec2-authorize my-group -o my-group -u XXXXXXXXXXX

Launching your cluster

Start by allocating instances of your image. Use ec2-describe-images to find the your image id, notated as ami-XXXXXXXX below.

To run a 20-node cluster:

% ec2-describe-images
% ec2-run-instances ami-XXXXXXX -k gsg-keypair -g my-group -n 20

Wait a few minutes for the instances to launch.

% ec2-describe-instances

Once instances are launched, register the first host listed in DNS with your master host name.

Create a slaves file containing the rest of the instances and copy it to the master.

% ec2-describe-instances | grep INSTANCE | cut -f 4 | tail +2 > slaves
% scp slaves master.mydomain.com:/usr/local/hadoop-X.X.X/conf/slaves

Format the new cluster's filesystem.

% ssh master.mydomain.com
# /usr/local/hadoop-X.X.X/bin/hadoop namenode -format

Start the cluster.

# /usr/local/hadoop-X.X.X/bin/start-all.sh

Visit your cluster's web ui at http://master.mydomain.com:50030/.

Shutting down your cluster

% ec2-terminate-instances `ec2-describe-instances | grep INSTANCE | cut -f 2`

Future Work

Ideally Hadoop could directly access job data from Amazon S3 (Simple Storage Service). Initial input could be read from S3 when a cluster is launched, and the final output could be written back to S3 before the cluster is decomissioned. Intermediate, temporary data, only needed between MapReduce passes, would be more efficiently stored in Hadoop's DFS. This would require an implementation of a Hadoop FileSystem for S3. There are two issues in Hadoop's bug database related to this:

Please vote for these issues in Jira if you feel this would help your project. (Anyone can create themselves a Jira account in order to vote on issues, etc.)

last edited 2006-10-28 03:35:10 by DougCutting
 
 

GettingStartedWithHadoop

Downloading and installing Hadoop

Hadoop can be downloaded from one of the Apache download mirrors. You may also download a nightly build or check out the code from subversion and build it with Ant. Select a directory to install Hadoop under (let's say /foo/bar/hadoop-install) and untar the tarball in that directory. A directory corresponding to the version of Hadoop downloaded will be created under the /foo/bar/hadoop-install directory. For instance, if version 0.6.0 of Hadoop was downloaded untarring as described above will create the directory /foo/bar/hadoop-install/hadoop-0.6.0. The examples in this document assume the existence of an environment variable $HADOOP_INSTALL that represents the path to all versions of Hadoop installed. In the above instance HADOOP_INSTALL=/foo/bar/hadoop-install. They further assume the existence of a symlink named hadoop in $HADOOP_INSTALL that points to the version of Hadoop being used. For instance, if verision 0.6.0 is being used then $HADOOP_INSTALL/hadoop -> hadoop-0.6.0. All tools used to run Hadoop will be present in the directory $HADOOP_INSTALL/hadoop/bin. All configuration files for Hadoop will be present in the directory $HADOOP_INSTALL/hadoop/conf.

Startup scripts

The $HADOOP_INSTALL/hadoop/bin directory contains some scripts used to launch Hadoop DFS and Hadoop Map/Reduce daemons. These are:

  • start-all.sh - Starts all Hadoop daemons, the namenode, datanodes, the jobtracker and tasktrackers.

  • stop-all.sh - Stops all Hadoop daemons.

  • start-mapred.sh - Starts the Hadoop Map/Reduce daemons, the jobtracker and tasktrackers.

  • stop-mapred.sh - Stops the Hadoop Map/Reduce daemons.

  • start-dfs.sh - Starts the Hadoop DFS daemons, the namenode and datanodes.

  • stop-dfs.sh - Stops the Hadoop DFS daemons.

Configuration files

The $HADOOP_INSTALL/hadoop/conf directory contains some configuration files for Hadoop. These are:

  • hadoop-env.sh - This file contains some environment variable settings used by Hadoop. You can use these to affect some aspects of Hadoop daemon behavior, such as where log files are stored, the maximum amount of heap used etc. The only variable you should need to change in this file is JAVA_HOME, which specifies the path to the Java 1.5.x installation used by Hadoop.

  • slaves - This file lists the hosts, one per line, where the Hadoop slave daemons (datanodes and tasktrackers) will run. By default this contains the single entry localhost

  • hadoop-default.xml - This file contains generic default settings for Hadoop daemons and Map/Reduce jobs. Do not modify this file.

  • mapred-default.xml - This file contains site specific settings for the Hadoop Map/Reduce daemons and jobs. The file is empty by default. Putting configuration properties in this file will override Map/Reduce settings in the hadoop-default.xml file. Use this file to tailor the behavior of Map/Reduce on your site.

  • hadoop-site.xml - This file contains site specific settings for all Hadoop daemons and Map/Reduce jobs. This file is empty by default. Settings in this file override those in hadoop-default.xml and mapred-default.xml. This file should contain settings that must be respected by all servers and clients in a Hadoop installation, for instance, the location of the namenode and the jobtracker.

More details on configuration can be found on the HowToConfigure page.

Setting up Hadoop on a single node

This section describes how to get started by setting up a Hadoop cluster on a single node. The setup described here is an HDFS instance with a namenode and a single datanode and a Map/Reduce cluster with a jobtracker and a single tasktracker. The configuration procedures described in Basic Configuration are just as applicable for larger clusters.

Basic Configuration

Take a pass at putting together basic configuration settings for your cluster. Some of the settings that follow are required, others are recommended for more straightforward and predictable operation.

  • Hadoop Environment Settings - Ensure that JAVA_HOME is set in hadoop-env.sh and points to the Java installation you intend to use. You can set other environment variables in hadoop-env.sh to suit your requirments. Some of the default settings refer to the variable HADOOP_HOME. The value of HADOOP_HOME is automatically inferred from the location of the startup scripts. HADOOP_HOME is the parent directory of the bin directory that holds the Hadoop scripts. In this instance it is $HADOOP_INSTALL/hadoop.

  • Jobtracker and Namenode settings - Figure out where to run your namenode and jobtracker. Set the variable fs.default.name to the Namenodes intended host:port. Set the variable mapred.job.tracker to the jobtrackers intended host:port. These settings should be in hadoop-site.xml. You may also want to set one or more of the following ports (also in hadoop-site.xml):

    • dfs.datanode.port

    • dfs.info.port

    • mapred.job.tracker.info.port

    • mapred.task.tracker.ouput.port

    • mapred.task.tracker.report.port

  • Data Path Settings - Figure out where your data goes. This includes settings for where the namenode stores the namespace checkpoint and the edits log, where the datanodes store filesystem blocks, storage locations for Map/Reduce intermediate output and temporary storage for the HDFS client. The default values for these paths point to various locations in /tmp. While this might be ok for a single node installation, for larger clusters storing data in /tmp is not an option. These settings must also be in hadoop-site.xml. It is important for these settings to be present in hadoop-site.xml because they can otherwise be overridden by client configuration settings in Map/Reduce jobs. Set the following variables to appropriate values:

    • dfs.name.dir

    • dfs.data.dir

    • dfs.client.buffer.dir

    • mapred.local.dir

Formatting the Namenode

The first step to starting up your Hadoop installation is formatting the Hadoop filesystem, which is implemented on top of the local filesystems of your cluster. You need to do this the first time you set up a Hadoop installation. Do not format a running Hadoop filesystem, this will cause all your data to be erased. To format the filesystem (which simply initializes the directory specified by the dfs.name.dir variable), run the command:
% $HADOOP_INSTALL/hadoop/bin/hadoop namenode -format

Starting a Single node cluster

Run the command:
% $HADOOP_INSTALL/hadoop/bin/start-all.sh
This will startup a Namenode, Datanode, Jobtracker and a Tasktracker on your machine.

Stopping a Single node cluster

Run the command
% $HADOOP_INSTALL/hadoop/bin/stop-all.sh
to stop all the daemons running on your machine.

Separating Configuration from Installation

In the example described above, the configuration files used by the Hadoop cluster all lie in the Hadoop installation. This can become cumbersome when upgrading to a new release since all custom config has to be re-created in the new installation. It is possible to separate the config from the install. To do so, select a directory to house Hadoop configuration (let's say /foo/bar/hadoop-config. Copy the hadoop-site.xml, slaves and hadoop-env.sh files to this directory. You can either set the HADOOP_CONF_DIR environment variable to refer to this directory or pass it directly to the Hadoop scripts with the --config option. In this case, the cluster start and stop commands specified in the above two sub-sections become
% $HADOOP_INSTALL/hadoop/bin/start-all.sh --config /foo/bar/hadoop-config and
% $HADOOP_INSTALL/hadoop/bin/stop-all.sh --config /foo/bar/hadoop-config.
Only the absolute path to the config directory should be passed to the scripts.

Starting up a larger cluster

  • Ensure that the Hadoop package is accessible from the same path on all nodes that are to be included in the cluster. If you have separated configuration from the install then ensure that the config directory is also accessible the same way.

  • Populate the slaves file with the nodes to be included in the cluster. One node per line.

  • Follow the steps in the Basic Configuration section above.

  • Format the Namenode

  • Run the command % $HADOOP_INSTALL/hadoop/bin/start-dfs.sh on the node you want the Namenode to run on. This will bring up HDFS with the Namenode running on the machine you ran the command on and Datanodes on the machines listed in the slaves file mentioned above.

  • Run the command % $HADOOP_INSTALL/hadoop/bin/start-mapred.sh on the machine you plan to run the Jobtracker on. This will bring up the Map/Reduce cluster with Jobtracker running on the machine you ran the command on and Tasktrackers running on machines listed in the slaves file.

  • The above two commands can also be executed with a --config option.

Stopping the cluster

  • The cluster can be stopped by running % $HADOOP_INSTALL/hadoop/bin/stop-mapred.sh and then % $HADOOP_INSTALL/hadoop/bin/stop-dfs.sh on your Jobtracker and Namenode respectively. These commands also accept the --config option.

last edited 2006-10-12 05:38:36 by NigelDaley

life-coding » Blog Archive » How to Make Pound And SSL Play Nice With OS X
lifecoding.com/blog/?p=29

How to Make Pound And SSL Play Nice With OS X

Though building sites with ssl is cool and gives your users a sense of security, configuring a webserver with ssl can be a royal pain. Thankfully, there's pound. Pound is a 「is a reverse proxy, load balancer and HTTPS front-end for Web server(s).」

Pound is dead simple to setup and configure. Unfortunately, the darwin port for pound is old and does not work. So this guide will help you build pound on your own. Besides, everyone feels cooler after compiling that hot fire. Click on for the steps.

  1. Open terminal.
  2. Install zlib
  3. mkdir ~/temp
    curl -O http://www.zlib.net/zlib-1.2.3.tar.gz
    tar xzvf zlib-1.2.3.tar.gz
    cd zlib-1.2.3
    ./configure --prefix=/usr/local --shared
    make
    sudo make install
    
  4. Install openssl
  5. curl -O http://www.openssl.org/source/openssl-0.9.8b.tar.gz
    tar xzvf openssl-0.9.8b.tar.gz
    cd openssl-0.9.8b
    ./config -L/usr/local/lib --openssldir=/usr/local/etc/openssl \
        zlib no-asm no-krb5 shared
    make
    sudo make install
    
  6. Install Pound 2.0 or greater
  7. curl -O http://www.apsis.ch/pound/Pound-2.0.tgz
    tar xzvf Pound-2.0.tgz
    cd Pound-2.0
    sed "s/-o bin -g bin //g" < Makefile.in > Makefile.in.new
    mv Makefile.in.new Makefile.in  # Hit y to override any restrictions
    ./configure –with-ssl=/usr/local/etc/openssl/ –prefix=/usr/local
    make
    sudo make install
    
  8. Generate an ssl certificate.
  9. cd /usr/local/etc
    

    You can put the next line in ~/.bash_profile if you want openssl available everytime you open the terminal.

    export PATH="/usr/local/etc/openssl/bin:$PATH"
    sudo openssl/bin/openssl req -x509 -newkey rsa:1024 -keyout our_cert.pem \
        -out our_cert.pem -days 365 -nodes
    

    Fill out the required information to generate the ssl certificate.

  10. Create a pound.cfg file.
  11. cd /usr/local/etc # You should already be here
    

    This next piece uses a trick called a heredoc. Instead of copying and pasting the below command into terminal, you could also just copy and paste the text into pound.cfg in /usr/local/etc/pound.cfg

    cat > ~/tmp_file <<EOF
    ListenHTTP
      Address 0.0.0.0
      Port    80
      Service
        BackEnd
          Address 127.0.0.1
          Port    3000
        End
      End
    End
    
    ListenHTTPS
      Address 0.0.0.0
      Port    443
      Cert    "/usr/local/etc/our_cert.pem"
      # pass along https hint
      AddHeader "X-Forwarded-Proto: https"
      HeadRemove "X-Forwarded-Proto"
      Service
        BackEnd
          Address 127.0.0.1
          Port    3000
        End
      End
    End
    EOF
    sudo mv ~/tmp_file ./pound.cfg
    
  12. Make sure that pound works.
  13. pound -v -c
    

    You should see Config file /usr/local/etc/pound.cfg is OK.
    If not, make sure that you copied your config file correctly
    using 『cat pound.cfg' to view what is in the file.

  14. Turn off apache (assuming it's running).
  15. sudo apachectl stop
  16. Turn on pound.
  17. sudo pound -v
  18. Start mongrel/webrick/lighty.
  19. cd ~/work/my_killer_app
    mongrel_rails start # Or ruby script/server (if you're still on lighty/webrick)
    
  20. Marvel at your wonderous creation by pointing your browser to http://localhost/ or https://localhost/

TODO (feel free to do these and post how to do them in the comments):

  • Make pound start when the computer loads.
  • Use darwin ports to install zlib and openssl, but not pound.
嘗試用capistrano部署ruby on rails應用 » 石鍋拌飯 | 互聯網 Mac & 軟件開發
www.robinlu.com/blog/archives/117

部署網絡應用服務是很麻煩的事情,安裝程序,升級數據庫schema,切換版本,重啟服務。步驟越多,人工參與越多,越容易出問題。
Capistrano是ruby on rails提供的部署方案,原名叫SwitchTower,多好記的名字,結果和人重了,換成現在這個,估計是為了賭氣才起成這樣,反正我再也不能拼對了。它集成了很多部署程序必須的步驟,借助ssh、版本管理系統(支持svn、cvs等等好幾種)和rails的migration,只要做好配置,就可以在很大程度上實現部署自動化。
Capistrano的相關文檔可以看這裡。最簡單的使用方法是:

  1. 安裝

    gem install capistrano

  2. 加入capistrano支持

    cap --apply-to /path/to/my/app MyApplicationName

  3. 修改配置文件

    修改config/deploy.rb文件。

  4. 初始化服務器上的運行環境

    rake remote:exec ACTION=setup
    這一步會連上你的服務器,創建一些目錄。

  5. 部署

    rake deploy
    連接服務器,完成部署。

說起來好像挺簡單,麻煩的地方主要在於修改config/deploy.rb配置文件。一般來說主要配置的參數包括:

  • application:應用名。
  • repository:版本管理系統的鏈接URL。
  • web:web服務器名列表。
  • app:應用服務器列表。
  • db:數據庫服務器列表。
  • user:ssh用戶名。
  • deploy_to:應用部署路徑。

如果你的svn鏈接需要用戶名和密碼,可以通過svn_username和svn_password配置,文檔中沒寫,算我免費友情提示。
除了setup和deploy,缺省支持的命令主要有:

  • disable_web:生成maintenance.html,你的系統需要能夠自檢測這個文件。
  • enable_web:刪除maintenance.html。
  • update_code:和版本管理器做代碼同步。
  • rollback_code:如果部署完發現有問題,可以用這個命令換回上一個。
  • restart:重啟,其實就是調用了reaper。
  • migrate:在服務器端運行rake RAILS_ENV=#{rails_env} migrate。
  • deploy:其實就是update_code+symlink+restart。
  • deploy_with_migrations:update_code+migrate+symlink+restart。
  • rollback: rollback_code + restart。

還有一些其它的命令,具體可以查看gem capistrano安裝目錄下lib/recipes中的standard.rb。另外,可以自己在deploy.rb中定義新的命令。
目前的capistrano(版本1.1.0)還有一些局限,比如你所有服務器都必須使用同樣的賬號,這個讓我在dreamhost上部署碰到了一點麻煩。總的來說還是比較方便的。

EasyTime - Ruby学习笔记 | Agile Web Development with Rails 翻译(七十九)
my4java.itpub.net/post/9983/102465

Agile Web Development with Rails 翻译(七十九)

附录B 配置参数

就像在180页解释的,各种Rails组成部分可以通过设置选项全局environment.rb文件,或config/environment目录内的一个指定的环境文件来配置。


B.1 Active Record Configuration

1ActiveRecord::Base.logger =logger 接受一个logger对象。内部使用它记录活动的数据库。它也对希望日志一个动作的应用程序有效。

2ActiveRecord::Base.primary_key_prefix_type =option 如果optionnil,如果:table_name,表名称被预先计划的话,则每个表缺省主键为 id。设置选项的值为:table_name_with_underscore,则在表的名字和id之间添加一个下划线。

3ActiveRecord::Base.table_name_prefix ="prefix" 生成表名字时使用预先计划的给定字符串。例如,如果模型的名字是User,并且前缀字符串是”myapp-“,则Rails将查找表myapp-users。如果你必须在不同应用程序间共享一个数据库,或者你必须在同一数据库内完成开发和测试,则这就很有用处。

4ActiveRecord::Base.table_name_suffix ="suffix" 在生成表名称时附加给定的字符串。

5ActiveRecord::Base.pluralize_table_names = true | false 如果为false,则在创建相应的表名称时,类名称不必是复数。

6ActiveRecord::Base.colorize_logging = true | false 缺省地,活动记录使用ANSI控制序列日志消息,但使用的终端应用程序支持这些序列时,它彩色化某些行。设置选项为false可移除这种彩色化。

7ActiveRecord::Base.default_timezone = :local | :utc Set to :utc to have dates and times loaded from and saved to the database treated as UTC.

8ActiveRecord::Locking.lock_optimistically = true | false 如果为false,则乐观锁被关闭。

9ActiveRecord::Timestamp.record_timestamps = true | false 设置为false则关闭列created_atcreat_onupdated_at,和updated_on的自动更新。在267页讨论。

10ActiveRecord::Errors.default_error_messages =hash 一个标准的确认失败消息哈希表。你可以用你自己的消息替换它,或许是出于国际化目的。缺省设置是

ActiveRecord::Errors.default_error_messages = {

:inclusion => "is not included in the list",

:exclusion => "is reserved",

:invalid => "is invalid",

:confirmation => "doesn't match confirmation",

:accepted => "must be accepted",

:empty => "can't be empty",

:too_long => "is too long (max is %d characters)",

:too_short => "is too short (min is %d characters)",

:wrong_length => "is the wrong length (should be %d characters)",

:taken => "has already been taken",

:not_a_number => "is not a number",

}

B.2 Action Pack Configuration

1ActionController::Base.asset_host =url Sets the host and/or path of stylesheet and image assets linked using the asset helper tags.

ActionController::Base.asset_host = "http://media.my.url"

2ActionController::Base.view_controller_internals = true | false By default, templates get access to the controller collections request, response, session, and template. Setting this option to false removes this access.

3ActionController::Base.consider_all_requests_local = true | false 在产品模式内设置为false,会阻止用户查看堆栈backtraces.。它的深入讨论在450页的22.3节。

4ActionController::Base.debug_routes = true | false 如果为true,则在路由组件解析URL失败时给出详细信息。产品模式下关闭它。

5ActionController::Base.logger =logger 设置由这个控制器使用的loggerLogger对象对你的应用程序代码也有效的。

6ActionController::Base.template_root =dir 在这个目录下查找模板文件。缺省是 app/views

7ActionController::Base.template_class =class 缺省是ActionView::Base。你或许不应该修改它。

8ActionController::Base.ignore_missing_templates = false | true 如果为true,则在模板没有找到时不会引发一个错误。

9ActionController::Base.perform_caching = true | false 设置为false会关闭所有缓存。

10ActionController::Base.page_cache_directory =dir 指出缓存文件存储在哪儿。必须是你的web 服务的文档根目录。

11ActionController::Base.page_cache_extension =string 覆写用于被缓存文件的缺省 .html扩展名。

12ActionController::Base.fragment_cache_store =caching_class 决定用于存储被缓存段的机制。段缓存的存储在369页讨论。

13ActionView::Base.cache_template_loading = false | true 转向提交缓存的模板,这会提高性能。但是,在你修改了磁盘上模板时,你将需要重启服务器。

14ActionView::Base.field_error_proc =proc This proc is called to wrap a form field that fails validation. The default value is

Proc.new do |html_tag, instance|

%{<div class="fieldWithErrors">#{html_tag}</div>}

end

B.3 Action Mailer Configuration

这些设置描述399页的19.1中。

ActionMailer::Base.template_root = directory

ActionMailer::Base.logger = logger object

ActionMailer::Base.server_settings = hash

ActionMailer::Base.raise_delivery_errors = true | false

ActionMailer::Base.delivery_method = :smtp | :sendmail | :test

ActionMailer::Base.perform_deliveries = true | false

ActionMailer::Base.default_charset = "string"

B.4 Test Case Configuration

下面选项可以设置成全局的,但通常多被设置在特定测试案例内部。

# Global setting

Test::Unit::TestCase.use_transactional_fixtures = true

# Local setting

class WibbleTest < Test::Unit::TestCase

self.use_transactional_fixtures = true

# ...

1use_transactional_fixtures = true | false 如果为true,对数据库的修改将在每个测试结束时回滚。在17012.7节讨论。

2use_instantiated_fixtures = true | false | :no_instances 设置选项为false,会关闭自动将fixture数据加载到一个实例变量中。设置它为 :no_instances 可创建实例变量但不能使用它。

3pre_loaded_fixtures = false | true 如果为true,则测试案例假设在测试运行之前fixture数据已被加载到数据库。使用事务的fixture会加速测试的运行。

Serving Rails with lighttpd, pound and mongrel
iamrice.org/articles/2006/07/08/serving-rails-with...

Serving Rails with lighttpd, pound and mongrel

Posted by Damien Tanner on 07/09/2006

Recently David pointed out Pound could be used as as the proxy for a mongrel cluster, until lighttpd's mod_proxy is fixed of course. So this weekend I took the plunge, and migrated our server from SCGI to Pound and Mongrel.

There have been several great articles describing deployment with mongrel:

You may have noticed that there has been a bit of a switch to Apache recently. But using lighttpd and Pound we can still achieve the same functionality. If you're only hosting one application on your box, simply running Pound and mongrel will suffice. However if you're planning to host several apps (and domains) you'll want to use lighttpd's conditionals to send requests to the correct Pound ports.

Here is how I've setup the Mac Mini.

Get the right tools

First thing, get our tools installed.

Lighttpd

Simplest method is sudo port install lighttpd

Mongrel

sudo gem install daemons gem_plugin mongrel mongrel_cluster sendfile --include-dependencies

Pound

First time round I thought I'd be cool and install Pound 2.0.3. Don't do this, it seems a bit broken. I followed this: How to Make Pound And SSL Play Nice With OS X (watch out for a few typos if you're cuttin'n'pasting commands)

Configure it all

Hopefully you should have lighttpd, mongrel and pound installed and working now.

With this setup the idea is each application (and/or domain) you're running has several ports reserved for it. One for pound, and several others for mongrel processes. Requests for you server will be handled by lighttpd, which will check the requested domain against the regexp in your lighttpd conf. If it finds a match, it will proxy the request to pound, on the specified port. Pound will then load balance between the mongrel processes you have running.

Lighttpd config

I find it nice to have all the conditionals in a separate file (pound_sites.conf) in this case. So your main lighttpd.conf can be more readable.

server.port                = 80
server.bind = "0.0.0.0"

server.pid-file = "/tmp/lighttpd.pid"

server.modules = ( "mod_auth", "mod_rewrite", "mod_redirect", "mod_access", "mod_accesslog", "mod_compress", "mod_fastcgi", "mod_proxy" )

server.document-root = "/Users/iamrice/www_root"
server.indexfiles = ( "index.html", "index.php" )
static-file.exclude-extensions = ( ".php" )
accesslog.filename = "/var/log/lighttpd_access.log"
server.errorlog = "/var/log/lighttpd_error.log"
url.access-deny = ( "~", ".inc" )

include "pound_sites.conf"

# mimetype mapping
mimetype.assign = (
".rpm" => "application/x-rpm",
".pdf" => "application/pdf",
".sig" => "application/pgp-signature",
".spl" => "application/futuresplash",
".class" => "application/octet-stream",
".ps" => "application/postscript",
".torrent" => "application/x-bittorrent",
".dvi" => "application/x-dvi",
".gz" => "application/x-gzip",
".pac" => "application/x-ns-proxy-autoconfig",
".swf" => "application/x-shockwave-flash",
".tar.gz" => "application/x-tgz",
".tgz" => "application/x-tgz",
".tar" => "application/x-tar",
".zip" => "application/zip",
".mp3" => "audio/mpeg",
".m3u" => "audio/x-mpegurl",
".wma" => "audio/x-ms-wma",
".wax" => "audio/x-ms-wax",
".ogg" => "audio/x-wav",
".wav" => "audio/x-wav",
".gif" => "image/gif",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".png" => "image/png",
".xbm" => "image/x-xbitmap",
".xpm" => "image/x-xpixmap",
".xwd" => "image/x-xwindowdump",
".css" => "text/css",
".html" => "text/html",
".htm" => "text/html",
".js" => "text/javascript",
".asc" => "text/plain",
".c" => "text/plain",
".conf" => "text/plain",
".text" => "text/plain",
".txt" => "text/plain",
".dtd" => "text/xml",
".xml" => "text/xml",
".xul" => "application/vnd.mozilla.xul+xml",
".mpeg" => "video/mpeg",
".mpg" => "video/mpeg",
".mov" => "video/quicktime",
".qt" => "video/quicktime",
".avi" => "video/x-msvideo",
".asf" => "video/x-ms-asf",
".asx" => "video/x-ms-asf",
".wmv" => "video/x-ms-wmv",
".bz2" => "application/x-bzip",
".tbz" => "application/x-bzip-compressed-tar",
".tar.bz2" => "application/x-bzip-compressed-tar"
)

Then for each of your sites you'll want something similar to this in your pound_sites.conf file.

$HTTP["host"] =~ "^(www.)?iamrice\.org" {
server.document-root = "/Users/iamrice/www/public"
accesslog.filename = "/Users/iamrice/logs/lighttpd.access.log"
server.errorlog = "/Users/iamrice/logs/lighttpd.error.log"
proxy.balance = "fair"
proxy.server = ( "/" => ( ( "host" => "127.0.0.1", "port" => 6000 ) ) )
}

The port you set above (6000) will need to be used as the listen port in your Pound config.

Pound config

In your pound.cfg you'll also need to add a few lines for each site.

ListenHTTP
Address 0.0.0.0
Port 6000
Service
BackEnd
Address 127.0.0.1
Port 6001
End
BackEnd
Address 127.0.0.1
Port 6002
End
BackEnd
Address 127.0.0.1
Port 6003
End
Session
Type BASIC
TTL 300
End
End
End

The ports 6001, 6002 and 6003 will be your mongrel processes.

Mongrel config

Last stage is your mongrel config. Mongrel cluster comes with a configure option, so cd into your rails project and run the following.

mongrel_rails cluster::configure -e development -p 6001 -N 3 -c /Users/iamrice/www -a 127.0.0.1

And we're go!

Start up lighttpd, pound and mongrel from your rails app (mongrel_rails cluster::start) and you should have some mongrel serving goodness :) If you're getting any errors, a good first step is to hit Pound and Mongrel using their port. You'll need to do this from the local machine though.

ReservedWords in Ruby on Rails
wiki.rubyonrails.com/rails/pages/ReservedWords

Names You Can't Use (aka reserved words, keywords) from Ruby and RubyOnRails

  • ADDITIONAL_LOAD_PATHS
  • ARGF
  • ARGV
  • ActionController
  • ActionView
  • ActiveRecord
  • ArgumentError
  • Array
  • BasicSocket
  • Benchmark
  • Bignum
  • Binding
  • CGI
  • CGIMethods
  • CROSS_COMPILING
  • Class
  • ClassInheritableAttributes
  • Comparable
  • ConditionVariable
  • Config
  • Continuation
  • DRb
  • DRbIdConv
  • DRbObject
  • DRbUndumped
  • Data
  • Date
  • DateTime
  • Delegater
  • Delegator
  • Digest
  • Dir
  • ENV
  • EOFError
  • ERB
  • Enumerable
  • Errno
  • Exception
  • FALSE
  • FalseClass
  • Fcntl
  • File
  • FileList
  • FileTask
  • FileTest
  • FileUtils
  • Fixnum
  • Float
  • FloatDomainError
  • GC
  • Gem
  • GetoptLong
  • Hash
  • IO
  • IOError
  • IPSocket
  • IPsocket
  • IndexError
  • Inflector
  • Integer
  • Interrupt
  • Kernel
  • LN_SUPPORTED
  • LoadError
  • LocalJumpError
  • Logger
  • Marshal
  • MatchData
  • MatchingData
  • Math
  • Method
  • Module
  • Mutex
  • Mysql
  • MysqlError
  • MysqlField
  • MysqlRes
  • NIL
  • NameError
  • NilClass
  • NoMemoryError
  • NoMethodError
  • NoWrite
  • NotImplementedError
  • Numeric
  • OPT_TABLE
  • Object
  • ObjectSpace
  • Observable
  • PGError
  • PGconn
  • PGlarge
  • PGresult
  • PLATFORM
  • PStore
  • ParseDate
  • Precision
  • Proc
  • Process
  • Queue
  • RAKEVERSION
  • RELEASE_DATE
  • RUBY
  • RUBY_PLATFORM
  • RUBY_RELEASE_DATE
  • RUBY_VERSION
  • Rake
  • RakeApp
  • RakeFileUtils
  • Range
  • RangeError
  • Rational
  • Regexp
  • RegexpError
  • Request
  • RuntimeError
  • STDERR
  • STDIN
  • STDOUT
  • ScanError
  • ScriptError
  • SecurityError
  • Signal
  • SignalException
  • SimpleDelegater
  • SimpleDelegator
  • Singleton
  • SizedQueue
  • Socket
  • SocketError
  • StandardError
  • String
  • StringScanner
  • Struct
  • Symbol
  • SyntaxError
  • SystemCallError
  • SystemExit
  • SystemStackError
  • TCPServer
  • TCPSocket
  • TCPserver
  • TCPsocket
  • TOPLEVEL_BINDING
  • TRUE
  • Task
  • Text
  • Thread
  • ThreadError
  • ThreadGroup
  • Time
  • Transaction
  • TrueClass
  • TypeError
  • UDPSocket
  • UDPsocket
  • UNIXServer
  • UNIXSocket
  • UNIXserver
  • UNIXsocket
  • UnboundMethod
  • Url
  • VERSION
  • Verbose
  • YAML
  • ZeroDivisionError

Other names that have been reported to cause trouble:

  • action
  • attributes – if you have a has_many called attributes, you can't access to your object attributes anymore; only the associated objects
  • application2
  • @base_path – setting this variable name in a controller method seems to break the ablity to render a partial in the view. The view will render with no content and no errors will be generated .
  • connection – there seems to be a connection class already
  • dispatcher
  • display1
  • format
  • key
  • load – When making an Ajax call to an action named load, the action's code will be skipped (or otherwise rendered useless). This is made apparent by: a) @variables are not available in the view, b) calling render :layout => false still yields the layout.
  • new, override to news if you want a news table
  • notify – not a valid column name
  • open – not a valid column name
  • quote 『quote' cannot be used as a column name
  • request
  • records – a table named records seemed to cause duplicate entries to be found by find
  • responses – scaffold borks with 「undefined method 『body=』 」
  • send
  • session (session_controller or SessionController will not work)
  • system – a table column named system causes problems when trying to generate scaffold
  • template – a table named templates causes an error when you try to invoke the create method of the default controller
  • test (however those work with ruby test/unit/axis_test.rb and rake test_units)
  • timeout – an ActiveRecord attribute named timeout will clash with the global function 「timeout」 defined in Ruby's timeout.rb
  • to_s—naming a model instance method to_s resulted in 『File not found' for any view an object of this class (should have) appeared in (no matter which method called) and WebRick had to be restarted. I couldn't drag the very cause into light, but in the traces 『to_s' gave me a hint. After renaming everything worked well again.
  • type—or any of the other MagicFieldNames
  • URI

singular names finishing in 「s」: Axis → Axes, Access → Accesses, will break the pluralization in rake: Axi, Acces

Names You Can't Use from SQL

The list of reserved words is dependent on the database you use, for portability reasons it would be wise to not chose a field name listed in any of these tables:

If you aren't sure, you can check the word against the SQL Reserved Words Checker

Also note that numerous field names have special properties. See the full list of MagicFieldNames.

Typical Errors

The errors that occur when you use a reserved word tend to be very confusing. Things that you think are happening in your code, are actually happening somewhere in the framework. Sometimes you can look at the stack trace and see that its not going through your class, but through some framework class. If you have an error that makes no sense at all, I would check to make sure you don't have a name that conflicts with the above list.

In one instance I got a mysql error when I tried to save a model that belongs_to :quote. The belongs_to made a method that overrode the quote method in activerecord::base, which caused Quote objects to be returned where activerecord was expecting a quoted string!

Requests

  1. Please explain the problems with 『display'. I am using a controller called image, and if I define 『display' as an action method name, I get 「Unknown action: No action responded to display.」

1 For action, the only problem comes into play when you have a model named Action and feed that to the form helpers. The problem is that the array action[field_names] in the params[] method will be overwritten by Rails. Instead of the form data, you will only get the Controller method, as in 『create' or 『edit'.

By feeding a different word, such as action_data, to form helpers in place of the actual model name you can easily work around this problem. Then, simply access it as params[:action_data] instead of params[:action]. If you are experiencing this problem, you will probably receive the error undefined method `stringify_keys!' for "create":String.

2 Seen when trying to map legacy APPLICATION table to an Application ActiveRecord. Causes confusion in rails dynamic require between application.rb (app defaults) and application.rb (ActiveRecord model).


I've been having problems when I try to generate a scaffold for Applications. I'm trying to develop a web app that will manage software Applications; thus, I created an applications table in my database and generated a scaffold off of it. It doesn't seem to work! Any ideas or is this just not possible?

For some reason I've been having problems with Base and pagination.

I found this page had been spammed, and so did a rollback to a good version.

Remember Me

The guts of this code was borrowed from
http://www.onrails.org/articles/2006/02/18/auto-login

Migration:


  def self.up
      add_column "users", "remember_token", :string, :limit => 40
  end

User model (user.rb):



  def remember_me
      if self.remember_token.nil?
          update_attributes(:remember_token => Digest::SHA1.hexdigest("#{salt}--#{self.email}--#{Time.now}"))
      end
  end

  def forget_me
      update_attributes(:remember_token => nil)
  end

Application controller (application.rb):



  before_filter :login_from_cookie

    def login_from_cookie
        # what it:s saying is, if theres no session var but there is the token cookie then do this shit
        return unless @session[:user].nil? && cookies[:auth_token]
      self.current_user = User.find_by_remember_token(cookies[:auth_token])
    end

Account controller (account_controller.rb):



  def login
     return unless request.post?
     self.current_user = User.authenticate(params[:login], params[:password])
     if current_user
     # breakpoint()
       redirect_back_or_default(:controller => '/account', :action => 'index')
       flash[:notice] = "Logged in successfully" 
       if params[:rememberme] == "1" 
         self.current_user.remember_me
         cookies[:auth_token] = {:value => self.current_user.remember_token, :expires => 90.days.from_now}
       else
         # for security reason, if user forgot logout and relogin using other machine
         current_user.forget_me if current_user.remember_token
       end
     else
       flash[:notice] = "Invalid username or password" 
     end

  def logout
      self.current_user.forget_me if current_user
    self.current_user = nil
    cookies.delete :auth_token

    reset_session
    flash[:notice] = "You have been logged out." 
    redirect_back_or_default(:controller => '/account', :action => 'index')
  end

Question: instead of using a before_filter: to call login_from_cookie(), can we call it inside :login_required() instead? This way methods that do not require login will not have the overhead of calling login_from_cookie()?

Answer: Actually I think the ideal place to put this code would be in the logged_in? method – thereby only getting called on a page requiring login if not already logged in. Unfortunately my nuby ruby skills failed me when I tried to extend the elegant method of the generator.

PS. In the plugin version of 3 July, 2006 – Remember Me functionality is baked right in to the plugin so you don't need to integrate the above code. It is done with the before_filter method though, which I still think could be improved as mentioned in the Answer above…

HowtoChangeSessionOptions in Ruby on Rails
wiki.rubyonrails.com/rails/pages/HowtoChangeSessio...

HowtoChangeSessionOptions

Home Page | All Pages | Recently Revised | Feed

This page introduces how to change session options, both globally (for your entire application) and per controller/per action.

This page covers:

  • The Basics
  • Disabling Session
  • Sessions Storage Engine
  • Session File Storage Location
  • Session Duration
  • Session Domain

The Basics

What can be changed?

The session options that can be set are described with the process_cgi method in the ActionPack API documentation.

Owing to the fact that nearly all of the session controls in Rails derive from Ruby's CGI module, the CGI module Session documentation is a good source of information about how sessions work.

From where can it be changed?

The two places that session options can be set are:

  1. an environment configuration file
  2. a controller

Setting session options via an environment file

Don't forget to restart the server to reload any changes you've made changes to your environment files.

The Rails session options can be set in config/environment.rb (or even one of the files in config/environments/*.rb if you want options per environment) as such in early versions of Rails:

# Include in environment.rb's Rails::Initializer block:
config.action_controller.session :prefix => 'eliteapp.'

In later versions of Rails (e.g. 1.1.2 to 1.1.6 inclusive) many of the config entries no longer work as they used to. In the above case we need to use different code outside the Rails::Initializer block, as shown below:

# Include in environment.rb at end:
ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS[:session_key] = 'eliteapp_session_id'

This is a bit clumsy so in ActionPack 1.10.0, October 2005, a neater way (see the Changelog) to do exactly the same thing was added. The following code is directly equivalent to the second example above:

ActionController::Base.session_options[:session_key] = 'eliteapp_session_id'

The session_options method just returns ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS . Note that the value in the second and third example is not just a prefix – it is the full key name to be used. With the above code, you would see a cookie called 「eliteapp_session_id」 rather than the default 」_session_id」 in your Web browser's cookie list, if it provides one.

In Firefox 1.5, go to Tools -> Options, Privacy pane, Cookies tab, and click on the View Cookies button. The dialogue box that appears can be left open while you close the Options window and navigate your site, which is a good way to see new cookies appear and easily delete them for testing purposes.

It's a good idea to change the session cookie's key to prevent conflicts with other Ruby apps from the same server. Otherwise, they all try to use a cookie called 」_session_id」 and thus only one application will work properly at a time from any particular Web browser.

LarryK, Andrew Hodgkinson

If you want to set a session option differently for one environment, you can set specific session options differently for each environment. For example, on an older version of Rails we might do this:

# in config/environments/development.rb
config.action_controller.session :domain => 'mybox'

Again, this doesn't work in newer versions of Rails; where you used to be able to do this:

config.action_controller.session :key => 'value'

...you must now do this:

ActionController::Base.session_options[:key] = 'value'

AdrianD, Andrew Hodgkinson

Setting session options from a controller

Rails Session Management module exposes session options in controllers via a simple session call

class ApplicationController < ActionController::Base
    session :disabled => true
end

The session call allows for fine grain control over session options, even permitting per-action differences. See HowtoPerActionSessionOptions for more information.

Disabling Sessions

To disable session support in one controller, add session :disabled => true in that controller. To disable session support in the entire application, add it to ApplicationController.

class ApplicationController < ActionController::Base
    session :disabled => true
end

To re-enable it in a inheriting controller:

class SessionController < ApplicationController
    session :disabled => false
end

For more info about the session macro, see HowtoPerActionSessionOptions.

A shorthand way to disable sessions is to use session :off but unlike the above method, sessions cannot be enabled later on. Only use this form if you are sure you want to permanently disable sessions in the controller or application.

Rails 0.13.1 and Earlier

Add the following to your enviroment configuration.

ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS=false

Note: This solution only works for WEBrick and is not recommended for general use.

Changing sessions storage engine

See HowtoChangeSessionStore.

The default file system based session store, PStore, will eventually overflow the file system (pre-Rails 1.1 anyway) if you don't do something to prevent it.

See:
http://weblog.textdrive.com/article/196/on-rails-sessions
http://www.bigbold.com/snippets/posts/show/729
http://www.realityforge.org/articles/2006/03/01/removing-stale-rails-sessions

Changing default session storage location

This is probably most useful in a shared hosting environment especially if you are paranoid about what you are storing in your sessions. On older Rails:

config.action_controller.session :tmpdir => "#{RAILS_ROOT}/sessions/" 

The above example will cause all of your sessions to be stored in a directory named sessions instead of the default /tmp on Unix.

For newer Rails this isn't necessary. The Initializer will try to store session details in #{RAILS_ROOT}/tmp/sessions/ by default. It will fall back to a system-wide temporary directory if the above location doesn't exist, so make sure the directory is present and writable if you want sessions stored within your application. If you still want to override this and set your own location, the modern equivalent of the example above is:

ActionController::Base.session_options[:tmpdir] = "/path/to/session/folder/" 

Note the trailing slash character in both examples.

Changing session duration

You can control when the current session will expire by setting the :session_expires value with a Time object. If not set, the session will terminate when the user's browser is closed.

Note: This sets the longevity of the session cookie – I spent a lot of time looking for ways to get a persistent session cookie in Rails, but didn't try this solution initially as I assumed it concerned the session data on the server side.

—HenrikN

Fixed expiry

You can set the :session_expires value in an environment file as directed above.

# session should not last past Jan 1, 2007
config.action_controller.session :session_expires => Time.local(2007,"jan"))

# ...or for newer Rails:
ActionController::Base.session_options[:session_expires] = Time.local(2007,"jan"))

Sliding window since last access

The idea here is to keep the session expiry a fixed duration from the users last visit. There are two ways to accomplish this, though you may use them both (which is recommended):
  1. Set the session cookie expire time.
  2. Check the session life time on the server side.

Set the session cookie expire time

Unfortunately Rails has no way to dynamically set the expiry time of the session cookie. So it is recommended that you use the following plugin, which allows you to accomplish it: http://blog.codahale.com/2006/04/08/dynamic-session-expiration-times-with-rails/

Use the plugin by adding the following to your config/environment.rb:

CGI::Session.expire_after 1.month
# or whatever value you want

Caveat:
When in development mode, the following code works:

class ApplicationController < ActionController::Base
  session :session_expires => 1.hour.from_now
end

But this does not work in production mode! In development mode, ApplicationController is reloaded each time a request is made, and session() is called again for each request. In production mode, session() is only called once. So it effectively becomes a fixed expiry when in production mode.

Check the session life time on the server side

The approach of session expiration through cookie expiration is flawed due to the following:
  • Clients with cookies turned off will never get a session (this, however, is always going to be an issue if you're using cookies to control sessions)
  • Clients with offset time may never be able to set a session cookie, or may be able to avoid session expiration. Try setting session expiration using the above method, giving a window of, say, 10 minutes, and then set your client's clock to localtime+20 minutes. Blam. No session. Set it to last year, and your sessions last for a year and 10 minutes.
  • The fact that session expiration is controlled by the client can impose a security risk.

It would be better to combine cookie expiry with a server-side expiry check. It works by setting a session[:expiration] variable when the session is first assigned, and checking whether the current time has past the expiration time.

# The expiration time.
MAX_SESSION_TIME = 60 * 60

before_filter :prepare_session

def prepare_session
   creation_time = session[:creation_time] || Time.now
   if !session[:expiry_time].nil? and session[:expiry_time] < Time.now
      # Session has expired. Clear the current session.
      reset_session
   end

   # Assign a new expiry time, whether the session has expired or not.
   session[:expiry_time] = MAX_SESSION_TIME.seconds.from_now

   return true
end

Changing session domain

You might want to change the domain of your sessions if your application is accessible by more than one domain. (your-site.com or www.your-site.com, for example.) By default, Rails seems to set the domain as simply the domain that the site was requested with, which is sane, but if you are, say, storing the fact that a certain user is logged in in your session, they'll be logged out if they somehow navigate to the alternate domain.

You can prevent this if you set your session domain to something like .your-site.com. In my production.rb, I have this line:

# Old Rails:
config.action_controller.session :session_domain => '.mysite.com'

# Newer Rails:
ActionController::Base.session_options[:session_domain] = 'mysite.com'

Note the period in front of mysite in the first example – some people find that this is required, others find it must be omitted (it probably depends upon your Rails version; try it and see what works for you).

Put this in your development.rb file if you want to have it work in development (even if you have different domains, all hail different evironments!)

Change session domain for admin only?

How would one go about changing the session domain only for a subset of users, say admin users that are allowed to access all domains without logging in for each domain?
-d723

category:Howto

keywords: session expires

0xced: Dealing with outdated open source libs in Mac OS X
0xced.blogspot.com/2006/07/dealing-with-outdated-o...

Dealing with outdated open source libs in Mac OS X

Mac OS X system frameworks heavily rely on open source libraries. For example, the NSXML classes of the Foundation framework are wrappers around libxml2. The problem is that libxml2 bundled into Mac OS X (10.4.7 as of writing) is version 2.6.16, dating back from november 2004! Current version is 2.6.26 and obviously has fixed a lot of bugs since version 2.6.16.

A specific bug I discovered was rather annoying: NSXMLDocument's validateAndReturnError: method would validate an invalid document. You guessed it, an up-to-date version of libxml2 doesn't suffer from this bug. So the solution to the problem would be to compile the latest version of libxml2 yourself and use this one for your application instead of the system version. This sounds easy but is in fact far from being trivial.

Compiling an universal binary version of libxml2 is easy, this is achieved with the following commands:
$ env CFLAGS="-arch i386 -arch ppc" LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" ./configure --disable-dependency-tracking --enable-static=no --without-python
$ make

Now, libxml2.2.6.26.dylib is almost ready to use inside the .libs directory. I said almost because its install_name is /usr/local/lib/libxml2.2.dylib. Unless you plan to make an installer for your application, you should change it so that it is relative to your application. For example, if your application bundle looks like this:

Contents
  Info.plist
  MacOS
    MyGreatApp
  PkgInfo
  Resources
    ...
  lib
    mygreatlib.dylib
    libxml2.2.6.26.dylib

copy the built library (so that you still have the original dylib in case of problem) and change its install_name with the following commands:
$ cp .libs/libxml2.2.6.26.dylib .
$ install_name_tool -id @executable_path/../lib/libxml2.2.6.26.dylib libxml2.2.6.26.dylib

Now, your application must link against your version of libxml2. To do so, add libxml2.2.6.26.dylib into your Xcode project and check that it has been added to the Link Binary With Libraries phase of your current target.

The latest step is to make sure your libxml2.2.6.26.dylib is going to be used instead of /usr/lib/libxml2.2.dylib at runtime. The problem is that /usr/lib/libxml2.2.dylib uses two-level namespace, meaning that the Foundation framework will always use this one instead of yours. The solution is to force flat namespace by setting the DYLD_FORCE_FLAT_NAMESPACE environment variable. This is achieved by adding the following key in your Info.plist file:

<key>LSEnvironment</key>
<dict>
  <key>DYLD_FORCE_FLAT_NAMESPACE</key>
  <string>YES</string>
</dict>


Your application now uses the latest bug-free version of the lib :-)

This example used libxml2 but obviously apply to any other open source library.

Labels: Tech

Removing Stale Rails Sessions

edit

Posted by Peter Donald on 03/01/2006

By default rails does not clear out stale sessions from the session store. To implement this feature I added the following small snippet of code;

class SessionCleaner
  def self.remove_stale_sessions
    CGI::Session::ActiveRecordStore::Session.
      destroy_all( ['updated_on <?', 20.minutes.ago] ) 
  end
end

And then invoke the remove_stale_sessions method every 10 minutes via;

*/10 * * * * ruby /full/path/to/script/runner 
   -e production "SessionCleaner.remove_stale_sessions"
How non-ActiveRecord models access session objects - Rails Weenie
rails.techno-weenie.net/forums/1/topics/553?page=1

How non-ActiveRecord models access session objects

3 posts, 2 voices

 
antonyf 2 posts, 0 points

I'm creating a class that will run with script/runner to clean up expired sessions in ActiveRecordStore. Although! These sessions has an :cart attribute containing a Cart object with Products in it. These products are taken from a Stock and since the session/cart is expired I need to return the products to the stock.

This is done with Cart#return_items_to_stock which is used in my controllers when a user deletes his Cart manually etc.

  1. class SessionCleaner
  2. def SessionCleaner.clean_expired
  3. expired_sessions = CGI::Session::ActiveRecordStore::Session.find(:all, :conditions => ["(now() - updated_at) > ?", 600])
  4. expired_sessions.each do |session|
  5. session.data[:cart].return_items_to_stock
  6. session.destroy
  7. end
  8. end
  9. end

html | txt

The error i get is of type ArgumentError: undefined class/module Cart | CartItem | Product

If I put this code in an ActionController all I needed to do to solve the problem was:

  1. class SessionController < ActionController
  2. model :cart, :cart_item, :product
  3. end

html | txt

I'm guessing the SessionCleaner needs access to the given models Cart, CartItem, Product to be able to deserialize the objects?

Any suggestions of how to solve to do this or maybe you can recommend an alternative way of doing this?

Thank you!

 
rbates 214 posts, 163 points

Have you tried requiring the the models?

  1. require 'cart'
  2. require 'cart_item'
  3. require 'product'
  4. class SessionCleaner
  5. #...
  6. end
SqlSessionStore now available as a plugin
railsexpress.de/blog/articles/2006/09/15/sqlsessio...

SqlSessionStore now available as a plugin

Posted by Stefan Kaes on 09/15/2006

Rails core team member Rick Olson, the author of many rails plugins, mephisto and techno-weenie, has turned SQLSessionStore into a plugin and sent his code for me to publish. Thanks a lot Rick!

I incorporated the latest changes from my sql session store implementation, most notably postgresql support, and added sql_session_store to my trac installation.

Using sql_session_store is now easier than ever before. Check it out into your vendor/plugins directory:
 
svn co http://railsexpress.de/svn/plugins/sql_session_store/trunk  \
 sql_session_store
and follow the instructions in the README file. If you find any problems with the plugin, which would be entirely my own fault, not Rick's, please use Trac to report them.

BTW, I'd love to get support in for additional database adapters, especially Oracle. I you find the time to write an adapter, please submit the code using Trac, or email me.

Posted in Rails performance, Rails plugins | Tags plugins, sessions | 13 comments

Comments

Leave a response

  1. Ray Slakinski said 8 days later:

    I installed the plugin, and adding the proper things into the environments.rb file via the instructions in the RB. But when I start my app I get

    /sw/var/lib/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123:in `const_missing': uninitialized constant SqlSessionStore (NameError)

  2. Stefan said 8 days later:

    I have no idea why that happens to you. Sorry.

    And please report bugs via trac.

  3. Feurio said 8 days later:

    I get just the very same error.

    How can find out if the plugin is working (e.g rails know about the libs in it?)

    Feurio

  4. Stefan said 8 days later:

    There's a ticket for this problem here

    As of now, I don't know what causes the problem.

  5. chao said 8 days later:

    FYI, for some reason

    rake db:sessions:create

    creates a table without the 「created_at」 field.

    I had to manually add: t.column :created_at, :datetime

    I'm running Rails 1.1.6

    Hope this helps, chao

  6. Stefan said 8 days later:

    @chao: thx for the info.

  7. Ray Slakinski said 8 days later:

    I created my db migration file as per the readme and it created the created_at field

    add_column :sessions, :created_at, :timestamp unless columns.include?(『created_at')

  8. Ray Slakinski said 20 days later:

    Thanks for the update, its all good now on that front. Just a quick question, does SqlSessionStore auto-clear out old sessions from the db? if not how do I perform this?

  9. Stefan said 20 days later:

    SqlSessionStore doesn't clear old sessions automatically.

    Write a cron job to clear the table.

    For Mysql, you can use the following code snippet:
    secs = 3600 # 1hr
    old = "DATE_ADD(updated_at, INTERVAL #{secs} SECOND) < NOW()" 
    SqlSession.delete_all(old)
    
  10. Ray Slakinski said 20 days later:

    Stupid question time, how would I launch that?

  11. Stefan said 20 days later:

    I suggest to ask the Rails mailing list, if you can't figure this one out.

  12. Ray Slakinski said 21 days later:

    Figured out where my problem was in regards to cleaning out old sessions. In your example the last line should have read:

    MysqlSession.delete_all(old)

    Also a tip for those with the problem of 「marshal data too short」 change your data field in your DB from text to mediumtext.

  13. Stefan said 21 days later:

    Oops. You can also use SqlSessionStore.session_class.delete_all.

ubuntu+apache+mongrel_cluster+rails

 

由於沒使用過linux,而rails在linux下方性能良好,因此需要在linux下部署。曾經rails的流行配置是:lighthttpd+fastcgi,但是由於lighthttpd的不穩定,現在有了更好的選擇Apache2.2+Mongrel。Mongrel是Zed Shaw在以前他本人開發的scgi上改進而來,Mongrel跟Apache的關係有些類似Tomcat和Apache——前者負責處理動態內容,後者負責處理靜態內容。由於Apache2.2新引入的若干模塊,包括:mod_proxy_balancer,mod_deflate,前者可以搭配mongrel_cluster從而可以充分利用服務器處理能力,後者則可以以gzip壓縮javascript從而減少網絡傳送延遲,因此當Apache2.2穩定下來以後,Apache2.2+Mongrel_cluster成為我認為更好的rails部署選擇。

經過幾天的摸索,終於在ubuntu5.1上部署成功(version 6應該是一樣的),現記錄如下:

1 安裝Ubuntu必須

gcc
make
autoconf
automake
libtool

用sudo apt update更新軟件倉庫,然後安裝以上套件

sudo apt-get install build-essential

安裝freetype ,jpeg, zlib ,libpng, gd2,可以選擇用sudo apt install,也可以用make install源碼安裝

2 安裝Apache2.2

tar -zxvf httpd-2.2.3.tar.gz

cd httpd-2.2.3
./configure --prefix=/etc/apache2 --enable-cache  --enable-mem-cache
--enable-deflate --enable-proxy --enable-proxy-html --enable-proxy-balancer
--enable-rewrite
make

make install

ln -s /usr/local/apache2/bin/apachectl /usr/local/bin
ln -s /usr/local/apache2/conf/httpd.conf /etc/httpd.conf
mv /usr/local/apache2/htdocs /var/www


3 安裝MySQL

這裡採用的是MySQL 5的Binarary 版本,不需要編譯

tar zxvf mysql-standard-5.0.24-linux-i686-glibc23.tar.gz

groupadd mysql

useradd -g mysql mysql

cd /usr/local

ln -s /usr/local/mysql-standard-4.0.23-pc-linux-i686 mysql

cd mysql

scripts/mysql_install_db --user=mysql

chown -R root .

chown -R mysql data

chgrp -R mysql .

./bin/mysqld_safe --user=mysql &

讓mysql開機自動運行:

cp /usr/local/mysql/support-files/mysql.server /etc/init.d/
sudo update-rc.d mysql.server defaults

實際情況中當執行mysql -uroot -p時會發生var/run/mysqld/mysqld.sock無法找到的錯誤,而在/tmp/mysql.sock是存在的,因此ln -s /tmp/mysql.sock var/run/mysqld/mysqld.sock(這個地方需要商榷一下,因為並沒有發現相關資料)
3 安裝Ruby和Rails以及Rails相關包mongrel,mongrel_cluster

這兩者最為簡單,只需要根據主頁的步驟來做即可

4 讓rails程序可以跑起來

database.yml需要將socket: /path/to/your/mysql.sock調整為socket: /tmp/mysql.sock——接前邊,按照網上mysql的安裝步驟,mysql.sock是應當在var/run/mysqld下的。同時,Dirk在他的文章http://www.dirkye.net/diary/rails_connect_mysql_under_ubuntu中提到Rails 1.1.4在/usr/lib/ruby/gems/1.8/gems/activerecord-1.14.3/lib/active_record/vendor/mysql.rb中定義MYSQL_UNIX_ADDR = "/tmp/mysqld.sock",而他本人的mysql.sock位於/var/run/mysqld/mysqld.sock,因此需要修改MYSQL_UNIX_ADDR = "/var/run/mysqld/mysqld.sock",同時還要安裝sudo apt-get install libmysql-ruby1.8,不過我並沒有遇到該問題,在database.yml配置好之後rails應用程序就可以跑起來了。

5 安裝RMagick

首先安裝Imagemagick

這個沒什麼好說的,也是源代碼安裝

隨後在RMagick的安裝說明中提到的其他庫如zlib,freetype,jpeg等在前邊都已經安裝成功,因此直接下載rmagick編譯安裝即可。

沒有採用Ubuntu的apt

6  配置Mongrel

配置mongrel

mongrel_rails cluster::configure -e development -p 8000 -a 127.0.0.1 -N 2 -c
/var/www/<your_app>

cd /etc/init.d
sudo ln -s /usr/local/lib/ruby/gems/1.8/gems/mongrel_cluster-0.2.0/resources/mongrel_cluster
mongrel_cluster
sudo chmod +x mongrel_cluster
sudo /usr/sbin/update-rc.d mongrel_cluster defaults

cd /etc
sudo mkdir mongrel_cluster
cd mongrel_cluster
sudo ln -s /var/www/<your app>/config/mongrel_cluster.yml <your_app>.yml

實際配置中mongrel_cluster沒有自動運行起來,因此先手動mongrel_rails cluster::start

7  配置Apache

修改httpd.conf中DocumentRoot為 "/var/www" 應用程序放在/var/www下

同時

<Directory /var/www/>
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

打開extra下的httpd-vhosts.conf

NameVirtualHost *:80
<VirtualHost *:80>
  ServerAdmin <your_email> <andrew.n.stone at gmail.com>
  ServerName <your_app>.com
  DocumentRoot /var/www/<your_app>/public
  ErrorLog /var/log/apache2/<your_app>_error_log
  CustomLog /var/log/apache2/<your_app>_access_log combined

  <Directory /var/www/<your_app>/public>
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>

  ProxyPass /images !
  ProxyPass /stylesheets !
  ProxyPass /javascripts !
  ProxyPass /favicon.ico !
  ProxyPass / balancer://<your_app>_cluster
  ProxyPreserveHost On

  <Proxy balancer://<your_app>_cluster>
    BalancerMember http://127.0.0.1:8000
    BalancerMember http://127.0.0.1:8001
  </Proxy>

  RewriteEngine On

  RewriteRule ^/$ /index.html [QSA]

  RewriteRule ^([^.]+)$ $1.html [QSA]

  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
  RewriteRule ^/(.*)$ balancer://<your_app>_cluster%{REQUEST_URI} [P,QSA,L]

  # Deflate
  AddOutputFilterByType DEFLATE text/html text/plain text/xml
application/xml application/xhtml+xml text/javascript text/css
  BrowserMatch ^Mozilla/4 gzip-only-text/html
  BrowserMatch ^Mozilla/4.0[678] no-gzip
  BrowserMatch bMSIE !no-gzip !gzip-only-text/html

</VirtualHost>

然後在httpd.conf中將Include conf/extra/httpd-vhosts.conf的註釋去掉

啟動apache: apachectl start

這樣ubuntu+mongrel+apache+rails就配置成功了

以上記錄並不完整,大體步驟是這樣子的。

Onion的天空 » SSH不输入密码连接远程Linux主机
72.14.203.104/search?sourceid=navclient-ff&ie=UTF-...

SSH不输入密码连接远程Linux主机

By Onion under Linux  | Tag : skill, ssh, Ubuntu, 技巧
You can skip to the end and leave a response. Pinging is currently not allowed.  

系统环境 : ubuntu 6.06 / ubuntu 6.10
SSH服务 : openssh-server

1) 在本地主机生成密钥对

ssh-keygen -t rsa

这个命令生成一个密钥对:id_rsa(私钥文件)和id_rsa.pub(公钥文件)。默认被保存在~/.ssh/目录下。

2) 将公钥添加到远程主机的 authorized_keys 文件中

将文件上传到远程主机中

scp ~/.ssh/id_rsa.pub root@192.168.1.23:/root/

SSH到登陆到远程主机,将公钥追加到 authorized_keys 文件中

cat /root/id_rsa.pub >> /root/.ssh/authorized_keys

或直接运行命令:

cat ~/.ssh/id_dsa.pub|ssh root@192.168.1.23 `cat - >> ~/.ssh/authorized_keys`

3) 重启 open-ssh 服务

/etc/init.d/ssh restart

4) 测试

scp /home/onion/.ssh/id_rsa.pub root@192.168.1.23:/root/

呵呵,不用输入密码了:)

 

3 Responses to “SSH不输入密码连接远程Linux主机”

  1. rae Says:
    November 3rd, 2006 at 10:02 pm

    可以简化一点操作,
    scp ~/.ssh/id_rsa.pub root@192.168.1.23:/root/.ssh/authorized_keys

    而且个人的公钥操作不影响服务器,服务器不必重启。

  2. rae Says:
    November 3rd, 2006 at 10:16 pm

    解决多人合用一个authorized_keys的情况:
    cat ~/.ssh/id_rsa.pub|ssh root@192.168.1.23 ‘cat >> /root/.ssh/authorized_keys’

    修正 :
    cat ~/.ssh/id_rsa.pub|ssh 192.168.1.23 `cat - >> ~/.ssh/authorized_keys`

compiling imagemagick on mac os x (v10.2+)
www.mudpub.com/docs/imagemagick-osx.html
ompiling imagemagick on mac os x (v10.2+)
written by: takashi okamoto january 4, 2004
NOTE: start by compiling libjpeg, libpng, libtiff and freetype2. if you already have them, just skip all the way down to compile imagemagick. YOU WILL NEED THE DEVELOPER TOOLS INSTALLED!

libjpeg - for jpeg support
curl -O http://www.ijg.org/files/jpegsrc.v6b.tar.gz
tar zxvf jpegsrc.v6b.tar.gz
cd jpeg-6b
cp /usr/share/libtool/config.sub .
cp /usr/share/libtool/config.guess .
./configure --enable-shared --enable-static
make
sudo make install
sudo ranlib /usr/local/lib/libjpeg.a

libpng - for png support
curl -O ftp://swrinde.nde.swri.edu/pub/png/src/libpng-1.2.5.tar.bz2
bzcat libpng-1.2.5.tar.bz2 | tar xf -
cd libpng-1.2.5
sed 's/LDFLAGS=-L/LDFLAGS=-dynamiclib -L/' < scripts/makefile.macosx > Makefile
make ZLIBLIB=/usr/lib ZLIBINC=/usr/include
sudo make install
sudo ranlib /usr/local/lib/libpng.a

libtiff - for tiff support
curl -O ftp://ftp.remotesensing.org/pub/libtiff/tiff-v3.5.7.tar.gz
tar zxvf tiff-v3.5.7.tar.gz
cd tiff-v3.5.7
./configure
make
sudo make install
sudo ranlib /usr/local/lib/libtiff.a

freetype2 - for type support
curl -O http://umn.dl.sourceforge.net/sourceforge/freetype/freetype-2.1.5.tar.gz
tar zxvf freetype-2.1.5.tar.gz
cd freetype-2.1.5
./configure
make
sudo make install
sudo ranlib /usr/local/lib/libfreetype.a

imagemagick
curl -O ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-5.5.7-15.tar.gz
tar zxvf ImageMagick-5.5.7-14.tar.gz
cd ImageMagick-5.5.7
./configure --enable-shared
sudo make -k -i install
sudo ranlib /usr/local/lib/libMagick*

The content on this page is provided by a Google Notebook user, and Google assumes no responsibility for this content.