Friday, December 10, 2010

How to run Ruby On Rails on Google AppEngine

There is a tutorial by John Woodell,
however it was costumed running in linux environment,
and there is some slightly changes they didnt update their tutorial script,
the script will not work out of box,
So in here i am making a post, especially benefit users who is using Windows, since i am doing this in Windows 7 environment.

PS : i assume u have already had a google app engine account, if u dont, do some bing or google and get one

Specially thanks for the help from Andrew Myers.

1) Install Java Development Kit  6 (JDK6)
make sure you have similar stuff show up in your windows command prompt

>java -version
java version "1.6.0_21"
Java(TM) SE Runtime Environment (build 1.6.0_21-b07)
Java HotSpot(TM) Client VM (build 17.0-b17, mixed mode, sharing)
>javac -version
javac 1.6.0_21

2)  Install AppEngine Java SDK
Download the  SDK, and make sure you put the "bin" directory into your system "Path"

3) Install Ruby 1.8.7 , not JRuby
Download it, and install it,
make sure you associate *.rb to be run by ruby  (for windows install, there is the check box)

again open a new command prompt, you should have something like this

>ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i386-mingw32]
>gem -v

4) Install plugins for ruby
Make sure you open a command prompt as administrator

and type in the following one by one

gem install google-appengine
gem install rails -v "2.3.10"
gem install rails_dm_datastore
gem install activerecord-nulldb-adapter

each one will take a while, so be patient..

5) Make a directory to contain you ROR application
mkdir railsv1
cd railsv1
Or you can use GUI to do it.

6) Copy the code below, save it to a file, for example rails2310_appengine.rb

# Copyright:: Copyright 2009 Google Inc.
# Original Author:: John Woodell (
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# See the License for the specific language governing permissions and
# limitations under the License.

require 'fileutils'
require 'open-uri'

def composite(source, fragment, index = nil, trim = nil), 'r+') do |f|
    lines = f.readlines
    lines = lines[0..trim] unless trim.nil?
    f.pos = 0 do |z|
      section = z.readlines
      if index and index.size < lines.size
        f.print lines[0,index] + section + lines[index..-1]
        f.print lines + section
  FileUtils.rm fragment

def download_file(path, url)
  open(url) do |r|
    open(path,"w"){|f| f.write( }
SET_CMD = RUBY_PLATFORM.include?('mswin32') ? 'set' : 'export'
MORE_GEMS = 'rails_appengine/active_support_vendored'
MOD_FILES = %w{ app/controllers/rails/info_controller.rb public/favicon.ico
       config/boot_rb config/environment_rb
                config/initializers/gae_init_patch.rb config/database.yml
                script/ script/ script/ }
# Install Rails 2.3.10
FileUtils.touch ''
gemsrc = ARGV[0].eql?('tiny_ds') ? 'Gemfile_td' : 'Gemfile'
download_file("Gemfile", "#{FILE_BASE}#{gemsrc}")
download_file("gems_2310", "#{FILE_BASE}gems_2310")
composite('Gemfile', 'gems_2310', nil, -2)
FileUtils.mkdir_p 'WEB-INF'
download_file("WEB-INF/app.yaml", "#{FILE_BASE}WEB-INF/app.yaml")
system 'appcfg.rb bundle --update .'
# Remove dups and generate Rails app
# Generate rails, and skip APIs to escape the shell
system "rails _2.3.10_ ."
# Fetch configuration files
FileUtils.mkdir_p 'app/controllers/rails'
MOD_FILES.each { |path| download_file(path, "#{FILE_BASE}#{path}") }
if ARGV[0].eql? 'tiny_ds'
  download_file("config/environment_rb", "#{FILE_BASE}config/environment_td")
# Merge configs into boot.rb
composite('config/boot.rb', 'config/boot_rb', 108)
# Merge configs into environment.rb
composite('config/environment.rb', 'config/environment_rb', 30)
# install the nulldb adapter
system 'ruby script/plugin install'
puts "##"
puts "## Now type 'dev_appserver.rb .'"
puts "##"

and run it, e.g  ruby rails2310_appengine.rb, then you should see something like this:

Friday, July 16, 2010

When pervasive computing meet cloud computing, Infinite VS Infinite

Recently i am trying to propose a project base on my previous idea, of course a lot more detail than the pose, for my PhD study project. Now i am crazily reading papers from all kinds of area, Internet of Things, Pervasive Computing, Cloud Computing, try to identical the research value in my project.

Today just want to share some interesting finding during my reading.

In one paper "Pervasive commuting a paradigm for the 21st century" by Debasbis saba and Amitava Mukberjee. In the issues and challenges they mentioned,

"Though pervasive computing components are already deployed in many environments, integrating them into a single platform is still a research problem. The problem is similar to what researchers in distributed computing face, but the scale is bigger,. As the number of devices and applications increases, integration becomes more complex. For example, servers must handle thousands of concurrent client connections, and the influx of pervasive devices would quickly approach the host`s capacities. We need a confederation of autonomous servers cooperating to provide user services."

And as what we know from cloud computing, we can ask/rent as much as computing resource we want to deal with our need.

If a auto-scale framework/model with a multi tenancy architecture application can be created base on cloud computing, sounds like the issue mention in the paper will be easily solved.

And imaging if there is a standard that can build into all electric appliances, and there is a router like agent that can collect information/send control signal from/to those electric appliances, and process all these information in the multi tenancy architecture application up in the cloud, such kind of project will absolutely benefit human being a lot than we can expected

More, some of the projects demo in Oxygen MIT can be easily achieve without creating any new technologies as well.

Thursday, May 6, 2010

Relation Decoupling -- Migrate from Relational Database to Non Relational Cloud Database


Relation decoupling problem

When doing the migration, there are lots of complex join/ cross table selection query, or views which implemented by such kind of query

Possible solution:

1) To migrate such kind of data, seems we need to re-model the database, get rid of the relation in the database, and move these kind of logic to be application logic (code implementation).

Similar to Ruby On Rails, they handle all the relation in coding logic, database level relation is not a must.

2) Pre-Process relation, and keep all these result in database, when certain query come, server can return data right away.

This approach looks like some kind of data warehouse. Which might only suitable for application only do read action mostly.

But it doesn`t mean we cannot do write action.

We can ask the application direct request to another server which particularly  design for writing data.

The only drawback is that, the result might not be able to display in a instance manner.  It depends on how agile my “Process Engine” can be.

Thursday, April 22, 2010

A Peek at Multitenancy in Azure Table Stroage

As we know, the programming model for using Windows Azure Table Storage is  like the pic below

Every table can be partition base on custom Partition Key.
If we take the advantage of the Partition Key, we can easily create a multitenancy data struture.

For example, we want to create a multi user blog, just like blogger, we might have a table, call it "Posts", to store the post written by the users.
Obviously, we can use the username as Partition Key.
So base on the username, we can easily retrieve the corresponding set of data.

This kind of approach, data level multitenacy architecture, should be exist long time ago, but apply them onto cloud storage, will gain benefit that we cannot have in relational database.

Cloud storage will guarantee all data be stored highly distributed, and fully replicated.

Thursday, April 1, 2010

Cloud computing + IT management = Home Automation

I just have something in my mind that, there will be soon or already become real for some of the area, everything will be or has been able to connect internet.

And when talking about IT management, usually we will think of enterprise, company or education institute, when they are up to a certain size, they need a central administration to enable them to manage all the computing device.

However, people are invoke with technologies lot more than people live in the older days,
individual or family also need there own IT management ...

1) individual who want to use all his/her computer device all in one entry
2) family to share all the resource

And for sure, nowadays we can do things like this for enterprise/companies/education institutes

Migrate all the IT manage system onto the cloud,
instead of hosting the management system on-premises, we host it on to cloud.

For IT management vendor, they no longer need to maintain the server, they just focus on how to make there management software to meet customers needs.

And because the cloud can provide "pay as you go", now the IT Management vendor can also easily offer "pay as you go" for using there service.

Now interesting things happened.
The idea of office automation, home automation have been there for ages, but why still lots of the people cannot take the benefit, that is because it will be too much to buy, install and maintain the devices, software system all kind of stuff.

But now all the price are going down,
for IT management vendors, they can just create a multi-tenancy architecture IT management system.
and i assume all digital device will be able to control via internet,
and the multi-tenancy architecture IT management system will allow user to plug-in Printer, Fax machine, Fridge, Air Conditioner, Washing machine etc .....

then something like this can be happened

Imaging just like nowadays mobile plan service, maybe in the future IT management vendor will also offer home device management service, let say you pay 30 bucks a month, it will allow you to have a dashboard to see how much electricity you have been consume, what are the condition of the device, and allow you to schedule or even make a work-flow to turn on and shutdown devices etc.....

Bullshit ends...

A pattern for how to do monitoring on distributed, large scattered applications

Coming soon ...

Real project was done, will be update once some copy right issue is done ...

Friday, February 12, 2010

WCF Service In Windows Azure Worker Role

I am going to show you how to create a WCF web service which will be hosted on a worker role in Windows Azure.

Before we continue, i assume you already know how to create a helloworld WCF web service, also know how to deploy application onto Windows Azure and know what is input endpoint.

Step 1 - Create a WCF Service

When creating a WCF service, always we need to define a contract and them implement it.

There is one thing i need to raise. When implementing the contract, and if you want to use any other binding rather than BasicHttpBinding, we need to specify
[ServiceBehavior(AddressFilterMode = AddressFilterMode.Any)]

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;

namespace WorkerRole1
public interface IMessageDeliver
void DoWork(string message);

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
using System.Diagnostics;
using Microsoft.WindowsAzure.ServiceRuntime;

namespace WorkerRole1
[ServiceBehavior(AddressFilterMode = AddressFilterMode.Any)]
public class MessageDeliver : IMessageDeliver
public void DoWork(string message)
Trace.TraceInformation("{0} Receive Message - {1}.", RoleEnvironment.CurrentRoleInstance.Role.Name, message);

Step 2 - Open a port to listen

In the ServiceDefinition.csdef, we need to define a InputEndpoint, so that the outside world can talk to our worker role.

I am going to use NetTcpBinding, so i make the port as tcp. The other options you can have are http and https, which need to be match to the binding u are going to use.

Step 3 - Combine WCF with the port we open

ServiceHost serviceHost = new ServiceHost(typeof(MessageDeliver));

NetTcpBinding binding = new NetTcpBinding(SecurityMode.None);

// define an external endpoint for client traffic
RoleInstanceEndpoint externalEndPoint = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["MessageDeliver"];

string endpoint = String.Format("net.tcp://{0}/MessageDeliver", externalEndPoint.IPEndpoint);

serviceHost.AddServiceEndpoint(typeof(IMessageDeliver), binding, endpoint);


Step 4 - Create a client to double check

ChannelFactory cfactory = new ChannelFactory(new NetTcpBinding(SecurityMode.None), "net.tcp://");

var client = cfactory.CreateChannel();

client.DoWork("hi from shrimpy");

Done !!! Have fun ...