Friday, May 1, 2015

Deploy your Go app onto Azure App Services (Websites) in ease

Try it:

             Click this link, you will deploy a sample Go web app onto your Azure subscription

Details:

Azure App Services now supports Go 1.4.2 with continues deployment. Once continues deployment is setup, every time you push code to your continues deployment branch, a new deployment will be trigger.

if you read the deployment log detaily, you will notice that Azure will create a Go workspace in temp folder, then copy your code to src\azureapp, build against azureapp folder and produce an "azureapp.exe".

Last, it generate a web.config and use HttpPlatformHandler to run your Go app.

Restriction:

  • Have to place main package at the root of your app
  • Currently only support Go 1.4.2, no able to select a specific Go version yet


Sample Go app:
    https://github.com/shrimpy/gotry

Sample Deployment Log:


Handling Go deployment.
Prepare workspace
GOROOT D:\Program Files\go\1.4.2
Creating GOPATH\bin D:\local\Temp\30a88a27-1ce2-49ef-b1b8-6f32716c9652\gopath\bin
Creating GOPATH\pkg D:\local\Temp\30a88a27-1ce2-49ef-b1b8-6f32716c9652\gopath\pkg
Creating GOPATH\src D:\local\Temp\30a88a27-1ce2-49ef-b1b8-6f32716c9652\gopath\src
Creating D:\local\Temp\30a88a27-1ce2-49ef-b1b8-6f32716c9652\gopath\src\azureapp
Copy source code to Go workspace
 -------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------
   Started : Thursday, April 30, 2015 5:52:54 PM
   Source : D:\home\site\repository\
     Dest : D:\local\Temp\30a88a27-1ce2-49ef-b1b8-6f32716c9652\gopath\src\azureapp\
     Files : *.*
 Exc Files : .deployment
   deploy.cmd
  Exc Dirs : .git
   .hg
   Options : *.* /NDL /NFL /S /E /DCOPY:DA /COPY:DAT /NP /R:1000000 /W:30 
 ------------------------------------------------------------------------------
 ------------------------------------------------------------------------------
                Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         3         1         1         0         0         0
   Files :        13        13         0         0         0         0
   Bytes :    19.4 k    19.4 k         0         0         0         0
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00
    Speed :              182743 Bytes/sec.
   Speed :              10.456 MegaBytes/min.
   Ended : Thursday, April 30, 2015 5:52:54 PM
 Resolving dependencies
Building Go app to produce exe file
Copy files for deployment
KuduSync.NET from: 'D:\home\site\repository' to: 'D:\home\site\wwwroot'
Deleting file: 'hostingstart.html'
Copying file: 'azureapp.exe'
Copy web.config
        1 file(s) copied.
Finished successfully.


Thursday, April 16, 2015

Running Go web app on Azure App Services (Websites) with custom deployment script

Short version:


Perform a continues deployment with code from this repo to your Azure Website. Once deploy, you should be able to see below result, a perfect test web app that use Go "net/http" package, Gin and Martini all together.



Behind the scenes:


The core is to understand GoDeploy.cmd script from repo, below are the key concepts:

To run Go app:
     Create a web.config as below. If you build your go app (exe file), all you need is upload your exe file and update web.config file to reference to it.


    
        
            
        
        
        
    



To build:

GOROOT:
    There is no GOROOT environment variable yet, but the binary is reside in "D:\Program Files\Go\1.4.2". define your own GOROOT and point to it

Build Script:

  •     Create workspace and config GOPATH point to it

              workspace:
                    {folder}/src
                    {folder}/bin
                    {folder}/pkg

    ECHO creating %GOPATH%\bin
    MKDIR "%GOPATH%\bin"
    ECHO creating %GOPATH%\pkg
    MKDIR "%GOPATH%\pkg"
    ECHO creating %GOPATH%\src
    MKDIR "%GOPATH%\src"


  • Create app folder under "{folder}/src", and copy source code into it
ECHO creating %GOAZUREAPP%
MKDIR %GOAZUREAPP%

ECHO copying sourc code to %GOAZUREAPP%
CP gotry.go %GOAZUREAPP%

  • Resolve dependencies and build
ECHO Resolving dependencies
CD "%GOPATH%\src"
%GOEXE% get %FOLDERNAME%

ECHO Building ...
%GOEXE% build -o %WEBROOT_PATH%\%FOLDERNAME%.exe %FOLDERNAME%

Monday, January 30, 2012

Windows Azure Storage Mapper

Open source project: http://azuredbmapper.codeplex.com/
The StorageClient library API is not that easy while you try to understand how the REST API is working for you.

Here I created an Azure Storage Mapper, which is purelly expose the REST API.
All what you read from MSDN about REST API, you will find it in Azure Storage Mapper.
Same wording, same way to use. Give you the very native feeling of Azure Storage.

Right now only support Table Storage... Blob Storage and Queue Storage is coming soon ...

Friday, December 10, 2010

How to run Ruby On Rails on Google AppEngine

There is a tutorial https://gist.github.com/671792 by John Woodell,
however it was costumed running in linux environment,
and there is some slightly changes they didnt update their tutorial script,
the script will not work out of box,
So in here i am making a post, especially benefit users who is using Windows, since i am doing this in Windows 7 environment.

PS : i assume u have already had a google app engine account, if u dont, do some bing or google and get one

Specially thanks for the help from Andrew Myers.

1) Install Java Development Kit  6 (JDK6)
make sure you have similar stuff show up in your windows command prompt

>java -version
java version "1.6.0_21"
Java(TM) SE Runtime Environment (build 1.6.0_21-b07)
Java HotSpot(TM) Client VM (build 17.0-b17, mixed mode, sharing)
>javac -version
javac 1.6.0_21

2)  Install AppEngine Java SDK http://code.google.com/appengine/downloads.html
Download the  SDK, and make sure you put the "bin" directory into your system "Path"
















































3) Install Ruby 1.8.7 , not JRuby
Download it http://www.ruby-lang.org/en/downloads/, and install it,
make sure you associate *.rb to be run by ruby  (for windows install, there is the check box)

again open a new command prompt, you should have something like this

>ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i386-mingw32]
>gem -v
1.3.7

4) Install plugins for ruby
Make sure you open a command prompt as administrator

and type in the following one by one

gem install google-appengine
gem install rails -v "2.3.10"
gem install rails_dm_datastore
gem install activerecord-nulldb-adapter

each one will take a while, so be patient..

5) Make a directory to contain you ROR application
mkdir railsv1
cd railsv1
Or you can use GUI to do it.

6) Copy the code below, save it to a file, for example rails2310_appengine.rb

#!/usr/bin/ruby
#
# Copyright:: Copyright 2009 Google Inc.
# Original Author:: John Woodell (mailto:woodie@google.com)
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

require 'fileutils'
require 'open-uri'

def composite(source, fragment, index = nil, trim = nil)
  File.open(source, 'r+') do |f|
    lines = f.readlines
    lines = lines[0..trim] unless trim.nil?
    f.pos = 0
    File.open(fragment) do |z|
      section = z.readlines
      if index and index.size < lines.size
        f.print lines[0,index] + section + lines[index..-1]
      else
        f.print lines + section
      end
    end
    f.truncate(f.pos)
  end
  FileUtils.rm fragment
end

def download_file(path, url)
  open(url) do |r|
    FileUtils.mkpath(File.dirname(path))
    open(path,"w"){|f| f.write(r.read) }
  end
end
SET_CMD = RUBY_PLATFORM.include?('mswin32') ? 'set' : 'export'
MORE_GEMS = 'rails_appengine/active_support_vendored'
FILE_BASE = 'http://appengine-jruby.googlecode.com/hg/demos/rails2/'
MOD_FILES = %w{ app/controllers/rails/info_controller.rb public/favicon.ico
                config.ru config/boot_rb config/environment_rb
                config/initializers/gae_init_patch.rb config/database.yml
                script/console.sh script/publish.sh script/server.sh }
# Install Rails 2.3.10
FileUtils.touch 'config.ru'
gemsrc = ARGV[0].eql?('tiny_ds') ? 'Gemfile_td' : 'Gemfile'
download_file("Gemfile", "#{FILE_BASE}#{gemsrc}")
download_file("gems_2310", "#{FILE_BASE}gems_2310")
composite('Gemfile', 'gems_2310', nil, -2)
FileUtils.mkdir_p 'WEB-INF'
download_file("WEB-INF/app.yaml", "#{FILE_BASE}WEB-INF/app.yaml")
system 'appcfg.rb bundle --update .'
# Remove dups and generate Rails app
# Generate rails, and skip APIs to escape the shell
system "rails _2.3.10_ ."
# Fetch configuration files
FileUtils.mkdir_p 'app/controllers/rails'
MOD_FILES.each { |path| download_file(path, "#{FILE_BASE}#{path}") }
if ARGV[0].eql? 'tiny_ds'
  download_file("config/environment_rb", "#{FILE_BASE}config/environment_td")
end
# Merge configs into boot.rb
composite('config/boot.rb', 'config/boot_rb', 108)
# Merge configs into environment.rb
composite('config/environment.rb', 'config/environment_rb', 30)
# install the nulldb adapter
system 'ruby script/plugin install http://svn.avdi.org/nulldb/trunk/'
puts "##"
puts "## Now type 'dev_appserver.rb .'"
puts "##"


and run it, e.g  ruby rails2310_appengine.rb, then you should see something like this:

Friday, July 16, 2010

When pervasive computing meet cloud computing, Infinite VS Infinite

Recently i am trying to propose a project base on my previous idea, of course a lot more detail than the pose, for my PhD study project. Now i am crazily reading papers from all kinds of area, Internet of Things, Pervasive Computing, Cloud Computing, try to identical the research value in my project.

Today just want to share some interesting finding during my reading.

In one paper "Pervasive commuting a paradigm for the 21st century" by Debasbis saba and Amitava Mukberjee. In the issues and challenges they mentioned,

"Though pervasive computing components are already deployed in many environments, integrating them into a single platform is still a research problem. The problem is similar to what researchers in distributed computing face, but the scale is bigger,. As the number of devices and applications increases, integration becomes more complex. For example, servers must handle thousands of concurrent client connections, and the influx of pervasive devices would quickly approach the host`s capacities. We need a confederation of autonomous servers cooperating to provide user services."

And as what we know from cloud computing, we can ask/rent as much as computing resource we want to deal with our need.

If a auto-scale framework/model with a multi tenancy architecture application can be created base on cloud computing, sounds like the issue mention in the paper will be easily solved.

And imaging if there is a standard that can build into all electric appliances, and there is a router like agent that can collect information/send control signal from/to those electric appliances, and process all these information in the multi tenancy architecture application up in the cloud, such kind of project will absolutely benefit human being a lot than we can expected

More, some of the projects demo in Oxygen MIT can be easily achieve without creating any new technologies as well.

Thursday, May 6, 2010

Relation Decoupling -- Migrate from Relational Database to Non Relational Cloud Database

Issue

Relation decoupling problem


When doing the migration, there are lots of complex join/ cross table selection query, or views which implemented by such kind of query


Possible solution:


1) To migrate such kind of data, seems we need to re-model the database, get rid of the relation in the database, and move these kind of logic to be application logic (code implementation).


Similar to Ruby On Rails, they handle all the relation in coding logic, database level relation is not a must.



2) Pre-Process relation, and keep all these result in database, when certain query come, server can return data right away.


This approach looks like some kind of data warehouse. Which might only suitable for application only do read action mostly.


But it doesn`t mean we cannot do write action.

We can ask the application direct request to another server which particularly  design for writing data.


The only drawback is that, the result might not be able to display in a instance manner.  It depends on how agile my “Process Engine” can be.


Thursday, April 22, 2010

A Peek at Multitenancy in Azure Table Stroage

As we know, the programming model for using Windows Azure Table Storage is  like the pic below



Every table can be partition base on custom Partition Key.
If we take the advantage of the Partition Key, we can easily create a multitenancy data struture.

For example, we want to create a multi user blog, just like blogger, we might have a table, call it "Posts", to store the post written by the users.
Obviously, we can use the username as Partition Key.
So base on the username, we can easily retrieve the corresponding set of data.



This kind of approach, data level multitenacy architecture, should be exist long time ago, but apply them onto cloud storage, will gain benefit that we cannot have in relational database.

Cloud storage will guarantee all data be stored highly distributed, and fully replicated.