Tuesday, December 15, 2009

Auto migrate exising ASP.NET application onto Windows Azure

Source code Package AutoDeployRobot_WithCommandFile.zip

In this video I am going to show how to migrate a existing ASP.net Application to Windows Azure.

There will be two way, 1) via application 2) via powershell script

SOmething you need to know:

We will use Azure Management API and Azure storage API for this demo.

In order to pass Azure Management API authentication, we need the Subscription ID and a Self-signed X509 certificate

In order to pass Azure storage API authentication, we need the Account Name and Access Key

When try to do auto deploy, we have to save deployment file onto azure blob storage first, then ask Windows azure to look for the file from the storage to make the deployment.

Demo 1, migrate via an application which implentment by using Azure Management API and Azure storage API

1) show the existing asp.net application

2) create a service definition file

3) use cspack to pack the application

4) upload file onto blob storage

5) deployment onto windows Azure

a web role "Ganda" will call the management API to deploy the application onto Azure, after that, the web role pass a message to a worker role "Ant", the worker

role will keep checking the satus of the deployment, once it was deployed, the worker role will ask the deployment to run

Demo 2, migrate via powershell script

automatically do all the job from step 3 to 5 in demo 1

Thank you for watching...

Monday, September 28, 2009

All the bad things in Amazon Web Service

Hi folks.... This article is going to list all what i experience the shit stuff from Amazon Web Service...

I admit that AWS is very powerful, but you have to be convinced that nothing is perfect, they will have their down side...

1) You can`t delete an instance from your list after you terminate it.
Your instance list will keep growing, depending on the "garbage clean up" of AWS. AWS say they will delete your termination instance at later certain period of time. But when??? Only God will know.

2) You can`t restart a terminated instance
Amazon will keep billing you if you do not terminate the instance. This is painful, coz sometime we just want to do some test setup, not really launch the server for real. Let say i want to install five piece of software into my machine, today i do two, and want to do another three tomorrow. In EC2, I cannot say turn off the computer and restart it again tomorrow. I have to pay for the extra idle time.

3) It is hard to do Authentication programatically
Either Query Approach or SOAP Approach, Amazon do not provide detail in-depth tutorial or document to guild us developer to do the authentication.
All we got is a simple page, to show us, we need to do URL Encoding when using Query API, we have to attache the X509 certificate with every request we make when usning SOAP API.
That is all....
No wonder the opensource project Typica and Netflexity-amazonws-ec2 is so popular in Amazon support forum....

I was trying to use native Query API and SOAP API at the begining...but give up after a few hour try...i give up..coz there is no documentation..i end up reading source code from Typica and Netflexity.....so i just us Typica........so frustrated...

But on the other hand..Microsoft Windows Azure management API is so easy to use.
I learn and play straight away...

4) Amazon-EC2-AMI-Tools Linux only
This tool set, there is no support for Windows.
Have to use third-party tools such as ElastixFox (Firefox plugin)

5) Disk limitation when bundle Windows Instance (create custom windows AMI)
The basic template we can get from Amazon is a windows 2003 server with C drive only 10G.
And we cannot chance the size of the C drive.
Why i keep mention the C drive, coz all custom stuff, if you do not put into C drive, when you perform bundle, only C drive will be bundle, all other data in other drivers will be ignnore.
(Linux instance also has similar problem, all custom stuff have to put under /mnt, otherwise your own data won`t be bundled)
Coz the work i am doing is that, i need to raise multiple instance which running the same application.
So my solution is that, i create my own AMI, which has been config everything, when i what more, i just raise a instance from that AMI.
But with 10G limitation, i just cannot feel satisfaction, lucky everything i need after installing into the C driver, there still 3 to 4 G space left...but i just cannot imaging...how about my application become larger?...dose it mean i have to give up using Amazon EC2???
Anyway, i haven`t totally confirm that there is no way to walk around this limitation.
I saw one of the post in the Amazon support forum, they say we can plug-in volumn for extra space...hmm..i am thinking, whether when create a custom AMI, we can bundle the volumn as well.
Further investigation is needed... and i will keep update after if anything is found...

To be continue ...

Monday, September 21, 2009

Time to build a version control app for Azure

Introducing the Windows Azure Service Management API

Azure has release its management API.

Now azure has most of the stuff, but there is one thing Azure do not have.

Deployment version control....

If you want to down-grade your deployment, and if your stageing deployement is not the down grade version.

there is no way for you to down grade.

Since the management API has been release....it is time for version control now....


1) Do not store any user information
2) Store data in user`s storage

Howto Implement:
1) The Version Control App(VCA) is a web portal
3) When user come to VCA, user need to provide keys, subscritionID etc,
for VCA to access SQL Azure or Table Stroage, in order to store version info, to access Blob storage, in order to storage deployment files.
4) VCA is only a graphical interface. But behind the scene, some logic were applied to organize those data, in order to provide version control. And for sure, VCA will invoke Azure Service Management API, to help user swap deployement doing upgrade or downgrade.

Monday, August 31, 2009

How to do stateful Registration In Windows Azure when using multi instances

Microsoft Windows Azure cloud enable you to horizontally scale out your application by modified the number of instance on the fly, and the fabric controller will handler the load balance for you.

But one issue is that, u cant do stateful application for this kind of scale.

For example, user login.
Normally the way we do login is that. Once the user has been successfully login, we put some of the data into session, so that our application will know the user has been login within the time out duration.

However, when multi-instance running the same piece of application, if you store info in one of those instances, the other instance wouldn`t be aware of the user has been login.

Below is the work around for using multi-instances.

What i am going to do is that,

Web Roles:
Every time a web role receive one request, check whether there is a login record in the table storage, if the data existed (e.g we can use the userid for RowID, if the userid existed), then see the lable "expired" for every row,

if it is false, which mean the user has already login, update the label "last_visit"
otherwise, hasn`t login, direct to login page.
after successful login, create one entry in the table.

Worker Role:
Schedule tasks to check the session table, see whether "current_time" - "last_visit" > time out
if true, set "expired" to true, otherwise set to false.

1) Data consistency
Data might updated by one instance, but when another instance call for the same piece of data, that data might not be updated, maybe still getting the old value.

2) Too much over head

I think this approach is doable.

For consistency, even the value we read is the old value, which mean the time out wouldn`t be the exact time as we specify. Maybe it will be a bit larger or smaller then the expected value.

For over head. There is something we need to compromise, in order to have horizontal scalability.

How about other Cloud

I only want to talk about Google App Engine here, coz gogrid or amazon web service, there are more likely offer "vertical scale out".

In Google App Engine, they already offer user the MEM-Cache, which can be shared by all the instances. And the access time or speed is fast.

Which mean it would be easy and simple to implement my approach in Google App Engine.
And we won`t has the two concerns as well.

Saturday, August 22, 2009

News about cloud computing evaluation on Microsoft Windows Azure, Google App engine, Amazon Web Service

Previously, Liang, Fei and I did a cloud evaluation project under the guidance of Professor Anna Liu.

Now seems our report is going to be release to be a publish....

Anna was interviewed by ITNews,and talked some of the result from our project.
Cnet also publish a short article about Anna`s talk.

Hopefully by the time our report is released, we will get some good feedback.

Friday, August 14, 2009

Poor .Net Service Bus Java SDk ...

In this article i am NOT going to show you how to do things with .Net Service Bus Java SDK.

But show you some of the facts i found out from my experiments.
(I used Metro to create service to connection to .Net Service Bus)

1) Java To Java Only support SOAP 1.2

I was struggling for some days, when i didn`t pay too much attention on the release notes,
and trying to create a demo, asking java subscriber to talk to java publisher.

Later my friend Liang found out that, Java SDK only support SOAP 1.2 protocal.

Coz by default, metro is using SOAP 1.1.
So when you program with metro, make sure u specify proper binding for your impl class

SOAP 1.2

2) Too much overheading

First let`s have a look how many packages would be sent if we use .Net technology to .Net technology

Start from the highlight.
Only four package will be sent for doing one round-trip conversation.

Now let have a look at the java to java (Be prepared and don`t be scared).
Yep..i am not joking, the whole page, starting from the top till the end of the pic,
it took so much to finished one round-trip conversation.

.Net Service Bus, another proposal between java and C# in REST approach

In this tutorial i will show you how to communicate between java techology and C# technology in REST Approach


A console C# application want to provide service onto internet, so register an endpoint in the .Net Service bus, which allow people making REST request.

After knowing there is a service in the .Net Service Bus, someone include such service into there java application with the help of HttpClient library.

C# service :
In this part, we are going to create a REST service provider by using WCF framework.

1) Create a normal C# console project from Visual Stuido, choose what even name you like.

2) Within the project, create a service contract.

3) Leave the contract blank first, coz we need to import some WCF reference.

4) Now we can implement our contact with WCF annotation.

5) After we have the contract, it is time to do the Implementation for the contract.

6) Bravo ... After all the boring jobs above, we now can create endpoint onto .Net Service Bus.
So here we include the Service Bus reference into our project first.

7) Create endpoint onto .Net Service Bus.
This time i will do something different. If you have read my previous articles, you would find that we normally need a App.config file, however it is not a must, we can do everything in programmatical way, just the matter which way u want.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.ServiceBus;
using System.ServiceModel.Web;
using System.ServiceModel.Description;
using System.ServiceModel;
using Microsoft.ServiceHosting.ServiceRuntime;

namespace ProposalService
class Program
static void Main(string[] args)
Console.WriteLine("Host starting ...");

//Console.Write("Your Solution Name: ");
string solutionName = "shrimpy";

//Console.Write("Your Solution Password: ");
string solutionPassword = "password";

// create the endpoint address in the solution's namespace
Uri address = ServiceBusEnvironment.CreateServiceUri(

// create the credentials object for the endpoint
TransportClientEndpointBehavior userNamePasswordServiceBusCredential =
new TransportClientEndpointBehavior();
userNamePasswordServiceBusCredential.CredentialType =
userNamePasswordServiceBusCredential.Credentials.UserName.UserName =
userNamePasswordServiceBusCredential.Credentials.UserName.Password =

WebServiceHost host = new WebServiceHost(typeof(ProposalContractImpl), address);

ContractDescription contractDescription =
ContractDescription.GetContract(typeof(ProposalContract), typeof(ProposalContractImpl));
ServiceEndpoint serviceEndPoint = new ServiceEndpoint(contractDescription);

serviceEndPoint.Address = new EndpointAddress(address);
serviceEndPoint.Binding = new WebHttpRelayBinding();


ServiceRegistrySettings settings = new ServiceRegistrySettings();
settings.DiscoveryMode = DiscoveryType.Public;


Console.WriteLine("Service address: " + address);



Testing our service.
Now the service is ready to go. Launch it, and go to your browser to test it.
Theoretically, you should see something like this:

Still remember how our C# contract look like???
I beg u must forget all about it...

[WebGet(UriTemplate = "/{words}")]
string says(string words);

In the contract we said that, anything follow by the link will tread as input of method "says"
So as in the pic, if we type something following the "proposal" should trigger method "says".

Let`s do it. See the location in my browser:

And then hit enter

Don`t be scared, this page is from Microsoft, asking for valid login. Type in our solution name and password, it will first ask the Access Control service to do a check up, whether we have the right to get into the service on the other side. If the solution name and password are all good, it will return us a security token. For browser, the token will put into cookie, so that we can continue to visit our service.

Bingo............ we get what we expected.......

Java RESTful subscriber
Now we have already half way to Rome. We just need to do a rest request from the java side.

1) Create a empty maven project. Choose any name u want.

2) Modify the POM, add HttpClient dependency, so that we can make REST request later.
3) It would be good to do logging when doing coding.
So create a folder "resources", and create log4j.properties file under the folder

Content of log4j.properties:

log4j.rootLogger=OFF, STDIO


log4j.appender.STDIO.layout.ConversionPattern=%14p [Cloudy Shrimpy] %30.30F:%L| %x %m%n

4) So good so far, now it is time to do some real coding.
Let think of what should we do first.

From the white paper, it said that, in REST approach, we first need to ask for security token from Access Control Service by providing solution name and password.
After we obtain the token, attache the token in http request header, then we can visit the REST service on the other side.

So very straight forward i will do things like this:

public static void main(String[] args) {
App app = new App();

* Get Authentication Token
String security_token = app.getAuthenticationToken();

* Send message
app.sendMessage(security_token, "Will you marry me");

Then we implement method "getAuthenticationToken" and "sendMessage"

private void sendMessage(String token, String words) {
// replace space with '%20'
String endpoint = String.format(SERVICE_URI, words.replaceAll(" ", "%20"));
log.debug("Endpont is : " + endpoint);

GetMethod get = new GetMethod(endpoint);
get.addRequestHeader(HEADER_KEY, token);
try {
int status = client.executeMethod(get);
log.debug("Request status is : " + status);

if (status == HttpStatus.SC_OK || status == HttpStatus.SC_ACCEPTED) {
byte[] responseBody = get.getResponseBody();
String responseContent = new String(responseBody);
log.debug(String.format("Response is : %s", responseContent));
} catch (Exception ex) {
log.error("Failed to send request to service is : " + endpoint, ex);

* https://accesscontrol.windows.net/issuetoken.aspx?u=SolutionName&p=SolutionPassword
public String getAuthenticationToken() {
String token = null;
try {
GetMethod get = new GetMethod(uri);
int status = client.executeMethod(get);
if (status == HttpStatus.SC_OK) {
byte[] responseBody = get.getResponseBody();
token = new String(responseBody);
log.debug(String.format("Token is : %s", token));
} catch (Exception ex) {
log.error("Failed to obtain authentication token.", ex);
return token;

You will see lots of upper case words in my code, just because i don`t want to hardcode string. So below are all the magic strings i used.

private static final Logger log = LoggerFactory.getLogger(App.class);
* Solution name and password
public static final String USERNAME = "shrimpy";
public static final String PASSWORD = "password";
* Link to get security token
public static final String ACCESS_CONTROL_LINK_TEMPLATE = "https://accesscontrol.windows.net/issuetoken.aspx?u=%s&p=%s";
* Target endpoint that message we are going to sent to
public static final String SERVICE_URI = "http://shrimpy.servicebus.windows.net/proposal/%s";
* Attribute that going to be add into the http header
public static final String HEADER_KEY = "X-MS-Identity-Token";
* Client that use for making REST request
private HttpClient client = new HttpClient();

Good now keep your C# service launching, and run your java application.

Do u get what i got?????

Saturday, August 8, 2009

Big Issue in .Net Service Bus When trying to communicate between JAVA and C# technology in SOAP approache

Last week i was thinking to create a tutorial to show people, how different technologies can interact by using .Net Service Bus.

However after last week`s experiment, seems that, .Net Service Bus hasn`t implement such functionality yet.

Ok..seeing is believing.....let me show you how i found out the fact.

My source code can be download from below.

Publisher In C# http://shrimpysprojects.googlecode.com/files/PublisherDemoInSoapV1.rar

Subscriber In Java http://shrimpysprojects.googlecode.com/files/DotNetServiceSubscriber.zip

Over View:

What i did for last week is that, i create a publisher service in C#, plugin into .Net Service Bus, then i create a java subscriber, and ask for service from the .Net Service Bus.

Part one, C# Publisher Service

In this service, i am going to use wsHttpRelayBinding.

Actually i had tried most of the bindings, basicHttpRelayBinding, webHttpRelayBinding and even netTcpHttpRelayBinding.

Q: why TcpHttpRelayBinding
A: 老板 also working on the same issue, try to make java communicate with C#, he try netTcp as well, i think the reason it that, when using java sdk, it only support sb as prefix.

The result turn out to be that,
with netTcp and wsHttp, i got http 500 internal error,
with basicHttp and webHttp i got http 400 bad request error.

Later in this artile i will show u the soap package as well.

Let start to set up the project first.

Step One:

Create a normal C# console application

Step Two:
Create interface for service contact and the impl of interface

Step Three, App.config

Final Step, Invoke service API, register the service onto .Net Service Bus

In the end, the whole project will look like this:

Now right click on the project, choose

Debug-->Start new instance

Theoretically you are supposed to get the app up and run

Part Two, create WSDL file

Base on the C# interface, render a set of WSDL file, so that later we can use it to create java subscriber.

I created a WCF project, base on the contact (interface) to create the WSDL, XSD files

When testing wshttpRelayBinding and netTcpRelayBinding
the WCF project use wsHttpBing to generate WSDL, XSD file

When testing basicHttpRelayBinding and webHttpRelayBinding
the WCF project use basicHttpBinding and webHttpBinding to generate WSDL, XSD file

Please refer to my source code .

The WCF project was create within the publisher project, it was a sub project.
Publisher In C# http://shrimpysprojects.googlecode.com/files/PublisherDemoInSoapV1.rar

if you want the WSDL file, go into DotNetServiceSubscriber.zip, under src/wsdl, you will find what u want.

Subscriber In Java http://shrimpysprojects.googlecode.com/files/DotNetServiceSubscriber.zip

PS: make sure you have fixed the endpoint and the url reference,

Cox i put all the wsdl and xsd file together in one folder,
when getting the file from WCF, in some file, you will see


make sure your change it to


also, change the endpoint to


Part Three, create java subscriber

Base on the WSDL file, create a subscriber to connect to .net service bus.

Step one, create a .net service bus java project

Please refer to my prevous article
to set up a java project first.

.Net service bus maven project set up with JavaSDK

Step two, add "wsimport" into POM

wsimport can easily parse WSDL and XSD file and generate us java code

Create a folder call "wsdl" under your src folder, and place all WSDL and XSD file into this folder.
Then edit your POM file as below

As you see from above, if you build the project, extra java code will place into your src folder,
these extra java code is the contact we can use with .Net service Bus Java SDK.

Final Step , create subscriber

Learnt from the sample of jdotnetservice.com, i create the subscriber as below

Now all the preparation have been done,
It is time to witness .Net Service cannot allow Cross techology communicate with SOAP approache.

Launch your publisher service,
then run the java subscriber..

and you will get error information like these:

With the error info above, we don`t know what stage we had up to, so i sniffer all the package,
you can see from the pic below, we actually pass all the authentication, and find the service on the bus.

However the bus failed to link the java end and C# end together.

Q : WHY i can say that???
A: Because my C# code hasn`t get invoke yet...all the error happened before getting into my C# code

Also from the package sniffer, we can see the fault message

Ok...let dig this problem one more step further..
In my C# project, i also create C# subscriber, it works perfectly. so i sniffer its package as well..

so now we can compare what are the difference between soap package sent out from the C# subscriber and java subscriber

SOAP package from C# subscriber:

SOAP package from Java subscriber when WSDL was generate from wsHttpRelayBinding:

SOAP package from Java subscriber when WSDL was generate from basicHttpRelayBinding or webHttpRelayBinding:

We can see that, the java SDK absolutely got some problem, otherwise, the chatroom will not always be there.

But this wouldn`t be the case that affect the communication, as we can see, in C# subscriber, it did not have a "from".

and when comparing these three file, we can see that, java subscriber use WSDL generate from basicHttp or webHttp binding , the soap package sent out would be almost the same as the C# subscriber..

But this raise a issue...

For what i experience, Java SDK for .Net Service Bus only support SOAP 1.2, (Another article will be come soon, taking about limitation of .Net Service Bus java sdk. which base on my prevouse prototype work, i created a java publisher and a java subscriber, and let them talk with each other)

So when creating C# publisher, we suppose to use wsHttpRelayBinding

However, from the SOAP package, there are big different betweent the C# subscriber and java subscriber which was using WSDL from wsHttp binding...

.Net service bus maven project set up with JavaSDK

I am a big fan of maven, this article will show u how to create a maven project for the .net service bus.

SourceCode : http://shrimpysprojects.googlecode.com/files/DotNetServiceBusSubscriber.zip


You have to go to this page:


make sure u have download everything they list, and configure you machine properly.

After you finishing everything listed in jdotnetservice, we can start now.

Step One: Create Empty Maven Jar Project

I am a Netbeans IDE user, coz it provide me friendly SVN and Maven support, but u can use what ever IDE you want, or even plain comman line to create a empty maven project.

I did not do any fancy stuff in here, all what i did is create a empty "Maven Quickstart Archetype" project.

And i name my project "DotNetServiceBusSubscriber". The name here is trivia, just because my following tutorial is going to illustrate how to use java application to connect to a C# application via .Net Service Bus, and my C# application is a publisher (coming soon ... :P ).

Step two: Change source version

By default, maven project is using java source version 1.3, we need to change them to a newer version, 1.5 or 1.6. It is up to you.

Right click on the project, choose Properties.

Select "Source" from the tree in the right hand side.
And change the source to 1.5 or 1.6.
Here i used 1.6.

Now go to your POM file, you will see someting like this:

PS: Hey how about i am not a NetBeans user, how can i generate this???

In this case, you have to manually type it into your POM file.....-_-!!!!

Step three: Dependency
I haven`t find any repository that provide ".Net Java SDK library" and all other jars that we need. So i decide to do all these by myself.

Speaking in the front: make sure u have download jdotnetservices-m3-ctp.zip , and metro. It will be better that u download jdotnetservices-m3-ctp_src.zip as well.

Edit your POM as show in the pic

Question: why some of the jar is specified as provided and some are 'compile'

Answer: coz some of the jar is in my classpath, so i said they are provided, but the others, i didn`t have them in my classpath, so set to compile.

NOW comes to the key point. All the dependency i specified here are fake, so we need to manually install all these jar.

In the pic showed how to install jar for JDotNetService, which is Java SDK for .Net service bus,
another three jar is from Metro. Do the same trick for them.

And do not forget install the source for JDotNetService. With source install, you can see the source code and java doc.

PS: Hey, how about i do not use Netbeans????

Maven - Guide to installing 3rd party JARs

This will help.

Final step: Do your coding

Now is time to create your own application ....have fun...

Monday, August 3, 2009

Peeking in .Net Service Bus "Router "

The pass two day, i played with the "Router" a little bit, and try to find out what the router can do for us.

The pic above was quoted from the .net service bus white paper. As we can see that, Router can distribute message to a queue, a receiver or another router.

Ok, time for criticism:


(a) Messages are directed in OneWay only.
Which means the relationship can only be One Sender --> Multi Receiver,
receiver can not interact with the sender.

(b) the queue need to be specify by the receiver.
I am a bit confusing in here as well.

<1> I cannot see big differents or important differents between the two receiver showed in the above pic.
I use the sample code which came alone with the .net service sdk (publisher subscriber).
I made the publisher keep sending message. When Service B and A both launched.
Messages displayed by Service B always random.
for example, the sender keep sending number in acceding order e.g 1 2 3 4 5 6 7 8 9 ,
bur what B received would be 4 6 1 8 9 2 3 5, and B keep doing like this.

The same thing happened on Service A as well, at the beginning, there is no order...
but after a little while, we can see the message displayed by Service A would become the same order as what the sender did.

but it doesn`t make sense to me, why at the beginning, Service A display stuff is out of order???

<2> What more, the queue in the pic, didn`t act as what i expected.
I was thinking, even though service A might down for a while, when the next time it come up again, it wouldn`t miss any message, coz the message will store in the queue.
However, the result turn out is that, if Service A is down, the next time u come up, u can only receive the latest message, if something were sent in down time of service A, service A will just miss them.

(c) We can only play with the Router in .Net Technology.
Up till now, i have looked into both Java SDK in Jdotnetservice.com and Ruby SDK dotnetservicesruby.com

Seems neither of them support router.
And i tried to look into the REST approach of how the .Net technology can do the router thing,
seems it is not enough information for us to do everything on our own.

Maybe Microsoft should release a set of document, to show us how to connect to all its API.
To be honest, the java sdk is so shit, and too many over head request.

(a) Sender can be down and up.
No matter what, a sender need to create a router into the service bus first.
After this, receiver can subscribe to the router.
And then sender would be allow down, up what ever time it wants, as long as it come back before the time out dration, which is 1 hours by default, and can be manually specify by the sender when creating the router.

(b) Receiver can ask the sender attache a secret key with all the message which is going to send to this receiver.
When the receiver subscribe to the router, the receiver can push a key to the router, ask the router attache that key with all message which will be send to this receiver.

(c) Load balance
Router can be configured to send message to "All" or send message to "One".
The usage of this kind of setting is that.

To "ALL" which means all the subscriber will receive the message.

To "One", if all the subscriber are the same set of service, either one of the service get the message from the router will do the job. In this case, the router will do the load balance job for you. The router will pick the most "Free" one, and send the message to that subscriber.