Tomcat & IntelliJ – Deploy war files outside webapps folder

At present I am working on developing an Android application that needs to be supported by a slew of REST services hosted in the cloud. I chose Google App Engine based on its support for Java, Groovy and most importantly Spring. I developed a Spring MVC based REST application and used ContentNegotiatingViewResolver to negotiate content based on request URL extensions. For example, an XML response will be returned if the request URL ends with .xml, a JSON response for .json and an HTML response if he URL doesn’t have any extension. Don’t get me started on Accept Header versus URL extension based content negotiation. That is a rant for another day.

I was attempting to Serialize a Map<Enum, List<Model>>. All was well and I was able to retrieve both HTML and JSON representations, but when I tested retrieving the XML representation, JAXB complained that it cannot handle a Map instance in the root although Jackson was totally cool about it. As usual, Googling revealed that JAXB expected a Container class at its root which I didn’t want to create. I didn’t want to give up either. So, I tried my luck using XStreamMarshaller. This time GAE complained that XStream used a restricted API. WTH?

Just out of curiosity, I wanted to check if XStreamMarshaller would work as expected when used outside of GAE. So, I created a Tomcat context file “myapp.xml” with the following definition and carefully placed it inside TOMCAT_HOME/conf/Catalina/localhost. I could have just started Tomcat from TOMCAT_HOME/bin/startup.bat to check if it works, but being an IDEA addict, I created a Run Configuration for the IDEA Tomcat plugin and started the server from inside IDEA. But the app refused to even be discovered, let alone be deployed. After a few frustrated attempts, I tried starting Tomcat directly outside IDEA. Thankfully the app got deployed successfully and to my surprise, the XStreamMarshaller skillfully streamed out the serialized XML. Problem Solved!

<?xml version='1.0' encoding='utf-8'?>
<Context docBase="PATH_TO_MY_APP"

But, why didn’t the app get deployed when I started Tomcat from inside IDEA? After all, I have linked IDEA to my local Tomcat installation and the script it executes is clearly in my TOMCAT_HOME/bin folder. Then, why why why in the world does the app refuse to be discovered? The solution came in the form of CATALINA_BASE. It seems that IDEA copies the contents of TOMCAT_HOME/conf folder into its HOME folder with some name like Unnamed_MyApp and sets this folder to be the CATALINA_BASE. That explains why “myapp.xml” is so totally ignored by Tomcat. Then, I navigated to “Tomcat Run Configuration -> Startup/Connection -> Environment Variables” and added CATALINA_BASE as an environment variable and pointed it to your local TOMCAT_HOME folder. After this configuration change IDEA started Tomcat as expected and my app was both discovered and deployed. Another Problem Solved!

But the real problem – JAXB complaining about Map and GAE rejecting XStreamMarshaller as restricted – is yet to be solved. Maybe I should try one of the CastorMarshaller, XmlBeansMarshaller or JibxMarshaller.

Any ideas?

No in Hadoop 0.23

The first thing to do after downloading and extracting Hadoop is to set JAVA_HOME in the $HADOOP_HOME/conf/ file. Almost all documentation on Hadoop site expects the above configuration but I guess that was for version 0.20. When I downloaded Hadoop 0.23 today and looked for the file, I could not find it anywhere in the extracted archive.

After a bit of Googling I realized that things have changed in 0.23 version and now I will have to look for the file in the $HADOOP_HOME/etc/hadoop directory instead. Unfortunately I couldn’t find the file there either.

As a last resort I created a file named in the $HADOOP_HOME/etc/hadoop directory and set my JAVA_HOME there. The one and only line in that file is given below. Fortunately after that addition, the hadoop command started working and everything went normal again.

export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/

Heroku – Trouble with Windows and SSH Keys

Heroku, one of my favorite cloud platforms, is a hosted platform built specifically for deploying Rails and other Ruby based web applications. Heroku makes deploying Rails applications to cloud ridiculously easy – as long as your source code is under version control with Git.

The general process of installing and using Heroku on Windows is fairly simple. You should already have Ruby, Rails and Git for Windows (msysgit) installed. Once you have them, go to and sign up for an account. After signing up, install the Heroku gem with the following command.

$ gem install heroku

Typically, as in GitHub you will need to create SSH keys if you haven’t already, and then tell Heroku about your public key so that you can use Git to push your application repository up to their servers. Detailed information for setting up Git and creating ssh keys on Windows can be found here.

# Create new keys
$ ssh-keygen -t rsa -C ""

# Tell heroku about your public key
$ heroku keys:add

Remember? Your source code should be under version control with git. So, from within the root of your application folder, use typical git gimmicks to create a local git repository and commit your application source to the local repository.

$ cd my_app
$ git init
$ git add .
$ git commit -m "initial version"

Finally, from within the root of your application folder, use the heroku command to create a place on the Heroku servers for the application to live. Then, you can deploy your application to the cloud by pushing your local git repo to heroku…

# Create an application space on the server
$ heroku create

# Deploy your application to the cloud
$ git push heroku master

If until now everything went well except for the last step and instead of getting your application deployed if you got the following error, then it was worth your time to have read this post until this point…

Permission denied (publickey).
fatal: The remote end hung up unexpectedly

It was a frustrating experience trying to figure out the problem. The keys were created fine and they got added to heroku just fine as well. In fact i could verify it with the following command and my keys were right there.

heroku keys:list

Whatever I do, I kept getting the same error. Finally after a lot of googling, experimenting and hair plucking the problem got resolved. Here goes the solution…

Typically once you create the keys as mentioned above, two files – “id_rsa” and “” – are stored in the “.ssh” folder within the user’s home folder. If you are working with linux that seems to be good enough. But for the windows version of git that doesn’t seem to cut it. It wants the keys to be stored inside the “.ssh” folder within the “msysgit” installation folder as well. If you don’t find a “.ssh” folder inside the “msysgit” installation folder, feel free to create one. Once you drop these two key files there and repeat the entire process, everything went as smooth and my application got deployed in the heroku cloud and the world is again a better place to live in.

I am sure many people will hit the same road block. At least I am sure I will come across it again and wouldn’t remember how I solved it in the first place. Happens, doesn’t it? So, I thought i will document it here for future reference.

If there is any other solution, or if this solution is incorrect, feel free to leave a comment.

Ceylon – Yet Another JVM Language (YAJL)

It is all too familiar, yet surprising; it is all too common, yet shocking; to see yet another JVM language created to scratch an itch that countless other languages are already trying to solve.

Of course Java is not expressive enough, it doesn’t have higher order functions, it doesn’t have modularity as a language feature, it doesn’t have clean way to do meta-programming, it does NOT have so many more features we love and does have so many more features we hate. These are most of what has frustrated Gavin King (the creator of Hibernate) as well and made him think about creating a new language – Ceylon. A few days ago, in his presentation at the QCon Beijing 2011 he gave a first glimpse of the language features and a few code snippets showing its beauty.

A lot of people have raised concerns and expressed strong opinions questioning a need for yet another language. Scala fans, the Groovy(++) camp, Gosu and Fantom hackers all think that Ceylon does not solve anything that is not solved already or could have been solved by just contributing to one of these modern languages. So, I am not going to be yet another anti Ceylon person but I am not a fan either.

I believe Ceylon is more of a strategic approach from RedHat than a language created out of true necessity. They would like to control a language and its followers like most other giant companies do today. Think about it. Oracle is controlling Java, Microsoft is controlling C# (VB, VC++, etc), Google is controlling Go (and Python?), Apple is controlling Objective C and VMWare is controlling Groovy. RedHat has just joined the party leaving only IBM out of the equation. I am sure they are not far behind. I just sincerely hope that they adopt an existing language (Scala?) instead of creating yet another one.

That said, the language itself looks cool, is very expressive and adds a lot of syntactic sweetness to say the least. I just wanted to highlight a few of them here…

String Interpolation:
In Java we use a “+” sign for string concatenation. There is no concept of string interpolation in either Java or Scala. In Groovy ${} construct is used while Ruby uses #{}. Ceylon’s syntax looks better than either of them though. It uses a “space” as the string interpolation operator and the result looks a lot cleaner. Don’t you think?

String name;
writeLine("Hello " name "!");

In Java we use getXxx() methods to get a property value. This looks like a method and quacks like a method and does not give the feeling of accessing a property at all. In Ceylon, a very simple innovation has resulted in much more succinct getter methods. They have decided to get rid of paranthesis to both define and call getter methods. Look at the last line of the following snippet. Does it look like a method call? NO. Does it look like accessing a property? Of course it does !!!

class Counter() {
   variable Natural count := 0;
   shared Natural currentCount {
      return count;
Counter c = new Counter();

In Ceylon there is no separate constructor. The class definition itself acts as a constructor as shown below. Since, there is no concept of method or operator overloading in Ceylon there is no necessity to have a separate method and this syntax induces a “Why didn’t I think of this before ?” moment… The necessity for an overloaded method is handled by optional /defaulted parameters concept. So, you don’t have to worry too much about it…

class Customer (String cName, Natural cAge, Date cDob) {
   variable String name :=  cName;

With the support of named parameters and higher-order functions and quite a bit of thought, a syntactic structure as expressive as given below has been achieved in a general purpose and statically typed language like Ceylon.

Html hello {
   Head head { title = "Squares"; }
   Body body {
      Div {
         cssClass = "greeting";
         "Hello" name "!";

And then there is the rest of the now so common features like array like access to Sequences (equivalent of List in Java), higher-order functions, Closures, Currying, etc

It is not like I loved every single feature of Ceylon. I hated a few, but I am reserving that rant for another post…

Twitter moves from Rails to Java

A colorful feather up Rails’ cap is on the ground now. Twitter has decided to go away from RoR in favor of Java, this time for their entire search stack. Earlier in 2008-09, they decided to move their message queue back-end from ruby to Scala (a Java Platform) and now it is the time for their front-end to move to Java as well.

They have built a scalable platform called Blender that uses Java NIO based server (Netty) to be efficient in the face of heavy incoming traffic, replaced MySQL with a Java based Lucene search engine, created an engine that parallelizes execution of multiple backend services with dependency management and more. With this setup there is a 3X drop in search latencies and can scale to 10X more requests per machine.

Wow, that is quite an achievement. Could this mean that Java is a better platform than Rails for high scalability needs? Even if that is the case, for simpler scenarios, the beauty of RoR out-weighs Java’s performance.

They say that this change will enable them to rapidly iterate on search features in the coming months. That along with the news that Twitter has hired 25 more employees kinda tells that Java’s code base is practically more maintainable than equivalent Ruby code – at least when the code base is huge and the team size is large. Or that could mean that this time they really put a lot of thought into designing a maintainable system than when they started out. But for smaller team size and code base, RoR is still an unbeaten champion.

Tomcat 5.5 and above doesn’t need the JDK

Most of us who install Tomcat on our development machines wouldn’t have noticed that Tomcat versions before 5.5 required the full blown Java Development Kit (JDK) to be installed on the machine. Just installing the Java Runtime Environment (JRE) wasn’t enough. The JDK is meant for developers to be able to compile Java programs, and has the development tools such as the compiler, debugger and other development libraries. Tomcat versions prior to 5.5 used the Java compiler that comes with the JDK to compile JSP pages at runtime and consequently required a full blown JDK.

Tomcat versions 5.5 and above have a Java compiler (the Eclipse JDT Java compiler) packaged along with it. Now, this is used to compile the JSP pages dynamically instead of the JDK compiler. Hence, it is enough if we just install JRE starting from Tomcat version 5.5 and above.

Hibernate never stops surprising me

I have a simple form in my web application where a user can fill in his personal details and address details. User specific fields include firstName, middleName and lastName while Address specific fields include street, city and zip. On the server side, I have POJOs for User and Address. Finally I use Hibernate to map these POJOs to the database. Since, the Address will not be used outside the context of the User, I decided to map it as a Component of the User class.

The User class and its corresponding mapping is given below:

@Entity @Table(name = "user")
public class User {

     @Column(name = "first_name")
     private String firstName;
     @Column(name = "middle_name")
     private String middleName;
     @Column(name = "last_name")
     private String lastName;
     private Address homeAddress;

     ... Getters and Setters

The Address class is mapped to User as a component. You can see the specifics of the mapping below.

public class Address {
     @Column(name = "address_street")
     private String street;
     @Column(name = "address_city")
     private String city;
     @Column(name = "address_state")
     private String state;

     ... Getters and Setters

As you may already know, mapping components using hibernate is a very useful feature. This feature and support for nested components and components referring to other entities are the primary source of support for rich and fine grained domain model in hibernate. But while i was using Component mapping, I recently stepped on an interesting Feature (or it could be an Issue) and I was happily surprised by it.

Want to know the surprise? Keep reading…

In this case you map an Address as a value type using @Embedded annotation in the “homeAddress” field of the User class. The Address class itself is declared to be @Emeddable. This is the standard Hibernate/JPA way to map value types. The Address class has street, city and zip and it gets stored into the same table as the User class’s table. Now, when you insert an instance of User into the database, while specifying all null values for Address‘s fields maybe because the user didn’t give his address, then what would you expect in return at some point in time when you retrieve this User back from the database.

I for one expected user.getAddress() will return an Address instance. Then address.getStreet() will return null. But that is not what happened. user.getAddress() returned null by itself. That was interesting and even helped me in my case because, if the user hasn’t given any details for his home address, then it probably means that his address itself is not there in the system. So, returning null for getAddress() is semantically the right thing to do. I was surprised and when i checked the hibernate documentation it was even mentioned there that if all properties of a component are null, then the component itself is considered null.

In another situation this could have been bad, I don’t know, but for my purpose I was happily surprised with this nice touch from hibernate. These kind of small things is what differentiates a great product from a good product. Ain’t it?

Final Modifier for method arguments. What do you think?

The IT industry today is sodden with TLAs like SOA, ESB… and FLAs like AJAX, SOAP and JUNK. i was thinking about refreshing myself with some fundamentals again. Blogging about a basic concept may not be cool, but refreshing – don’t you think? I know what you are thinking. You are thinking that i am digressing too much. Ok, lets cut to the chase.

One of the best practices i follow religiously is to use final modifiers for method arguments where applicable. This is “supposedly” a best-practice written by somebody somewhere. Regardless of whether it is documented as a best-practice or not it is an important concept to understand and use. I have 2 valid reasons to use them for my method arguments.

First, final variables cannot be modified. Come on, everybody knows that. Maybe, but its use is significantly enhanced when it is a method argument and more importantly when you are in a big team environment.

Lets assume that a method takes List as its argument. Typically, the intention of that method is to work with the List – add to it, remove elements from it, use its elements in some way, sort it and what not. Consequently when the method returns, the caller can investigate the passed List and work with the modifications the callee introduced. But the caller will be on for a big surprise if the callee changes the instance that reference points to itself.

We know that java uses “Pass by copy of reference”. If the callee points the received reference to a different List and then modifies this List, the caller will not be able to see any change at all. This is because the copy of the reference held by the caller still points to the same old List. More often than not this is done by mistake and is not intentional. If such a behavior is intentional, final modifier is not required. In all other cases since this leads to bugs in code, it is a good practice to use final modifier for method arguments.

Second, if the method uses the infamous anonymous inner-class syntax to do something, and that inner class wants to use the methods arguments, java requires those arguments to be declared final. This is more of a rule than a valid reason.

Are there more valid reasons? I will be glad to receive information from you guys.

Hibernate: Why should I Force Discriminator?

Hibernate is an ambitious project that aims to be a complete solution to the problem of managing persistent data in java. Even with such an arduous task before them, the hibernate team tries very hard to expose a simple API for developers like us. Still, the complexity behind the API shows its ugly face time and again and I believe it is unavoidable as long as the mismatch between Object and Relational world exists.

That said, although I have worked with hibernate for many years and have been its advocate in all my organizations, I keep facing newer issues and keep finding newer ways to work with it efficiently and effectively. Recently, when I was working for, I faced an issue when mapping a OneToMany relationship to the sub-classes of “Single Table Inheritance” strategy. After a frustrating couple of hours of debugging I finally landed on the correct solution. So, I thought other developers who will travel this path could get benefited and started writing this blog post.

Let me explain the issue I faced with an example. Assume you have a normal User Entity with the typical id, version and loginId properties. Assume this User can have many AboutUs sections and many Service sections. You don’t need to be an architect to model them as OneToMany relationships from User. So, I modelled UserAboutSection and UserServiceSection entities and created a OneToMany relationship between User and these entities. Looking at the commonality between these two, I decided to factor out the common fields into a superclass called UserSection. Now, both UserAboutSection and UserServiceSection extends UserSection. I chose to map this Inheritance hierarchy using “Single Table Inheritance” strategy to keep it simple and since most of the fields were common and only a few were specific.

The User entity is given below. Notice the List<UserAboutSection> and a List<UserServiceSection> mapped using OneToMany relationship.

The getters, setters, adders, imports and static imports are omitted for brevity.

@Entity @Table(name = "user")
public class User extends BaseEntity {

    @Column(name = "login_id")
    private String loginId;

    @Column(name = "password")
    private String password;


    @OneToMany(cascade = {CascadeType.ALL})
    @JoinColumn(name = "user_id", nullable = false)
    @IndexColumn(name = "user_section_position")
    List<UserAboutSection> aboutUs;

    @OneToMany(cascade = {CascadeType.ALL})
    @JoinColumn(name = "user_id", nullable = false)
    @IndexColumn(name = "user_section_position")
    List<UserServiceSection> services;

    ... Getters, Setters and Adders

Here goes the UserSection entity that acts as the base class in this “Single Table Inheritance” strategy. Hibernate uses the @DiscriminatorColumn annotation to distinguish between sub-classes.

@Entity @Table(name = "user_section")
@Inheritance(strategy= InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(name="user_section_type", discriminatorType = STRING) 
public class UserSection extends BaseEntity {
    @Column(name = "title")         
    protected String title;

    @Column(name = "description")   
    protected String description;

    ... Getters and Setters

Here goes the UserAboutSection entity that derives from the UserSection entity. Hibernate uses the @DiscriminatorValue annotation to decide if a row in the database belongs to an instance of this class.

@Entity @DiscriminatorValue("ABOUT")
public class UserAboutSection extends UserSection {
    protected User user;

    ... Other Properties specific to UserAboutSection

Here goes the UserServiceSection entity that derives from the UserSection entity. Hibernate uses the @DiscriminatorValue annotation to decide if a row in the database belongs to an instance of this class.

@Entity @DiscriminatorValue("SERVICE")
public class UserServiceSection extends UserSection {
    protected User user;

    ... Other Properties specific to UserServiceSection

Pretty straightforward… huh! When you try to retrieve an instance of User along with its aboutUs and services collections eagerly (or lazily – doesn’t matter), what do you expect?

I expected an instance of User with the aboutUs collection filled with only UserAboutSection instances and the services collection filled with only UserServiceSection instances corresponding to only the rows they represent in the database. And I believe this expectation is valid, because that is what the mapping looks like and hibernate also has all the information it needs to make this work.

But I got something different. Both the aboutUs and services collections had all the UserSection rows that belong to this User. I mean, aboutUs collection had all the UserSection instances including UserAboutSection and UserServiceSection instances. This was surprising because hibernate has all the information it needs to populate the right instances.

After quite a bit of debugging, googling and RTFM-ing I landed upon @ForceDiscriminator annotation. This annotation has to be applied to the base class in the Inheritance hierarchy for “Single Table Inheritance” strategy. In my case, I had to apply it to UserSection entity. The UserSection entity after applying this annotation is given below…

@Entity @Table(name = "user_section")
@Inheritance(strategy= InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(name="user_section_type", discriminatorType = STRING) 
public class UserSection extends BaseEntity {
    @Column(name = "title")         
    protected String title;

    @Column(name = "description")   
    protected String description;

    ... Getters and Setters

Once I ask hibernate to Force Discriminiator, it is happy and populates the aboutUs and services collections with its respective instances.

Ok, Problem Solved! But why did I have to tell hibernate to Force Discriminator. Shouldn’t that be the default behaviour. Is it a bug in hibernate or is it a feature? Am I missing something? If any one of you hibernate fans have walked this path and know the answer, please feel free to drop in a comment. I sincerely hope this post will be a valuable time-saver for other hibernate developers who step on this Bug/Feature.

Is it an AJAX Request or a Normal Request?

In modern web application development, many a times you would want to know if the incoming HTTP Request is an AJAX request or just a Normal request. Have you come across this requirement? I have, and the solution that I found turned out to be pretty straight-forward and I will be sharing it with you here.

Whenever an AJAX request is sent to the server, a special header named X-Requested-With with a value of XMLHttpRequest is attached to the request. So, a simple check to see whether the X-Requested-With header has a value of XMLHttpRequest solves the challenge. An example will give you more clarity. Take a look at the examples of isAjax() methods in two famous server-side programming languages – Java and PHP.


public static boolean isAjax(request) {
   return "XMLHttpRequest"


function isAjax() {
   return (isset($_SERVER['HTTP_X_REQUESTED_WITH']) 
            ($_SERVER['HTTP_X_REQUESTED_WITH'] == 'XMLHttpRequest'));

From the code above, it should be clear that if the method returns true, then you received an AJAX request. If the method returns false, you received a Normal HTTP Request.

Now you know how to find out if it is an AJAX request or not. But how is it useful? What do you do with it?

Have you heard of the term Unobtrusive javascript. Come on, It is a buzzword these days. There is a lot to writing Unobtrusive javascript code, but one of its features – Progressive Enhancement – stands out from the rest. Again, let me explain this with an example.

Let us assume that you have designed a beautiful looking web page with tabbed navigation. Being a hard-core AJAX fan and an efficiency aficionado you don’t like to bring back a full page with the headers and the footers and the sidebars when your user clicks on a Tab. Instead you would love to bring back only the content area for the tab via AJAX and innerHTML it into its respective place. Correct? The good programmer inside you has also told that you should not hard-code the URL for the AJAX call in your javascript. Instead, your tab should be designed as an anchor whose href attribute points to the URL for the content. So far so good? Now, when the user clicks on the Tab, you collect the URL from its href attribute and fire an AJAX call to the server. The server obediently accepts the request and sends back the content just for the active tab.

   <li><a href="/" rel="#tab1">Tab 1</a></li>
   <li><a href="/" rel="#tab2">Tab 2</a></li>
   <li><a href="/" rel="#tab3">Tab 3</a></li>

   <div id="tab1">Content for Tab1</div>
   <div id="tab2"> Content for Tab2 </div>
   <div id="tab3"> Content for Tab3 </div>

You would like to believe that all is well and good… Unfortuntately this is where the challenge of Progressive Enhancement begins. How do you plan to support browsers with JavaScript disabled. Yes, there are quite a few people who still do that for the fear of script related worms. How do you plan to support automated bots? Don’t you want the Google bots and the Yahoo bots to index all your pages with full content? Do you know what these users will see? They will see a half-baked page with just the active tab’s content and no styles – not even the header, footer or the sidebar. I am sure you don’t want this to happen. That is when the isAjax() method comes useful.

Whatever you did on the client side – HTML, CSS and javascript – is great and perfect. You separated the behavior from style and markup. So, you don’t have to touch it. You just need to do a small bit of work on the server-side.

In our case, when JavaScript is disabled, the browser will take over and execute its default behaviour of firing a Normal request to the URL specified in the href attribute. When JavaScript is enabled, JavaScript will take over and will fire an AJAX request to the same URL. In both cases the same server-side resource (like in Java or PHP) will receive the request. Now that you have decided to support Progressive Enhancement, you will have to check if the incoming request is an AJAX request or a Normal request using the isAjax() method shown earlier. In case of an AJAX request, you will send back only the content area of the tab thereby providing an efficient version of your application to JavaScript enabled users. In case of a Normal request, you will send back an entire page including the tab’s content, thereby providing a less efficient yet fully functional version of your application to JavaScript disabled users.

// Pseudo Code
if(isAjax(request) {
   return "Content area for the tab";
} else {
   return "Full page including tab content";

Now you can feel proud that you are not one of those who just talk about Unobtrusive JavaScript but you are one of those who have implemented it. I sincerely hope this article was useful. Feel free to drop in a comment with suggestions and/or feedback.

Update 1:

Based on the input provided by a buddy called Sabob from dzone, X-Requested-With is set only by AJAX libraries like jQuery, Mootools, Prototype etc. If you are coding AJAX by hand, then you will have to explicitly set this header while sending a request. More importantly you should somehow abstract it using good OO principles so that you don’t have to explicitly code this line every time you send an AJAX request.