Update: The expando-mongodb-hook plugin is now committed to SVN and available from trunk.

Over the past few months the hype around NoSQL type databases had really been heating up the tech news and blog feeds. There seems to be an overwhelming desire to find scallability solutions that don't seem to be addressed with an RDBMS. What is Liferay to do?

Could Liferay support some form of NoSQL integration? I think so, and I surely couldn't go long without doing something to draw attention to the fact that Liferay is a prime candidate as a viable platform for scalling dynamically via a NoSQL backend.

The most obvious way I could see to leverage a NoSQL solution was with perhaps the most dynamic aspect of the portal, Expandos (and by association Custom Fields).

In order prove the concept of NoSQL with Liferay we decided to write an adapter (using a Liferay Hook pattern) to build a backend for Expando on MongoDB. I had no real idea how long it would take to accomplish but we decided to try. As it turns out it was not too difficult. We now have a fully functional adapter to store all of Liferay's dynamic Expando data in a highly scalable MongoDB. But note that Expandos still support all Liferay permissions. And Custom Fields are still indexed along with the entities anywhere they would be normally. This is a fantastic demonstration of just how extensible Liferay portal really is.

I tested against the version of mongodb that was readily available for Ubuntu 10.04 (1:1.2.2-1ubuntu1.1). I also tried to make sure to support cluster configurations. So check out the portlet.properties file in the plugin as well as the mongodb driver javadocs for what and how to set that up.

I did several small usage tests (none of which were load testing, since this was an informal design) to see that everything was working the right way. I created several Custom Fields on several different entites and tested CRUD opperations to make sure that the data was landing (as well as being removed/updated) where I wanted it, in MongoDB.

Meanwhile, I was also using the Mongo DB command line client mongo to make sure that everything was working from that end. I added a custom field called test to the User entity, and for the first user in the system, I set the value to test value . Here is an example of what we see via mongo:

[rotty@rotty-desktop  expando-mongodb-hook]$ mongo
MongoDB shell version: 1.2.2
url: test
connecting to: test
type "exit" to exit
type "help" for help
> show dbs
admin
local
lportal_0
lportal_10135
> use lportal_10135
switched to db lportal_10135
> db.getCollectionNames()
[
	"com.liferay.portal.model.User#CUSTOM_FIELDS",
	"com.liferay.portlet.blogs.model.BlogsEntry#CUSTOM_FIELDS",
	"com.liferay.portlet.documentlibrary.model.DLFileEntry#CUSTOM_FIELDS",
	"system.indexes"
]
> db.getCollection("com.liferay.portal.model.User#CUSTOM_FIELDS").count()
1
> db.getCollection("com.liferay.portal.model.User#CUSTOM_FIELDS").find()
{ "_id" : ObjectId("4d28f318fcfcc08a7855ebe4"), "companyId" : 10135, "tableId" : 17205, "rowId" : 10173, "classNameId" : 10048, "classPK" : 10173, "valueId" : 17207, "test" : "test value" }
> 

So far so good! As you can see the data is landing nicely into the Mongo DB database.

While that was a good test I also wanted to make sure that other use cases would work just as well. I decided to revive the First Expando Bank example to see how that would work.

I first had to make a few small API changes in the Velocity template. The updated template is attached. See this article and the follow up for more information on that topic.

After adding some accounts into the First Expando Bank app, the mongo console results looked like this:

> db.getCollectionNames()
[
	"AccountsTable#AccountsTable",
	"com.liferay.portal.model.User#CUSTOM_FIELDS",
	"com.liferay.portlet.blogs.model.BlogsEntry#CUSTOM_FIELDS",
	"com.liferay.portlet.documentlibrary.model.DLFileEntry#CUSTOM_FIELDS",
	"system.indexes"
]
> db.getCollection("AccountsTable#AccountsTable").count()
3
> db.getCollection("AccountsTable#AccountsTable").find()
{ "_id" : ObjectId("4d29292abda2c08a05e35e67"), "companyId" : 10135, "tableId" : 17320, "rowId" : 1294543146642, "classNameId" : 17313, "classPK" : 1294543146642, "valueId" : 17336, "balance" : 55, "firstName" : "Ray", "lastName" : "Auge", "modifiedDate" : "Sat Jan 08 2011 22:19:06 GMT-0500 (EST)" }
{ "_id" : ObjectId("4d292945bda2c08a06e35e67"), "companyId" : 10135, "tableId" : 17320, "rowId" : 1294543173086, "classNameId" : 17313, "classPK" : 1294543173086, "valueId" : 17337, "balance" : 120, "firstName" : "Daffy", "lastName" : "Duck", "modifiedDate" : "Sat Jan 08 2011 22:19:33 GMT-0500 (EST)" }
{ "_id" : ObjectId("4d292958bda2c08a07e35e67"), "companyId" : 10135, "tableId" : 17320, "rowId" : 1294543192848, "classNameId" : 17313, "classPK" : 1294543192848, "valueId" : 17338, "balance" : 300, "firstName" : "Mickey", "lastName" : "Mouse", "modifiedDate" : "Sat Jan 08 2011 22:19:52 GMT-0500 (EST)" }
> 

Excelent! It would appear that all our use cases are covered from automatic Custom Fields via the UI to programmatic use in a CMS template.

I'd love to get your feedback about it! Please note that there is currently no rich way to perform queries (à la MongoDB). But with a little enginuity we could probably make that possible.