Introduction

• This blog post is to introduce a full end to end proof of content (POC) to integrate Liferay DXP with RabbitMQ for asynchronous message processing using custom services and Client Extensions only, without any custom OSGi modules.
• RabbitMQ runs as a custom service in the Liferay PaaS environment.
• An Object Action Client Extension publishes a message to a RabbitMQ queue using Spring AMQP.
• A 'remote' Spring Boot custom service listens for messages in the queue using Spring AMQP, processes the message and updates the original Object Record.
• For simplicity, the message payload used is the Liferay Object record JSON payload that is passed to the Object Action Client Extension.

Liferay Deployment Approaches

Although the POC focused in Liferay PaaS, the solution doesn't contain any custom OSGi modules, meaning it can also be deployed in Liferay SaaS (or Liferay Self Hosted with additional configuration):

• In Liferay PaaS, the rabbitmq and rabbitmqlistener custom services and the rabbitmqpublish client extension are all deployed as custom services to the environment e.g. prd.
• In Liferay SaaS, the rabbitmq and rabbitmqlistener custom services and the rabbitmqpublish client extension are all deployed as custom services to the ext environment e.g. extprd.

Repositories

The POC uses the following github repositories:

• https://github.com/michael-wall/rabbit-mq-demo.git which contains the DXP Cloud Workspace rabbitmq custom service definition, the rabbitmqlistener custom service definition (excluding the compiled jar file) and the rabbit-mq-publish client extension.
• https://github.com/michael-wall/rabbit-mq-demo-listener.git which contains the source code for the rabbitmqlistener jar file. The compiled jar file to be manually added to the rabbitmqlistener custom service.

The Code

All of the custom code is contained in the following 2 class:

• RabbitMQPublishObjectActionRestController.java is an Object Action Client Extension endpoint that sends the message to the 'demo-queue' using the Spring AMQP RabbitTemplate helper class.
• RabbitMQListener.java is a regular java class that also uses Spring AMQP. It contains a RabbitListener that listens for messages on the 'demo-queue', reads the message, extracts the 'id' and 'input' values and uses these to populate the 'output' value of the original Liferay Objects record using the headless REST API PATCH endpoint and the Headless Server OAuth 2 profile. It then uses the RabbitTemplate helper class to send the message to the 'processed-queue' or the 'error-queue' if applicable.

Setup Steps

See the 'Detailed Setup Steps' section of the README for the detailed setup steps, the high level setup steps are as follows:

• Setup Liferay PaaS secrets for RabbitMQ credentials
• Deploy the RabbitMQ custom service and configure RabbitMQ
• Create the Liferay Object
• Deploy the Object Action Client Extension
• Setup Liferay PaaS secrets for rabbitmqlistener
• Build and deploy rabbitmqlistener
• Add the Object Action to the 'RabbitTest' Object
• Verify custom services status

Hostnames in Liferay PaaS

• Each Liferay PaaS environment has it's own private network, meaning the LCP.json service id value can be used by a service to reference another service in the same environment.
• In this POC:
  • rabbitmqlistener\LCP.json and rabbit-mq-publish\LCP.json SPRING_RABBITMQ_HOST environment variable is set to rabbitmq as that's the service id specified in rabbitmq\LCP.json
  • rabbitmqlistener\LCP.json LIFERAY_BASE_URL environment variable is set to http://liferay:8080 as this is accessible from within the private network.

RabbitMQ Custom Service

• The rabbitmq/LCP.json in the repository is pre-configured.
• It is a StatefulSet service with a volume defined for /var/lib/rabbitmq to retain the RabbitMQ setup after a restart and uses the default 'RollingUpdate' deployment strategy which should avoid downtime during build deployments.
• Port 5672 is configured to be internal whereas port 15672 is configured to be external.
• The Spring AMQP APIs use port 5672 to interact with the RabbitMQ queues. Make this port public if the publisher or listener isn't inside the Liferay PaaS environment private network.
• The RabbitMQ administration GUI can be accessed from a browser using HTTPS with the credentials from rabbit-mq-default-user and rabbit-mq-default-pass secrets using the hostname from rabbitmq service > ingress endpoints:

• You can check the status of the queues in RabbitMQ:

• Run this command from the RabbitMQ service shell to see the current queue message counts:
  • rabbitmqctl list_queues

• Run this command from the RabbitMQ service shell to view the first message from the processed-queue (without 'consuming' it):
  • rabbitmqadmin --username= [rabbit-mq-default-user]  --password= [rabbit-mq-default-pass]  get queue=processed-queue count=1
    • Replace  [rabbit-mq-default-user]  and  [rabbit-mq-default-pass]  with the corresponding secret values.

Triggering the Integration

• Create a Liferay Objects record using the 'RabbitTest' Object Definition, populating the 'input' field, leaving the 'output' field empty, and Save.

• This will trigger the Object Action to send a message to the RabbitMQ 'demo-queue' using the rabbitmypublisher Client Extension Object Action.

JWT Claims: {sub=20124, aud=[https://rabbitmqpublish-xxxx-prd.lfr.cloud]...username=test}
JWT ID: f27e7558e85b16f3....db9
JWT Subject: 20124
rabbitMqDemoQueueName: demo-queue
############################# RabbitMQ MESSAGE SENT #############################

• The Listener class in rabbitmqlistener is listening for messages in the RabbitMQ 'demo-queue' and when it receives one, it will extract the 'id' and 'input' values from the message and use these to populate the 'output' value of the original Liferay Objects record using the headless REST API PATCH endpoint and the Headless Server OAuth 2 profile.
• Wait 15 seconds (sleep delay added for demo purposes) and refresh the Objects grid screen. The 'output' field of the Object Record should now be populated by the rabbitmqlistener custom service logic.

• On success it will send the processed message to the 'processed-queue'.
• On failure (e.g. due to an exception, unexpected message content or unexpected response from PATCH) it will send the original message to the 'error-queue'.

############################# RabbitMQ MESSAGE RECEIVED #############################
Queue: demo-queue, {"objectEntryDTORabbitTest":{...},"userId":"20124","preferredLanguageId":"en_US","status":0}
Message: {"objectEntryDTORabbitTest":{...},"userId":"20124","preferredLanguageId":"en_US","status":0}
id: 33768, input: this is a test
Response: (PATCH http://liferay:8080/o/c/rabbittests/33768) 200
Message processed as expected, moving to : processed-queue

Conclusion

While this is a fairly basic example of asynchronous message processing:

• It shows the possibilities of integrating with RabbitMQ, whether RabbitMQ is deployed in Liferay PaaS, in Liferay SaaS (or remote from Liferay).
• It shows how easy it is to send and receive RabbitMQ messages.
• It shows how the send and receive code can be implemented with the Liferay Client Extension architecture OR in 'non-Liferay' code that supports RabbitMQ / AMQP etc.
• The rabbitmqlistener is deployed as a Liferay PaaS custom service for convenience:
  • In a real world scenario the listener could be outside of Liferay PaaS and potentially built with another framework or technologies.
  • The use of a 'standalone' spring boot custom service shows that the listener can run completely outside of Liferay DXP, using OAuth 2 and the headless REST APIs to interact with Liferay DXP.
• Equally the message publisher could be outside of Liferay with the listener running as a microservice client extension, interacting with Liferay data using Liferay Objects and OAuth 2 etc.
• See Implementing Bridging APIs for more ideas on integrations.
• And it was done without creating a single custom OSGi module...