Liferay - Azurite integration

Emulation of Azure Blob Storage

I've recently been involved in an investigation regarding an issue with the Azure Blob Storage DL Store connector.

One of my concern was the reproduction of the issue.

Meet Azurite: https://github.com/Azure/Azurite

Azurite is an Azure emulator which supports Azure Blob Storage, Queue and Tables.

As far as Liferay is concerned, it's Azure Blob Storage we are interested with.

Here's a step by step guide to setup Azurite as your local DL Store in your Windows development environment (some adjustments required if you work with Linux or Mac).

Installing Azurite

Azurite is a Node.js application. Let's install it globally:

npm install -g azurite

And start the Azurite emulator:

azurite -s -l c:\azurite -d c:\azurite\debug.log

Done! Azurite Blob Storage emulator is now running on port 10000 with a default account / key we are going to use in our configuration script and in the Azure store connection string configuration.

Creating an empty blob storage container

We now have to create an empty container in Azurite where we are going to store our Document Library.

This is good news, Azurite will allow you to manage multiple Azure Store document libraries if you need to run several Liferay instances in your local workstation!

This step is not as straightforward.

I've found the suggestion here to be the easiest way to deal with it: https://github.com/Azure/Azurite/issues/1968#issuecomment-1556898331

In some empty directory, create a file named create-container.mjs:

import {
    BlobServiceClient,
  } from '@azure/storage-blob';

const containerToBeCreated = "liferay-container";
const connectionString = 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;';
const client = BlobServiceClient.fromConnectionString(connectionString);

console.log("Listing existing containers:");
let containerExists = false;
for await (const container of client.listContainers()) {
  if(container.name === containerToBeCreated) {
    containerExists = true;
  }
  console.log(`* ${container.name}`);
}

if(!containerExists) {
  try {
    console.log(`Container '${containerToBeCreated}' does not exist, creating...`);
    await client.createContainer(containerToBeCreated);
    console.log("Success!");
  } catch(e) {
    console.log("Container creation failed!");
    console.log(e);
  }
}

The name of the container to be created is specified at line 5. I've called it "liferay-container".

Open a command line in the directory where you have created this file and run the following commands:

npm install @azure/storage-blob

node --require @azure/storage-blob create-container.mjs

You should get the following output:

Listing existing containers:
Container 'liferay-container' does not exist, creating...
Success!

We now have an empty container ready to receive the document library.

Migrating data from the local file store to the Azurite store

In the Liferay Control Panel, go to System Settings and File Storage.

In the Azure Blob Storage section, provide the following configuration:

For Connection String:

AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;DefaultEndpointsProtocol=http;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;

For Container Name:

liferay-container

And for the sake of debugging, you can check the Enable HTTP Logging checkbox.

Save those settings and go to Server Administration in the Liferay Control Panel and to the Data Migration Tab.

Select com.liferay.portal.store.azure.AzureStore in the dropdown and trigger the data migration.

It might take more or less time depending on how much documents you have. With an empty Liferay workspace, this should finish in a few seconds.

This is what you should see in your Liferay logs:

2023-08-14 12:18:08.416 DEBUG [http-nio-8080-exec-10][MaintenanceUtil:27] Executing com.liferay.document.library.internal.convert.document.library.DocumentLibraryConvertProcess
2023-08-14 12:18:08.429 INFO  [liferay/convert_process-2][BaseConvertProcess:46] Starting conversion for com.liferay.document.library.internal.convert.document.library.DocumentLibraryConvertProcess
2023-08-14 12:18:08.445 DEBUG [liferay/convert_process-2][MaintenanceUtil:27] Migrating 11 documents and media files
2023-08-14 12:18:08.766 DEBUG [liferay/convert_process-2][MaintenanceUtil:27] Migrating 1 images
2023-08-14 12:18:08.782 DEBUG [liferay/convert_process-2][MaintenanceUtil:27] Migrating files from document_thumbnail/
2023-08-14 12:18:08.784 DEBUG [liferay/convert_process-2][MaintenanceUtil:27] Migrating files from document_preview/
2023-08-14 12:18:08.788 DEBUG [liferay/convert_process-2][MaintenanceUtil:27] Migrating images in 16 adaptive media image entries
2023-08-14 12:18:09.067 DEBUG [liferay/convert_process-2][MaintenanceUtil:27] Please set dl.store.impl in your portal-ext.properties to use com.liferay.portal.store.azure.AzureStore
2023-08-14 12:18:09.067 INFO  [liferay/convert_process-2][BaseConvertProcess:54] Finished conversion for com.liferay.document.library.internal.convert.document.library.DocumentLibraryConvertProcess in 639 ms

Restarting Liferay with the Azurite store

It's now time to update portal-ext.properties with this new setting:

dl.store.impl=com.liferay.portal.store.azure.AzureStore

And restart Liferay.

Add some document to the document library.

Open the C:\Azurite\debug.log file, you should be able to see the requests Liferay makes against the Azurite service.