Internal secrets of Liferay Screens: SendBuffer

Lately, we’ve being working on an event tracking system for Liferay Screens. Nothing fancy: just another Flurry or Google Analytics for mobile apps. The best part is the information collected will be used to feed the Audience Targeting Rules Engine, so you’ll be able to segment your users based on their behaviour on your mobile apps.
Anyway, on the mobile end the procedure is quite simple:
  • Some screenlets (or any other part of the app’s code) will generate and add tracking events (including metadata information). You know: this button has been clicked, that view has been shown, and so on.
  • When the event count reach one specific number, the events are batched and submitted to the server. Older events are sent first, so technically the events live in a FIFO queue.
  • The submission may take several seconds, so while this is working, more events can be generated and added to the queue.
  • When the submission is successfully completed, the submitted events are finally discarded. If an error occurs during the submission, those events are added back to the  queue (in order to try again later).
After thinking twice about these steps, we realized this follows a mixture of producer-consumer together with a write-buffer pattern.
It’s a producer-consumer because:
  • There are several processes (threads, queues, whatever) creating and adding events to the queue. These are the producers.
  • There is a process removing items from the queue and sending them to the server. This is the consumer.
  • The submission is an asynchronous process. While it’s in progress, the queue is accepting new events.

And it’s a write-buffer because:

  • The buffer (or queue) can be filled. There’s a size that represent the capacity of the buffer.
  • When the buffer is full, it’s flushed. Those items are “written” (aka submitted) to the server.
  • When the submission is successfully completed, the items sent are discarded. If there’s an error sending the items, the items are added back to the buffer (and they will be re-sent later).

So with these patterns identified, we coded a Swift class that help us to handle this scenario properly.

We called it “SendBuffer”, and it’s useful for several scenarios:
  • If you need to upload items in a background process, but you want to do it in batches (instead of one by one)
  • If you need a typical producer-consumer pattern, just create the SendBuffer object with size 1. As soon as one item is added to the buffer, it will be processed right away. The buffer will continue processing items while it has more items.

Also you can find a sample application that shows graphically how the buffer behaves.

In the right you can see how items are moved back and forth:

  • When an event is added, it appears in the buffer area.
  • When the buffer gets full, the batch is moved from the buffer to the "sending area"
  • When the (fake) submission is completed, the events are moved from "sending area" to sent area. But if the submission is failed, the events are added back to the buffer.

If you see it useful, just star it in Github!

Very useful for our further endeavours with Liferay Screens,
Really good is that limiting the number of backend calls is in consideration early in the development phase.