Introduction

Welcome back to my series on using Liferay's REST Builder tool to generate your own Headless APIs!

In part 1 of the series, we created a new project and modules, and we started to create the OpenAPI Yaml file defining our headless services by specifying the Reusable Components section.

In part 2 of the series, we completed the OpenAPI Yaml file by adding in our paths, working through common issues and generated code using REST Builder.

In part 3 we reviewed all of the generated code to understand what had been built for us and touched on where we would be adding our implementation code.

In this part, we're going to create a ServiceBuilder (SB) layer we'll need for persisting the values, paying close attention to those pieces we need to implement specifically to support the headless API.

Creating the Service Builder Layer

We're going to use Service Builder for our persistence layer. I'm not going to get into all of the details about how to do this, but I will highlight those things we're adding in order to facilitate the headless API.

The most complicated aspect of the service portion is what would seem to be the easiest - the /vitamins path to get all Vitamin components.

Why is this so hard? Well, we're following the Liferay model so we need to be able to:

• Support search, this is done via indexing, so our SB entity must be indexed.
• Support permissions since the new search implementation is permission aware by default.
• Support sorting of the results determined by the caller.
• Filtering results as defined using special strings defined here: https://portal.liferay.dev/docs/7-2/frameworks/-/knowledge_base/f/filter-sort-and-search#filter
• Support pagination of results, but with the page size determined by the caller.
• Remote Services so the permission checker is invoked at the right points.

In order to make all of this happen, we need to ensure that our entity is indexed. Find out how to do that here: https://portal.liferay.dev/docs/7-2/frameworks/-/knowledge_base/f/model-entity-indexing-framework

With the new indexing being permissions-aware by default, we also need to add permissions to our entities per: https://portal.liferay.dev/docs/7-2/frameworks/-/knowledge_base/f/defining-application-permissions

Because I called my component Vitamin, I didn't want my Service Builder code to also use Vitamin, otherwise I'd have to include package everywhere. Instead I opted to call my entity PersistedVitamin. This should help distinguish between the DTO class that Headless is using and my actual persisted entity that is managed by Service Builder.

Supporting List Filter, Search and Sort

The rest of this section covers adding support for list filtering, searching and sorting using Liferay supported mechanisms. If you are not going to support list filtering, searching or sorting, or if you are planning to support one or more of them but not using Liferay techniques, this section might not apply to you.

In many of Liferay's list methods such as /v1.0/message-board-threads/{messageBoardThreadId}/message-board-messages, there are additional attributes that you can provide in the query to support search, filter, sort, paging and field restrictions...

All of the Liferay documentation on these aspects are covered in the doco:

• https://portal.liferay.dev/docs/7-2/frameworks/-/knowledge_base/f/pagination
• https://portal.liferay.dev/docs/7-2/frameworks/-/knowledge_base/f/filter-sort-and-search
• https://portal.liferay.dev/docs/7-2/frameworks/-/knowledge_base/f/restrict-properties

The part that it doesn't really share is that filter, sort and search all require the use of the search index for the entities.

Search, for example, is performed by adding one or more keywords to the query. These feed into the index query to find matches on your entities.

Filtering is also managed by adjusting the index search query. To filter on one or more fields in your component, those fields need to be in the search index. Additionally you'll need the OData EntityModel for the fields that we'll cover in a different section below.

Sorting is also managed by adjusting the index search query. To sort on one or more fields in your component, those fields need to be in the search index. Additionally, they should be indexed using the addKeywordSortable() methods from the com.liferay.portal.kernel.search.Document interface. Sortable fields will also need to be added to the OData EntityModel implementation we'll cover soon.

Keeping this in mind, you're going to want to pay special attention to your search definitions for your custom entities:

• Use your ModelDocumentContributor to add important text and/or keywords to get appropriate search hits.
• Use your ModelDocumentContributor to add fields that you want to support filtering on.
• Use your ModelDocumentContributor to add the sortable keyword fields that you want to sort on.

Implementing the VitaminResourceImpl Methods

Once you have a Service Builder layer and fix up the headless-vitamins-impl dependencies, the next step is to actually start implementing the methods...

Implementing deleteVitamin()

Let's start with an easy one, the deleteVitamin() method. In VitaminResourceImpl we're going to extend the method from the base class (the one with all of the annotations, remember?) and then invoke our service layer:

@Override
public void deleteVitamin(@NotNull String vitaminId) throws Exception {
  // super easy case, just pass through to the service layer.
  _persistedVitaminService.deletePersistedVitamin(vitaminId);
}

Really easy, isn't it?

So I'm going to recommend that you use only your remote services to handle entity persistence, not the local services.

Why? Well, it is really your last line of defense to ensure that a user has permission to do something like delete a vitamin record.

Sure, you can exercise control using OAuth2 scopes to block activity, but do you really want to depend upon an admin getting the OAuth2 scope configurations correct? Heck, even when I'm my own admin, I don't trust that I'll get the scopes right every time...

By using the remote services w/ the permission checks, I won't have to worry about the scopes being correct... If an admin (me) screws up the OAuth2 scopes, the remote services will still block the operation if the user does not have the right permissions.

Handling Conversions

Before we can get further into some of our implementation methods, we have to talk about conversions from our backend ServiceBuilder entities into the headless Components that we're going to be returning.

At the current point in time, Liferay has not really settled on a standard for dealing with entity -> component conversion. The headless-delivery-impl module from the Liferay source does conversion one way, but the headless-admin-user-impl module handles the conversion in a different way.

Because of the simplicity, I'm going to present a method here based on the headless-admin-user-impl technique. You may have a technique that works better for you that is different than this one, or you might favor the headless-delivery-impl method. And Liferay could come out with a standard way to support conversion in the next release which might make all of this moot.

I guess I'm saying that you need to handle conversion, but you're not locked into a particular way. Liferay might come out with something better, but it will be up to you to adapt to the new way or run with what you have working.

So, we need to be able to convert from a PersistedVitamin to a Vitamin component to return as part of our headless API definition. We'll create a _toVitamin() method in the VitaminResourceImpl class:

protected Vitamin _toVitamin(PersistedVitamin pv) throws Exception {
  return new Vitamin() {{
    creator = CreatorUtil.toCreator(_portal, _userLocalService.getUser(pv.getUserId()));
    articleId = pv.getArticleId();
    group = pv.getGroupName();
    description = pv.getDescription();
    id = pv.getSurrogateId();
    name = pv.getName();
    type = _toVitaminType(pv.getType());
    attributes = ListUtil.toArray(pv.getAttributes(), VALUE_ACCESSOR);
    chemicalNames = ListUtil.toArray(pv.getChemicalNames(), VALUE_ACCESSOR);
    properties = ListUtil.toArray(pv.getProperties(), VALUE_ACCESSOR);
    risks = ListUtil.toArray(pv.getRisks(), VALUE_ACCESSOR);
    symptoms = ListUtil.toArray(pv.getSymptoms(), VALUE_ACCESSOR);
  }};
}

So first off, I have to apologize for using the double brace instantiation... I too see it as an anti-pattern (https://blog.jooq.org/2014/12/08/dont-be-clever-the-double-curly-braces-anti-pattern/), but my goal was to follow "the Liferay way" as laid out in the headless-admin-user-impl module, and that was the pattern Liferay used. Since Liferay doesn't use the Builder pattern often, I think the double brace instantiation is being used as a substitute.

Given my own preference, I would follow the Builder pattern or even a Fluent pattern to simplify object population. After all, Intellij will easily create Builder classes for me (you do know it is capable of doing that, right?).

The method relies on an external CreatorUtil class (that I copied from Liferay's code), a _toVitaminType() method that converts from an internal integer code to the component's enum, and a VALUE_ACCESSOR that handles the internal objects that are part of the implementation details into a String array thanks to ListUtil's toArray() method.

Long story short, this method can handle the conversion that we need to perform in our actual method implementations.

Implementing getVitamin()

Let's look at another easy one, the getVitamin() method, the one that will return a single entity given the vitaminId:

@Override
public Vitamin getVitamin(@NotNull String vitaminId) throws Exception {
  // fetch the entity class...
  PersistedVitamin pv = _persistedVitaminService.getPersistedVitamin(vitaminId);

  return _toVitamin(pv);
}

Here we retrieve the PersistedVitamin instance from the service layer, but then we pass the retrieved object to _toVitamin() method for conversion.

Implementing postVitamin(), patchVitamin() and putVitamin()

Since we've seen the pattern above, I'm lumping these together...

postVitamin() is the method for the POST on /vitamins and represents creating a new entity.

patchVitamin() is the method for the PATCH on /vitamins/{vitaminId} and represents patching an existing entity (only changing values given in the incoming object, leaving other existing properties alone).

putVitamin() is the method for the PUT on /vitamins/{vitaminId} and represents the replacement of an existing entity, replacing all persisted values with what is passed in, even if the fields are null/empty.

Since I created my ServiceBuilder layer and customized for these entry points, my implementations in the VitaminResourceImpl class looks pretty light:

@Override
public Vitamin postVitamin(Vitamin v) throws Exception {
  PersistedVitamin pv = _persistedVitaminService.addPersistedVitamin(
      v.getId(), v.getName(), v.getGroup(), v.getDescription(), _toTypeCode(v.getType()), v.getArticleId(), v.getChemicalNames(),
      v.getProperties(), v.getAttributes(), v.getSymptoms(), v.getRisks(), _getServiceContext());

  return _toVitamin(pv);
}

@Override
public Vitamin patchVitamin(@NotNull String vitaminId, Vitamin v) throws Exception {
  PersistedVitamin pv = _persistedVitaminService.patchPersistedVitamin(vitaminId,
      v.getId(), v.getName(), v.getGroup(), v.getDescription(), _toTypeCode(v.getType()), v.getArticleId(), v.getChemicalNames(),
      v.getProperties(), v.getAttributes(), v.getSymptoms(), v.getRisks(), _getServiceContext());

  return _toVitamin(pv);
}

@Override
public Vitamin putVitamin(@NotNull String vitaminId, Vitamin v) throws Exception {
  PersistedVitamin pv = _persistedVitaminService.updatePersistedVitamin(vitaminId,
      v.getId(), v.getName(), v.getGroup(), v.getDescription(), _toTypeCode(v.getType()), v.getArticleId(), v.getChemicalNames(),
      v.getProperties(), v.getAttributes(), v.getSymptoms(), v.getRisks(), _getServiceContext());

  return _toVitamin(pv);
}

Like I said, they are pretty light...

Since I'm going to the service layer, I need a ServiceContext. Liferay provides a com.liferay.headless.common.spi.service.context.ServiceContextUtil that has just the method I need to create my ServiceContext. It starts a context, I just need to add some additional stuff into it like the company id and the current user id. So I wrapped all of this into the _getServiceContext() method. And good news for me, in future versions of the REST Builder, I'm going to be getting some new context variables which will make getting a valid ServiceContext much easier.

My ServiceBuilder methods all use the blown out parameter passing we all know and love about ServiceBuilder. The PersistedValue instance I get back from the method calls gets passed off to _toVitamin() for conversion which is then returned.

And that's all of the simple methods to deal with. We still have to cover the getVitaminsPage() method, but before we do that we have to cover the EntityModels...

EntityModels

Earlier I discussed how Liferay supports list filtering, searching and sorting by using the search index. I also discussed how fields available for filtering or sorting must be part of an EntityModel definition for your components. Fields from the component that are not part of the EntityModel cannot be filtered nor sorted.

An additional side effect, since the EntityModel exposes those fields from the search index for filtering and sorting, those fields do not have to be connected to the Component fields.

For example, in an EntityModel definition, you could add an entry for a creatorId that would be a filter to the user id in the search index. The component definition might have the Creator field and not a creatorId field, but the creatorId can still be used in both filtering and/or sorting since it is part of the EntityModel.

So we have to build out an EntityModel, one that defines both the fields we want to support filtering on as well as the fields we want to support sorting on. We're going to be using mostly existing Liferay utilities to help put our EntityModel class together.

Here it is:

public class VitaminEntityModel implements EntityModel {
  public VitaminEntityModel() {
    _entityFieldsMap = Stream.of(
        // chemicalNames is a string array of the chemical names of the vitamins/minerals
        new CollectionEntityField(
            new StringEntityField(
                "chemicalNames", locale -> Field.getSortableFieldName("chemicalNames"))),
        
        // we'll support filtering based upon user creator id.
        new IntegerEntityField("creatorId", locale -> Field.USER_ID),
        
        // sorting/filtering on name is okay too
        new StringEntityField(
            "name", locale -> Field.getSortableFieldName(Field.NAME)),
        
        // as is sorting/filtering on the vitamin group
        new StringEntityField(
            "group", locale -> Field.getSortableFieldName("vitaminGroup")),
        
        // and the type (vitamin, mineral, other).
        new StringEntityField(
            "type", locale -> Field.getSortableFieldName("vType"))
    ).collect(
        Collectors.toMap(EntityField::getName, Function.identity())
    );
  }

  @Override
  public Map<String, EntityField> getEntityFieldsMap() {
    return _entityFieldsMap;
  }

  private final Map<String, EntityField> _entityFieldsMap;
}

So the Field names, those come from the names I used in the PersistedVitaminModelDocumentContributor class in the service layer to add my field values.

I've included definitions for chemicalNames, Field.USER_ID, Field.NAME, vitaminGroup and vType Fields from the search index. Of the definitions, the creatorId field the filter would use, that doesn't exist as a field of the Vitamin component definition.

The other fields that are part of the Vitamin component, well I just don't feel like I need to allow for sorting or filtering on the rest. Obviously this kind of decision will normally be driven by your requirements.

Liferay saves these classes in an "odata.entity.v1_0" package in your internal package, so I have the com.dnebinger.headless.delivery.internal.odata.entity.v1_0 package where I put my file.

Now that the class is ready, we must also decorate the VitaminResourceImpl class so it correctly reports that it can serve an EntityModel.

Here are the changes you need to make:

• The <Component>ResourceImpl class needs to implement the com.liferay.portal.vulcan.resource.EntityModelResource interface.
• The class must implement the getEntityModel() method that returns an EntityModel instance.

And that's it. Because my VitaminEntityModel is pretty simple and not very dynamic, my implementation is like:

public class VitaminResourceImpl extends BaseVitaminResourceImpl 
    implements EntityModelResource {

  private VitaminEntityModel _vitaminEntityModel = new VitaminEntityModel();

  @Override
  public EntityModel getEntityModel(MultivaluedMap multivaluedMap) throws Exception {
    return _vitaminEntityModel;
  }

It is important to note that this may not be a typical implementation. Liferay's component resource implementation classes have significantly more complicated and dynamic EntityModel generation, but this is due to the complexity of the entities involved (for example, StructuredContent is a mish-mash of a JournalArticle, a DDM structure and a template, and I think there may also be a kitchen sink in there too if you look hard enough).

So don't blindly copy my method and run with it. It may work in your case, but it may not. For more complicated scenarios, check out the Liferay implementations for EntityModel classes as well as the getEntityModel() methods in the component resource implementations.

Implementing getVitaminsPage()

So this is probably the most complicated method to implement. Not because it is challenging, per se. It is just dependent upon so many other things...

The Liferay list handling functionality here comes from the search index, not the database. So this requires our entities are indexed.

This is also the method that supports filter, search and sort parameters; these too require that the entity is indexed. And as we just saw, filter and sort are also dependent upon the EntityModel classes.

And finally, since it is calling out to Liferay methods, the implementation itself will seem pretty opaque and out of our control.

Here's what we end up with:

public Page<Vitamin> getVitaminsPage(String search, Filter filter, Pagination pagination, Sort[] sorts) throws Exception {
  return SearchUtil.search(
    booleanQuery -> {
      // does nothing, we just need the UnsafeConsumer<BooleanQuery, Exception> method
    },
    filter, PersistedVitamin.class, search, pagination,
    queryConfig -> queryConfig.setSelectedFieldNames(
      Field.ENTRY_CLASS_PK),
    searchContext -> searchContext.setCompanyId(contextCompany.getCompanyId()),
    document -> _toVitamin(
      _persistedVitaminService.getPersistedVitamin(
        GetterUtil.getLong(document.get(Field.ENTRY_CLASS_PK)))),
    sorts);
}

So we're using the SearchUtil.search() method which knows how to process everything...

The first argument is the UnsafeConsumer class which is basically responsible for tweaking the booleanQuery as necessary for your entities. I didn't need one here, but there are examples in the Liferay headless-delivery module. The StructuredContent's version that finds articles by site id will add the site id as a query argument. The "flatten" parameter will tweak the query to search a specific filter, those kinds of things.

The filter, search, and pagination arguments that we get from the headless layer are passed straight through; they will be applied to the boolean query to filter and search results, and pagination will make sure that we get a page worth of results.

The queryConfig is asking for just the return of the primary key values and none of the other field data. Since we don't convert from a search index Document, we will need the ServiceBuilder entity for that, so the query doesn't need to return any of the other Fields in the Documents.

The next to last argument is another UnsafeFunction which is responsible for applying the transformation from the Document to the component type; the implementation provided fetches the PersistedVitamin instance using the primary key value extracted from the Document, and that PersistedVitamin is passed through _toVitamin() to handle the final conversion.

Wrapping Up

So now we're actually done with all of the coding activities, but we're not completely done...

We want to re-run the buildREST command again. We've added methods into our VitaminResourceImpl method and we want to make sure we have the test cases ready to apply to them.

Next, we need to build and deploy our modules and clean up any deployment issues such as unresolved references and stuff. We deploy the vitamins-api and vitamins-service for the ServiceBuilder tier and the vitamins-headless-api and vitamins-headless-impl modules for the Headless tier.

When those are ready, we should drop into our headless-vitamins-test module and run all of our test cases (and if there are some that are missing, well we can recreate those too).

When all of that is ready, we might want to consider publishing our Headless API to Swaggerhub so others can consume it.

We don't want to use the Yaml file we created for REST Builder. Instead we want to point our browsers at http://localhost:8080/o/headless-vitamins/v1.0/openapi.yaml and use that file for the submission. It will have all of the necessary parts in place plus some additional components such as the PageVitamin type, etc.

Conclusion

And there we have it!

We started in part 1, creating our workspace and modules for our new Headless adventure. We also started the OpenAPI Yaml file that REST Builder would eventually use to generate code by defining our Reusable Components section with our Component type definitions.

In part 2, we completed the OpenAPI Yaml file for REST Builder by adding in our path definitions. We had a REST Builder generation failure once and covered some of the common formatting errors that can cause generation failures. We fixed those and then successfully generated code using REST Builder.

In part 3 we reviewed all of the generated code in all of the modules to see what was created for us and hinted where our modifications were going to be made.

And finally here, in part 4, we created a Service Builder layer and included resource permissions (for permission checking in the remote services) and entity indexing (to support the list filter/search/sort capabilities of Liferay's Headless infrastructure). We then flushed out our VitaminResourceImpl methods, discussed how to handle entity to Component conversions as well as the EntityModel classes needed to facilitate filters and sorts.

We wrapped it all up with testing and possibly publishing our API to Swaggerhub for everyone to enjoy.

It's been a long road, but certainly an interesting one for me. I hope you enjoyed it also.

And once again, here's the repo for the blog series: https://github.com/dnebing/vitamins