Introducing the Semantic Search Capabilities (BETA) in DXP 7.4 U70+
Getting one (big) step closer to create a search experiences that understand the meaning and the context of the content and users searches leveraging new generation language ML models through third-party providers and the visual query builder of Search Blueprints.
Tibor Lipusz
May 23, 2023
2 Minute Read
As AI has become paramount recently and
Liferay DEVCON 2023 is
about to get started, it is exciting to highlight this recent release
and to promote the feature from development to beta phase.
Since the first demo of the
prototype and the live session delivered
on /dev/24 last year, further enhancements have been made and now
Semantic Search can be enabled as beta feature through -
Control Panel - Configuration - Instance Settings > Feature Flags
in DXP 7.4 U70+.
Configuring Semantic Search with Blueprints
To build a semantic search experience like the one above, there
is a new query element provided called Rescore by Text
Embedding ready to be used in a search blueprint.
Thanks to this element and the
visual query builder, users can easily configure the different aspects
of search and test how it performs to build the right solution for
their content and use-cases.
Once the blueprint is ready, just
like any blueprints, it can be applied
on a search page or used in the Search Bar to drive
search-as-you-typesuggestions.
While Search Blueprints
plays an integral part in this feature, there are new technologies
and concepts under the hood to make this possible. Let’s go through
briefly the main building blocks and how it integrates into
Blueprints.
This recording provides
a similar overview of the concepts of semantic search (including a
quick demo).
Text Embeddings
When this feature is enabled and a
provider (see later) is configured, a numeric (vector)
representation of the input text (obtained from a given content),
called embedding is stored in the documents in
Elasticsearch at indexing time.
Embeddings are
meant to capture the meaning and the context of the input they are
generated for example from the title and the first few sentences of
the content of a Basic Web Content Article and it can be used to
provide better results for user searches over the traditional
keyword matching.
Currently, the title/subject plus parts of the content/body of
the supported content types are used to generate the text embeddings
depending on the Text Truncation Strategy
configured.
Similarity Search with Blueprints
In order to achieve this, at search
time, the keywords entered by users need to go through the same
process making it possible to perform a similarity
search or vector search from Liferay DXP to provide
better, semantically more relevant results for users.
This is where the Rescore by
Text Embedding element mentioned earlier for Blueprints
comes in handy: this element is able to improve the results by
reordering the top matching (by the keywords) items using cosine similarity or dot
product function over the vector field Liferay DXP populates
at indexing time. It also comes with additional options to configure
how Elasticsearch will re-score the
documents.
Multiple Supported Providers for Different Scenarios
Currently, the feature supports txtai
(self-hosted / self-managed), Hugging
Face's Inference API (suitable for quick
testing/development purposes) and Hugging
Face's Inference Endpoints (enterprise-grade, paid
inference as a service) as text embedding providers. The provider
can be configured via Instance Settings - Search Experiences -
Semantic Search.
Machine Learning in Action
At heart of the
transformation process, there is always an ML model doing
the heavy-lifting.
Administrators can choose from a wide range of pre-trained
models from Hugging Face's Models
Hub and configure different properties of the provider
connection through the System / Instance Settings in Liferay
DXP.
Ready to Try?
To get
started,
Download a
DXP 7.4 U70+ Tomcat Bundle from the Customer
Portalor docker pull
liferay/dxp:7.4.13-u77
Enable the feature using
one of the available methods described here.
This website uses cookies and similar tools, some of which are provided by third parties (together “tools”). These tools enable us and the third parties to access and record certain user-related and activity data and to track your interactions with this website. These tools and the informationcollected are used to operate and secure this website, enhance performance, enable certain website features and functionality, analyze and improve website performance, and personalize user experience.
If you click “Accept All”, you allow the deployment of all these tools and collection of the information by us and the third parties for all these purposes.
If you click “Decline All” your IP address and other information may still be collected but only by tools (including third party tools) that are necessary to operate, secure and enable default website features and functionalities. Review and change your preferences by clicking the “Configurations” at any time.
