RE: Using external OSGi module in my service builder project

Bernd S, modified 5 Years ago. Junior Member Posts: 59 Join Date: 3/31/15 Recent Posts
Hello,
I want to use "Apache Commons Configuration2" in my service builder project. While this would be an easy task in a normal java environment I simply can't get it to work with OSGi.

Best resources I could find were https://liferay.dev/blogs/-/blogs/osgi-module-dependencies and https://portal.liferay.dev/docs/7-0/tutorials/-/knowledge_base/t/adding-third-party-libraries-to-a-module.

According to my understanding of it, "Apache Commons Configuration2" is already an OSGi module (MANIFEST.MF exists and contains OSGi configuration). Therefore it should be enough to add the library to my build file and the -includeresource instruction inside bnd.bnd.

Unfortunately this only leads to other unresolved requirements which are the library's dependencies. This would be understandable if I wanted to  use a non-OSGi jar but mine is a OSGi module.

I am using Liferay DXP 7.0 with Maven, if that is important.

What am I doing wrong?
thumbnail
Olaf Kock, modified 5 Years ago. Liferay Legend Posts: 6441 Join Date: 9/23/08 Recent Posts
Bernd S:

Hello,
I want to use "Apache Commons Configuration2" in my service builder project. While this would be an easy task in a normal java environment I simply can't get it to work with OSGi.
...
According to my understanding of it, "Apache Commons Configuration2" is already an OSGi module (MANIFEST.MF exists and contains OSGi configuration). Therefore it should be enough to add the library to my build file and the -includeresource instruction inside bnd.bnd.
For actual pointers, we'd need to know what classes are still missing. My recommendation would be to
  • not use -includeresource, but rather deploy the bundle as well. It's an OSGi bundle already, and deploying it separately will shrink your own bundle's size dramatically
  • Figure out the dependencies that it requires and deploy them as well to the runtime. (if you run into non-OSGi-bundles while figuring out those dependencies, -includeresource might be an ugly quickfix method again, but you should prefer to use it as last resort)
Bernd S, modified 5 Years ago. Junior Member Posts: 59 Join Date: 3/31/15 Recent Posts
Intially all compile dependencies as listet in https://mvnrepository.com/artifact/org.apache.commons/commons-configuration2/2.6 are unresolved (e.g. Unresolved requirement: Import-Package: com.fasterxml.jackson.databind). 
I can slowly resolve them through addtions to the bnd.bnd but I simply can't believe that this is how it is supposed to work.

Deploying the bundle as well results in a different error:

<code>Export-Package: org.apache.commons.configuration2; bundle-symbolic-name="org.apache.commons.commons-configuration2"; bundle-version="2.6.0"; version="2.6.0"
       org.apache.commons.commons-configuration2 [630]
         Unresolved requirement: Import-Package: org.apache.commons.jxpath; resolution:="optional"   
         Unresolved requirement: Import-Package: org.apache.commons.jxpath.ri; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.commons.jxpath.ri.compiler; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.commons.jxpath.ri.model; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.xml.resolver; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.xml.resolver.helpers; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.xml.resolver.readers; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.xml.resolver.tools; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.commons.jexl2; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.commons.vfs2; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.commons.vfs2.provider; resolution:="optional"
         Unresolved requirement: Import-Package: com.fasterxml.jackson.databind; resolution:="optional"
         Unresolved requirement: Import-Package: com.fasterxml.jackson.databind.type; resolution:="optional"
         Unresolved requirement: Import-Package: org.yaml.snakeyaml; resolution:="optional"
         Unresolved requirement: Import-Package: org.apache.commons.lang3</code>

I just want to use the library without listing all it's dependencies manually...
thumbnail
Olaf Kock, modified 5 Years ago. Liferay Legend Posts: 6441 Join Date: 9/23/08 Recent Posts
Bernd S:

I can slowly resolve them through addtions to the bnd.bnd but I simply can't believe that this is how it is supposed to work.
...
I just want to use the library without listing all it's dependencies manually...
Looking at its MANIFEST.MF, I can see that there are numerous optional imports - and you should only need to add those bundles if your code makes use of them. But there are also some that are not optional
Pardon the unreadable mess, this is straight from MANIFEST.MF:
Import-Package: org.apache.commons.beanutils;resolution:=optional,org.
&nbsp;apache.commons.codec.binary;resolution:=optional,org.apache.commons.j
&nbsp;xpath;resolution:=optional,org.apache.commons.jxpath.ri;resolution:=o
&nbsp;ptional,org.apache.commons.jxpath.ri.compiler;resolution:=optional,or
&nbsp;g.apache.commons.jxpath.ri.model;resolution:=optional,org.apache.xml.
&nbsp;resolver;resolution:=optional,org.apache.xml.resolver.helpers;resolut
&nbsp;ion:=optional,org.apache.xml.resolver.readers;resolution:=optional,or
&nbsp;g.apache.xml.resolver.tools;resolution:=optional,javax.servlet;resolu
&nbsp;tion:=optional,org.apache.commons.jexl2;resolution:=optional,org.apac
&nbsp;he.commons.vfs2;resolution:=optional,org.apache.commons.vfs2.provider
&nbsp;;resolution:=optional,org.springframework.beans.factory;resolution:=o
&nbsp;ptional,org.springframework.core.env;resolution:=optional,org.springf
&nbsp;ramework.core.io;resolution:=optional,org.springframework.util;resolu
&nbsp;tion:=optional,com.fasterxml.jackson.databind;resolution:=optional,co
&nbsp;m.fasterxml.jackson.databind.type;resolution:=optional,org.yaml.snake
&nbsp;yaml;resolution:=optional,[b]javax.naming,javax.sql,javax.xml.parsers,ja
&nbsp;vax.xml.transform,javax.xml.transform.dom,javax.xml.transform.stream,
&nbsp;org.apache.commons.lang3,org.apache.commons.lang3.builder,org.apache.
&nbsp;commons.lang3.concurrent,org.apache.commons.lang3.mutable,org.apache.
&nbsp;commons.logging,org.apache.commons.logging.impl,org.apache.commons.te
&nbsp;xt,org.apache.commons.text.lookup,org.apache.commons.text.translate,o
&nbsp;rg.w3c.dom,org.xml.sax,org.xml.sax.helpers[/b]
If you look at the end of this, there are some nonoptional packages: You'll need to make sure that they're available, no matter if your code makes use of them or not. But all the optional ones depend on your own code.
At compile-time, everything is handled automatically, but you're responsible to also make the code available at runtime. It might seem unfortunate, or a lot of work. This could be mitigated by commons-configuration being packaged as a single monolithic bundle, but what if you'd only be interested in org.apache.commons.text in another project? Would you expect it to be exported by the commons-configuration bundle? Or would you expect the same code to be available on the classpath in addition to what you have in the monolithic bundle?
Bernd S, modified 5 Years ago. Junior Member Posts: 59 Join Date: 3/31/15 Recent Posts
First exception is about an optional dependency which I never use.
Unresolved requirement: Import-Package: com.fasterxml.jackson.databind


Only way I got it to work was by adding the library and it's required depenecies to
-includeresource
and
Bundle-ClassPath

Additionally I had to exclude several packages in
Import-Package

(no clue about the logic behind it)

Am I missing something?
thumbnail
Olaf Kock, modified 5 Years ago. Liferay Legend Posts: 6441 Join Date: 9/23/08 Recent Posts
Bernd S:

First exception is about an optional dependency which I never use.
Unresolved requirement: Import-Package: com.fasterxml.jackson.databind
...
Additionally I had to exclude several packages in
Import-Package
I assumed that the optional dependencies were indeed optional, but it seems that this one isn't. Bummer, seems to rather ask for an update on the apache-commons-configuration side.
As I don't know what "several packages" are, I can't offer my version of an attempted explanation.
What I've learned in the past is that the dependency-resolution sometimes can be a bit weird, but it's a matter of the chosen dependencies. Some are straightforward, some come with transitive dependencies (that come with more transitive dependencies). One way or another, you'll have to make these available at runtime. You definitely save more time hunting down the dependencies than you'd spend implementing the underlying functionality yourself (without dependencies). So while it may be work to put them together, this step saves you a lot of work (and maintenance) in another step.
thumbnail
David H Nebinger, modified 5 Years ago. Liferay Legend Posts: 14933 Join Date: 9/2/06 Recent Posts
So here's a thing...

First, any module that is OSGi-enabled can be deployed into the OSGi container by dropping it into the Liferay deploy folder. From a project perspective, you are either going to use "compile" or "compileOnly" directives (in Gradle) or the "provided" scope (in Maven) to satisfy compile time dependencies.

As an OSGi module, though, it becomes your responsibility to resolve any 3rd party dependencies the OSGi-enabled module has. In your case, you've seen the long list of "optional" dependencies that you'd need to take care of resolving.


The good news for you is that each of the listed packages are also provided by OSGi-enabled modules. The bad news for you is that they each will come with their own list of other transient dependencies, so pretty soon your "I'll just deploy commons-config2 into the OSGi container becomes a nightmare.

What makes this even harder is that OSGi does not understand the "optional" designation bound to a transient dependency. This may sound obtuse, but if you think about it, the container really can't know if you (the developer) will want the org.apache.commons.jexl2 or not. After all, that depends upon what features of commons-config2 you are going to use, and at module deployment time the container can't read your mind to know if you plan on using features bound to JEXL or not. So even though the dependency is listed as "optional" in the manifest, to the OSGi container it must be satisfied or explicitly masked.

So you already know about how to satisfy the transient dependencies (and the headache that comes with that), so how about the masking part? Well, that is going to be done using a fragment bundle to adjust the manifest of the OSGi-enabled module. Similar to what I covered here: https://liferay.dev/blogs/-/blogs/fixing-module-package-access-modifiers but instead of adding packages for export, you're going to use the Import-Package directive to mask off those packages you know you don't need. So if you were going to use all of the transient dependencies except for JEXL, for example, you'd have the directive like:

Import-Package: !org.apache.commons.jexl2, *


This will block the load of JEXL but still allow for importing the rest of the dependencies.

As a fragment bundle, this will attach itself to the fragment host and modify its manifest so the details are controlled by you and not by the container.

Finally, as an aside, consider this: Liferay already comes with commons-configuration support baked in. The portal-ext.properties, the portlet-ext.properties and service-ext.properties support, the use of environment variables in those files, "include-and-override" support, all of these things come from just using the Liferay facilities.

So you really should be asking yourself, do I really need another configuration handling package in my environment? What is the key feature that I need from commons-configuration2 and is it really worth all of these issues to use its version instead of sticking with Liferay's commons-configuration support?

Another example is commons-lang3. I work with developers that just want to go off and pull in commons-lang3 (because it's the latest and greatest), but I always reject their PRs when I see it. Why? Because Liferay already provides commons-lang (commons-lang2 I guess is what it really should be called), and typically the devs aren't using a feature exclusive to lang3 in order to justify using it instead.

Is this important? Well, no, not really, OSGi is happy to have commons-lang2 and commons-lang3 as well as commons-configuration and commons-configuration2 in the same container. It's not going to complain about that at all as long as the transient dependencies are satisfied.

I reject it though because even though OSGi might support all of the different versions, I don't want my projects getting sucked into a vortex of each developer picking one or the other; instead i want to be consistent in all projects. Sticking with commons-lang2 and commons-configuration is a easy restriction to live under since functionality between the versions is still pretty similar. The consistency I get across my projects is knowing that all developers will be using commons-lang2 and no developer is making a choice on their own.

I would encourage you to consider the same when thinking about pulling in commons-configuration2... You may have a good reason to include it to justify all of this, and if so, please ignore this last bit of advice. But if you can't single out a needed feature that only commons-configuration2 provides that commons-configuration does not, I would encourage you to consider sticking with the provided commons-configuration support and keep your portal implementations as simple as possible.
Bernd S, modified 5 Years ago. Junior Member Posts: 59 Join Date: 3/31/15 Recent Posts
Wow, thanks for that in-dept explanation. Now I understand it a little bit better.
Didn't realise that commons-configuration was already included but now that I know I will switch over.