Blogs

Blogs

Best Practice: Track Package Versions, not Bundle Versions

Bundle versions are easy to track and change, but they don't speak to what really may have changed in the packages.

So recently I got to hang out with board members and officers of the OSGi Alliance, including Liferay's own Ray Auge. A great bunch of folks these guys are; they were easy to talk to and really wanted to help out. Can't wait to do it again, but that's a different story...

At one point we were talking about versioning and I was explaining how I version my bundles, but I don't really ever go back and version my packages correctly.

I mean, in a project with, say, 15 packages, I just didn't see why I had to track each package version separately. I mean, sometimes I feel lucky if I can get developers to add some javadoc to a method, let alone expect correct updates to individual package versions.

This led into a discussion about how I was not versioning correctly. It took awhile for their message to sink in, but now that I understand why, I want to share it with you...

Package Version Intent

Okay, so the idea is that the package version is the key for OSGi to decide if a dependency is satisfied or not. The bundle version really doesn't play a role in this.

It's why, when you check the MANIFEST.MF file or the OSGi meta files in your bundles, you really won't see bundle versions being referenced as dependencies, only package versions.

Bundle version really only has a role during the build process since the tooling and repositories don't really track artifacts by package versions. So the bundle version is used to identify a specific dependency.

So the fault in my thinking was that the bundle version, used as a build time dependency identifier, was also used during OSGi runtime reference resolution. It isn't used this way.

Alternatively you might think that a bundle version implies something about package version. If I make a change in my code, I bump the minor version of the bundle and that implies a subsequent package change. But if you only make a change in one package, your other packages haven't changed at all. If I bump a bundle version from 1.0 to 1.1, I can't really infer that all packages in the bundle really also changed from 1.0 to 1.1. There simply is no connection there.

And consider an inverse scenario; I have a bundle that is 1.0 and all of my packages are 1.0. I change one java class in the project and I properly bump just that package to version 1.1 and my bundle also goes to 1.1. When I deploy the 1.1 bundle to my OSGi container (Liferay), my other bundles that depend on this one may or may not resolve.

If you had a package, com.dnebinger.api, and that package was still at 1.0 because you changed the com.dnebinger.impl package version, other bundles that depended upon com.dnebinger.api:1.0 will be perfectly fine running with the 1.1 bundle version. Only bundles that depend upon com.dnebinger.impl:1.0 won't be happy because only 1.1 might be available.

Because it is the package version that OSGi cares about resolving with, not the bundle version at all.

Best Practice: Version Packages Correctly

So for OSGi resolution to be happy, as developers we need to version our packages correctly when we make changes.

This needs to be part of your build process to ensure that OSGi is able to resolve and bind to bundles that provide the packages at the right version.

Versioning packages correctly, well that phrase is kind of ambiguous. Which way is correct? Incrementing the dot until you hit 10, roll it over in the minor version and reset dot release to 0, then keep going? Is it following your corporate versioning policy (assuming you have one) or rolling your own?

Versioning packages correctly, in the OSGi realm, means applying proper semantic versioning rules.

OSGi's rules define how to version as classes are added, removed or renames, as methods are added, removed or renamed, as method signatures are added, etc. Some changes, such as adding a brand new method, end up being a minor version increment while others signify a major version change.

Remember that with OSGi version ranges for resolution, the version number of the package determines if it can be resolved. A component that relies on com.dnebinger.api [1.0.0,2.0.0) implies it can use any version in that range. But that means if I am going to create a new version of the package, if I give it a value between 1.x and 2.x, I have to kind of ensure that the dependent code can work. If I have removed classes, removed methods, etc and I call my version 1.1, the component that depends on [1.0.0,2.0.0) is just going to fail miserably. So my next release with all of those breaking changes needs to be 2.0.0 so components won't resolve with the new version.

I know what you're thinking, because I was thinking it too. What a pain it is going to be, reviewing all of the code changes in a package to determine if the version number I want to apply to the package is actually one that would stay in the 1.x range or whether it should flip over to the 2.x range (or beyond).

Well, it would be a pain, but OSGi has tooling to help with this...

BND Baselines to the Rescue

So I first introduced bnd and its useful print tool in my blog post, Finding Bundle Dependencies, to help see what packages our bundles were going to export but more importantly what they were going to import. This would help us deal with missing dependencies before we deployed and had to deal with the "unresolved references" errors during bundle start.

It turns out that bnd has another useful tool, baseline.

This tool is really helpful when it comes to figuring out your package version issues because it can actually compare packages and make recommendations!

So I created a simple Liferay Gradle workspace, added a single module to it, an API module named "baseline" where I added an interface. The package was com.dnebinger.baseline.api and the bundle and package version was 1.0.0.

Then I added a new method to my interface, changed the package and bundle versions to 1.1.0 and ran the bnd baseline tool:

$ bnd baseline -a build/libs/com.dnebinger.baseline.api-1.1.0.jar build/libs/com.dnebinger.baseline.api-1.0.0.jar 
===============================================================
* com.dnebinger.baseline.api 1.1.0-1.0.0 suggests 2.0.0
===============================================================
  Package                                            Delta      New        Old        Suggest    If Prov.  
* com.dnebinger.baseline.api                         MAJOR      1.1.0      1.0.0      2.0.0      1.0.0

Wow, it didn't like my 1.1.0 version and it thinks I should have made the package 2.0.0.

And, when you think about it, it makes perfect sense. I added a method to an interface, so any other component which depended on that interface is now broken.

I'll reset the package and bundle versions to 2.0.0 and run again:

$ bnd baseline -a build/libs/com.dnebinger.baseline.api-2.0.0.jar build/libs/com.dnebinger.baseline.api-1.0.0.jar 
===============================================================
  com.dnebinger.baseline.api 2.0.0-1.0.0
===============================================================
  Package                                            Delta      New        Old        Suggest    If Prov.  
  com.dnebinger.baseline.api                         MAJOR      2.0.0      1.0.0      ok         -

Now it is happy that the version is 2.0.0.

I can add a new interface, and now I'm going to say that this is version 2.0.1:

$ bnd baseline build/libs/com.dnebinger.baseline.api-2.0.1.jar build/libs/com.dnebinger.baseline.api-2.0.0.jar 
===============================================================
* com.dnebinger.baseline.api 2.0.1-2.0.0 suggests 2.1.0
===============================================================
  Package                                            Delta      New        Old        Suggest    If Prov.  
* com.dnebinger.baseline.api                         MINOR      2.0.1      2.0.0      2.1.0      -

So the new interface is not a major change, but it is a minor one and semantic versioning would suggest that the new package version number should be 2.1.0.

This too makes sense. If a component wanted to depend upon this new interface, this would have been no small addition to the module. Neither is it a major change since the existing API is unchanged for previous consumers.

Now we can use the "bnd baseline" command to verify our packages have the correct versions relative to contained changes.

Build Time Version Verification

So we know now why we have to version packages and we know how to see what our version numbers should be, but we still have to acknowledge that, like updating javadoc, there are times when this will not be properly done by developers.

To counter this, we can add baselining into our build processes so we can verify that packages are versioned correctly. Invoke these in the CI build and, after your developer's commits fail because they forgot to version the packages a few times, they'll quickly get onboard with maintaining those version numbers.

To add baseline verification to your Gradle builds, check out https://dev.liferay.com/fr/develop/reference/-/knowledge_base/7-1/baseline-gradle-plugin.

To add baseline validation to your Maven builds, check back soon. It is in the pipeline, just isn't ready to go just yet, but should be soon.

 

Blogs

Hey David! This post completely blow my mind! ;) You, obviously, are right; thank you for pointing us the "problem" out! ;)

Hi David,

thanks for the post! Very useful indeed and it seems developers are the same all around the world :)

One question maybe: couldn't we use the bnd tool to automatically put the version? The real question behind this is: is the tool trustable enough so that we can skip the versioning decision and leave it to the tool?

 

Laurent

The tool is solid, but it is not a code analysis tool.  It really doesn't know what class changes have gone on in a package from build to build and won't know what the versions should be.  It also wouldn't be able to distinguish between a minor version bump and a major version bump...

 

I admit that I am terrible at package versioning. I religiously maintain javadocs on the classes and methods that I write, but I will totally forget to go back and update the package versions...  I wish there was an easy way to do it, but so far the easy way has escaped me...