Liferay 7.2 SystemCheckers

Check your Liferay Environment at Startup Time

Recently I was one of many trying to help a coworker figure out why a Liferay OOTB feature was failing...

We had odd stack traces, null object references when OSGi never should have allowed a null to begin with, and nothing we tried would fix the problem.

Eventually we found that, for some reason, two different versions of a Liferay bundle had been deployed into the environment. As soon as the older one was taken out and the environment restarted, everything started working again.

My friend Ray Augé suggested that we really should have a SystemChecker in place to watch for this kind of situation, and that suggestion has lead to this blog and an attached Github repo!

SystemCheckers

In Liferay 7.2, Liferay created a new API called SystemChecker. One of the issues folks sometimes faced on previous versions is that when an environment started, they might have a false impression that everything was fine because there were no errors generated into the logs.

Meanwhile there might be DS services (those @Component classes) that maybe didn't start because they have an unsatisfied @Reference. Or maybe there was a circular reference, where @Component A had an @Reference on B, but @Component B also had an @Reference on A so neither could start.

Liferay created the SystemChecker interface to support a final stage of startup testing to verify an environment is in proper working order. Yours is if you've seen the following messages in your logs:

2019-08-27 05:11:26.483 INFO  [main][SystemCheckOSGiCommands:54] System check is enabled. You can run a system check with the command "system:check" in Gogo shell.
2019-08-27 05:11:26.483 INFO  [main][SystemCheckOSGiCommands:64] Running system check
2019-08-27 05:11:26.484 INFO  [main][SystemCheckOSGiCommands:95] Available checkers :[Declarative Service Soft Circular Dependency Checker, Declarative Service Unsatisfied Component Checker, Spring Extender Unavailable Component Checker]
2019-08-27 05:11:26.484 INFO  [main][SystemCheckOSGiCommands:111] Running "Declarative Service Soft Circular Dependency Checker". You can run this by itself with command "ds:softCircularDependency" in gogo shell.
2019-08-27 05:11:27.345 INFO  [main][SystemCheckOSGiCommands:123] Declarative Service Soft Circular Dependency Checker check result: No issues were found.
2019-08-27 05:11:27.346 INFO  [main][SystemCheckOSGiCommands:111] Running "Declarative Service Unsatisfied Component Checker". You can run this by itself with command "ds:unsatisfied" in gogo shell.
2019-08-27 05:11:32.165 INFO  [main][SystemCheckOSGiCommands:123] Declarative Service Unsatisfied Component Checker check result: No issues were found.
2019-08-27 05:11:32.166 INFO  [main][SystemCheckOSGiCommands:111] Running "Spring Extender Unavailable Component Checker". You can run this by itself with command "dm na" in gogo shell.
2019-08-27 05:11:32.170 INFO  [main][SystemCheckOSGiCommands:123] Spring Extender Unavailable Component Checker check result: No issues were found.

If you had any one of those nasty issues I talked about above, well then your startup logs are going to have a bunch of warnings that the above log sample does not.

The SystemCheckers will output warning message that include things like the Bundle Symbolic Name (BSN), bundle version numbers, component names, etc. Basically any info that would be useful in diagnosing and resolving a potential problem.

Note here that I said a potential problem. The warnings themselves are indicators that they could be problems, but they might just be false positives. I mean, maybe you want to have an @Component with an unresolved @Reference, perhaps by using a reference to a class you've blacklisted. If the reference was satisfied, that would be an indicator that the component you wanted to blacklist is no longer blacklisted.

Who knows? Well, you do of course, but these SystemChecker implementations do not. They'll report what are commonly considered issues, it will be up to you then to take action on them or ignore them, whatever is appropriate in your environment.

Duplicate BSN SystemChecker

So, given what we now know about SystemCheckers, I went ahead and actually created a SystemChecker implementation that checks for duplicate BSNs.

The SystemChecker interface is super simple to implement; there's a getName() method, a getOSGiCommand() method to return the command string, and a check() method that does the work and returns a String of what it might have found.

Nothing too fancy, but it does show you how the module is structured and what they normally contain. I followed Liferay's example and also made a Gogo command for the same SystemChecker functionality, so it too can be invoked from the Gogo shell.

Now, when I start up my environment, I now see:

2019-08-27 05:39:41.314 INFO  [main][SystemCheckOSGiCommands:54] System check is enabled. You can run a system check with the command "system:check" in Gogo shell.
2019-08-27 05:39:41.315 INFO  [main][SystemCheckOSGiCommands:64] Running system check
2019-08-27 05:39:41.316 INFO  [main][SystemCheckOSGiCommands:95] Available checkers :[Declarative Service Soft Circular Dependency Checker, Declarative Service Unsatisfied Component Checker, Spring Extender Unavailable Component Checker, Duplicate Bundle Symbolic Name Checker]
2019-08-27 05:39:41.316 INFO  [main][SystemCheckOSGiCommands:111] Running "Declarative Service Soft Circular Dependency Checker". You can run this by itself with command "ds:softCircularDependency" in gogo shell.
2019-08-27 05:39:42.118 INFO  [main][SystemCheckOSGiCommands:123] Declarative Service Soft Circular Dependency Checker check result: No issues were found.
2019-08-27 05:39:42.119 INFO  [main][SystemCheckOSGiCommands:111] Running "Declarative Service Unsatisfied Component Checker". You can run this by itself with command "ds:unsatisfied" in gogo shell.
2019-08-27 05:39:46.833 INFO  [main][SystemCheckOSGiCommands:123] Declarative Service Unsatisfied Component Checker check result: No issues were found.
2019-08-27 05:39:46.833 INFO  [main][SystemCheckOSGiCommands:111] Running "Spring Extender Unavailable Component Checker". You can run this by itself with command "dm na" in gogo shell.
2019-08-27 05:39:46.834 INFO  [main][SystemCheckOSGiCommands:123] Spring Extender Unavailable Component Checker check result: No issues were found.
2019-08-27 05:39:46.835 INFO  [main][SystemCheckOSGiCommands:111] Running "Duplicate Bundle Symbolic Name Checker". You can run this by itself with command "osgi:duplicatebsns" in gogo shell.
2019-08-27 05:39:46.874 WARN  [main][SystemCheckOSGiCommands:134] Duplicate Bundle Symbolic Name Checker check result: _Bundle {id: 297, name: javax.validation.api, version: 2.0.0.Final}_is duplicate of_	Bundle {id: 751, name: javax.validation.api, version: 2.0.1.Final} [Sanitized]

Wait, what? Why am I seeing a warning on my spiffy new SystemChecker?

The logger likes to replace newlines with underscores, so let's go to the Gogo command shell and check our output there:

Yep, same as the log message.

Apparently Liferay 7.2 already has a duplicate of the validation API jar.

So remember when I was talking about false positives? This one might actually be such a case. Maybe it is not. I'm not sure, but now I can go tell Liferay about it...

Conclusion

So here's a new tool you can add to your 7.2 environment. It will report to you all of the duplicate BSNs you might have and allow you to clean them up if you want.

It also demonstrates how you too could build a SystemChecker to do, well, whatever checking your particular environment needs to ensure it is ready to go... You could check for the existence of a file, connectivity to a service you are integrating with, space on the filesystem, who knows?

So here's the Github link: https://github.com/dnebing/portal-osgi-debug-duplicate-bsn

Enjoy!