RE: Table already exists (Service Builder)

Christopher Virtucio, modified 7 Years ago. Junior Member Posts: 33 Join Date: 9/26/17 Recent Posts

I'm migrating a service-builder portlet from 6.2 to DXP. I've managed to convert the project into a bundle, and get a "<name of table> already exists" error whenever I deploy it, resulting in none of the exported services getting registered. How do I prevent Liferay from trying to recreate the table during a service-builder bundle deployment?

thumbnail
Minhchau Dang, modified 7 Years ago. Liferay Master Posts: 598 Join Date: 10/22/07 Recent Posts
Christopher Virtucio:

I'm migrating a service-builder portlet from 6.2 to DXP. I've managed to convert the project into a bundle, and get a "<name of table> already exists" error whenever I deploy it, resulting in none of the exported services getting registered. How do I prevent Liferay from trying to recreate the table during a service-builder bundle deployment?

I believe Liferay will only create the tables if it thinks the version of your service is 0.0.0 (there's an implicit upgrade step that starts from version 0.0.0 that gets added by ModuleApplicationContextExtender to create the schema).

When migrating from an earlier version of Liferay, this can happen if you didn't follow all of the instructions on the Upgrade Processes for Former Service Builder Plugins tutorial.

Christopher Virtucio, modified 7 Years ago. Junior Member Posts: 33 Join Date: 9/26/17 Recent Posts
Minhchau Dang:
Christopher Virtucio:

I'm migrating a service-builder portlet from 6.2 to DXP. I've managed to convert the project into a bundle, and get a "<name of table> already exists" error whenever I deploy it, resulting in none of the exported services getting registered. How do I prevent Liferay from trying to recreate the table during a service-builder bundle deployment?

I believe Liferay will only create the tables if it thinks the version of your service is 0.0.0 (there's an implicit upgrade step that starts from version 0.0.0 that gets added by ModuleApplicationContextExtender to create the schema).

When migrating from an earlier version of Liferay, this can happen if you didn't follow all of the instructions on the Upgrade Processes for Former Service Builder Plugins tutorial.

Thanks; it seems like creating the Activator stopped Liferay from trying to create the tables, but it still prevents the services from being registered. I got past it though by creating a Data Upgrade Registrator. The step I register, however, is a Dummy Upgrade Step. Is an upgrade step really necessary to get my services registered? If it is, a dummy step should be perfectly fine, right? Although the documentation advises against using a dummy step, it refers specifically to the initial registration (which the service-builder code already takes care of):

Important: Modules that use Service Builder should not define a registration for their initial database schema version, as Service Builder already records their schema versions to Liferay DXP’s Release_ table. Modules that don’t use Service Builder, however, should define a registration for their initial schema.

thumbnail
Minhchau Dang, modified 7 Years ago. Liferay Master Posts: 598 Join Date: 10/22/07 Recent Posts
Christopher Virtucio:

Is an upgrade step really necessary to get my services registered? If it is, a dummy step should be perfectly fine, right?

Yes, adding a Liferay-Require-SchemaVersion and providing the appropriate upgrade step are both necessary when migrating between 6.x and 7.x. If there are no schema changes between your service builder service in 6.x and your service builder service in 7.x, then a dummy step should be perfectly fine.

Christopher Virtucio, modified 7 Years ago. Junior Member Posts: 33 Join Date: 9/26/17 Recent Posts
Minhchau Dang:
Christopher Virtucio:

Is an upgrade step really necessary to get my services registered? If it is, a dummy step should be perfectly fine, right?

Yes, adding a Liferay-Require-SchemaVersion and providing the appropriate upgrade step are both necessary when migrating between 6.x and 7.x. If there are no schema changes between your service builder service in 6.x and your service builder service in 7.x, then a dummy step should be perfectly fine.

 

I did a sanity-check and apparently only my Activator gets registered as a component. The others were just services declared as in use. Shouldn't all my services from the service-builder bundle get registered automatically after a proper upgrade?

 

 

thumbnail
Minhchau Dang, modified 7 Years ago. Liferay Master Posts: 598 Join Date: 10/22/07 Recent Posts
Christopher Virtucio:

I did a sanity-check and apparently only my Activator gets registered as a component. The others were just services declared as in use. Shouldn't all my services from the service-builder bundle get registered automatically after a proper upgrade?

Service builder services are registered as components through Felix Dependency Manager, so they're slightly different from regular OSGi components that have the @Component annotation. However, they should still be treated as a component in the very general sense, in terms of being used to satisfy @Reference and for ServiceReference lookups.

Christopher Virtucio, modified 7 Years ago. Junior Member Posts: 33 Join Date: 9/26/17 Recent Posts
Minhchau Dang:
Christopher Virtucio:

I did a sanity-check and apparently only my Activator gets registered as a component. The others were just services declared as in use. Shouldn't all my services from the service-builder bundle get registered automatically after a proper upgrade?

Service builder services are registered as components through Felix Dependency Manager, so they're slightly different from regular OSGi components that have the @Component annotation. However, they should still be treated as a component in the very general sense, in terms of being used to satisfy @Reference and for ServiceReference lookups.

Okay, so does this mean I have to register the services myself, whether on the ServiceActivator or with @Component? I thought service-builder does all of this for you.

 

 

 

 

thumbnail
Minhchau Dang, modified 7 Years ago. Liferay Master Posts: 598 Join Date: 10/22/07 Recent Posts
Christopher Virtucio:

Okay, so does this mean I have to register the services myself, whether on the ServiceActivator or with @Component? I thought service-builder does all of this for you.

No, you shouldn't need to add services yourself. Adding things to the upgrade registry requires some kind of activator, but once this is done, and all of the required pieces are in your .jar manifest, then service builder will register all the services for you.

Of course, to be honest, "all of the required pieces" is one of the nastier parts of the 7.x learning curve. It requires you to be conscious of the underlying Liferay implementation details, and to be conscious that OSGi is involved, which wasn't really necessary in the 6.x releases.

In 6.2, Liferay wouldn't do anything for you at runtime, unless certain information was available inside of web.xml. Liferay hid this by having what it referred to as "hot deploy", which was really just Liferay modifying the web.xml before dropping it inside of a place where the application server would read the .war. Those modifications to web.xml would then cause the application server to tell Liferay that something needed to be done. This provided the illusion that you just deployed, and everything worked.

In 7.0, Liferay again won't do anything for you at runtime, unless certain information is inside of the .jar manifest. This is where the learning curve comes in. Liferay no longer automatically does anything at deployment time; it simply moves the file to some folder based on its extension. Plainly speaking, because the .jar isn't touched at all, the .jar must be perfect at deployment time.

This means that Liferay relies on a lot of the necessary stuff to be done at build time, but this will only happen if you use Gradle with the Liferay plugins. This won't be automatic in Maven, for example, which is why the Blade samples for Maven and Gradle for service builder have different bnd.bnd files (the bnd.bnd files essentially get copied into the .jar manifest). The added wrinkle is all the tutorials probably assume Gradle.

One of the things that must be added is a Liferay-Spring-Context, which is a signal that tells Liferay that there are Spring components that it should register with Felix Dependency Manager.

Even then, you're not quite done. Assuming this signal is there, service builder will register all the service builder related Spring-managed components (such as the implementations of LocalService classes) only after all of the dependencies are met. It's all or nothing.

One of the dependencies is an up-to-date value in the Release_ table, where up to date is defined as whatever you have set in the Liferay-Require-SchemaVersion in your bnd.bnd.

When migrating Liferay from 6.x to 7.x, in order to have a correct value in the Release_ table, you need to write upgrade processes that get registered with the upgrade registry to take you through the proper versions. If the versions are incompatible, then Liferay believes that your bundle isn't fully upgraded and therefore it is unsafe to enable any of your bundle's Spring-related components.

thumbnail
Mirto Silvio Busico, modified 7 Years ago. Regular Member Posts: 240 Join Date: 1/18/12 Recent Posts

Hi,

is it possible to have a demo code of a service builder example?

I mean to be able to:

  1. download the code into a liferay workspace
  2. deploy and verify that everything is deployed and working
  3. change something in an entity and deploy
  4. verify that the schema is updated and everything is deployed and working

Thanks

thumbnail
Minhchau Dang, modified 7 Years ago. Liferay Master Posts: 598 Join Date: 10/22/07 Recent Posts
Mirto Silvio Busico:

is it possible to have a demo code of a service builder example?

Have you already tried the demo from liferay-blade-samples?

thumbnail
Mirto Silvio Busico, modified 7 Years ago. Regular Member Posts: 240 Join Date: 1/18/12 Recent Posts

Well, I've tried this sample.

The first try is working: I deployed basic-api, basic-service and basic-web

Everything worked as expected and I can insert /update records.

Then I tried to add a field in the foo entity (i.e. field6 of tipe String) updating accordingly the view and the portlet.

Strangely in FooLocalServiceImpl.java there is no reference to the field getter/setter that I expected to find in order to add the new field (I used ad a guide the documentation at https://dev.liferay.com/de/develop/tutorials/-/knowledge_base/7-0/developing-a-web-application - strangely this chapter disappeared in 7.1 documentation).

Trying to deploy basic-service gives this message

2018-08-28 09:15:04.114 INFO &nbsp;[Thread-206][BundleStartStopLogger:38] STOPPED com.liferay.blade.basic.service_1.0.0 [945]<br> 2018-08-28 09:15:04.657 WARN &nbsp;[Thread-212][ServiceComponentLocalServiceImpl:618] Auto upgrading SSB database to build number 70 is not supported for a production environment. Write an UpgradeStep to ensure data is upgraded correctly.<br> 2018-08-28 09:15:04.806 INFO &nbsp;[Thread-212][BundleStartStopLogger:35] STARTED com.liferay.blade.basic.service_1.0.0 [945]

Seems that the schema is updated but trying to add a record gives an internal error saying that the new field is unknown:

2018-08-28 08:43:44.365 ERROR [http-nio-8080-exec-6][BeanPropertiesImpl:417] jodd.bean.BeanException: Simple property not found: field6. Invalid property: $Proxy973#field6 ($Proxy973#field6, forced=false)<br> jodd.bean.BeanException: Simple property not found: field6. Invalid property: $Proxy973#field6 ($Proxy973#field6, forced=false)<br> &nbsp;&nbsp; &nbsp;at jodd.bean.BeanUtilBean.getSimpleProperty(BeanUtilBean.java:154)<br> &nbsp;&nbsp; &nbsp;at jodd.bean.BeanUtilBean.getIndexProperty(BeanUtilBean.java:246)

 

For now I'm not able to change a service builder entity and redeploy correctly.

 

So my question is: where can I find a working  example of (at least) a dummy upgrade step?

 

(btw my situation i still the one in this thread https://community.liferay.com/forums/-/message_boards/message/110340826 )

 

 

 

 

Christopher Virtucio, modified 7 Years ago. Junior Member Posts: 33 Join Date: 9/26/17 Recent Posts

Thank you, this is good info. I have an upgrade process ready that makes the inserts on the Release_ table. My services are all registered now, but now I'm running into a different issue (I'll have to create a separate thread for that).

thumbnail
Mirto Silvio Busico, modified 7 Years ago. Regular Member Posts: 240 Join Date: 1/18/12 Recent Posts

I started again with a fresh installation of LR 7.1 CE GA 1 and MySQL.

Having in portal.ext.properties 

include-and-override=portal-developer.properties

I can confirm that the adding of a new field works and the database table is changed also if there is the "table already exist" error:

2018-08-30 09:47:55.754 INFO  [Thread-181][BundleStartStopLogger:38] STOPPED msb.db.service_1.0.0 [943]
2018-08-30 09:47:56.256 WARN  [Thread-187][ServiceComponentLocalServiceImpl:618] Auto upgrading FOO database to build number 5 is not supported for a production environment. Write an UpgradeStep to ensure data is upgraded correctly.
2018-08-30 09:47:56.259 INFO  [Thread-187][ServiceComponentLocalServiceImpl:623] Upgrading database with tables.sql
2018-08-30 09:47:56.285 WARN  [Thread-187][BaseDB:484] Table 'foo_foo' already exists: create table FOO_Foo (_uuid_ varchar(75) null,_fooId bigint not null primary key,_groupId bigint,_companyId bigint,_userId bigint,_userName varchar(75) null,_createDate datetime(6) null,_modifiedDate datetime(6) null,_field1 varchar(75) null,_field2 tinyint,_field3 integer,_field4 datetime(6) null,_field5 varchar(75) null,_field6 varchar(75) null_) engine InnoDB;_ [Sanitized]
2018-08-30 09:47:56.287 INFO  [Thread-187][ServiceComponentLocalServiceImpl:424] No optional file META-INF/portlet-model-hints-ext.xml found
2018-08-30 09:47:56.343 INFO  [Thread-187][Table:168] Starting backup of FOO_Foo to C:\appoggio\liferay\bundles\mio71-1\tomcat-9.0.6\temp\temp-db-FOO_Foo-2050241993252037562.tmp
2018-08-30 09:47:56.365 INFO  [Thread-187][Table:204] Finished backup of FOO_Foo to C:\appoggio\liferay\bundles\mio71-1\tomcat-9.0.6\temp\temp-db-FOO_Foo-2050241993252037562.tmp in 22 ms
2018-08-30 09:47:56.761 INFO  [Thread-187][ServiceComponentLocalServiceImpl:645] Upgrading database with indexes.sql
2018-08-30 09:47:57.293 INFO  [Thread-187][BundleStartStopLogger:35] STARTED msb.db.service_1.0.0 [943]

Still I don't know how to create an upgradeStep.

Please can you share your upgrade code and directory structure?

thumbnail
Alberto Chaparro, modified 7 Years ago. Liferay Master Posts: 560 Join Date: 4/25/11 Recent Posts

Hi Mirto,

 

If you already handled the modification of your tables in your database (create upgrade steps for that https://dev.liferay.com/en/develop/tutorials/-/knowledge_base/7-0/creating-an-upgrade-process-for-your-app), maybe you should set the following property:

schema.module.build.auto.upgrade=false

To avoid that the system tries to recreate the table again.

Let me know if it works.

thumbnail
Mirto Silvio Busico, modified 7 Years ago. Regular Member Posts: 240 Join Date: 1/18/12 Recent Posts

Sorry, mybe I was not clear.

I need instructions on how to write the upgrade code.

The instructions at https://dev.liferay.com/en/develop/tutorials/-/knowledge_base/7-0/creating-an-upgrade-process-for-your-app for me are difficult to follow. Maybe an example is better than one thousand words :-) 

Thanks

thumbnail
Alberto Chaparro, modified 7 Years ago. Liferay Master Posts: 560 Join Date: 4/25/11 Recent Posts

Here you have an example about it:

https://github.com/liferay/liferay-portal/blob/master/modules/apps/wiki/wiki-service/src/main/java/com/liferay/wiki/internal/upgrade/WikiServiceUpgrade.java

 

Starts from 0.0.1 if you come from a plugin in previous versions or from the latest schema version if you have already created the tables in the database.

thumbnail
Mirto Silvio Busico, modified 7 Years ago. Regular Member Posts: 240 Join Date: 1/18/12 Recent Posts

Thanks. I'll try asap.

thumbnail
Mirto Silvio Busico, modified 7 Years ago. Regular Member Posts: 240 Join Date: 1/18/12 Recent Posts

Only a little question.

In https://github.com/liferay/liferay-portal/blob/master/modules/apps/wiki/wiki-service/src/main/java/com/liferay/wiki/internal/upgrade/v1_0_0/UpgradeSchema.java

I see

    protected void doUpgrade() throws Exception {
        String template = StringUtil.read(
            UpgradeSchema.class.getResourceAsStream("dependencies/update.sql"));

        runSQLTemplateString(template, false, false);
    }

 

but I was not able to find neither the dependecies directory nor the update.sql file.

The getResourceAsStream which path receive? 

thumbnail
Minhchau Dang, modified 7 Years ago. Liferay Master Posts: 598 Join Date: 10/22/07 Recent Posts
Mirto Silvio Busico:

I was not able to find neither the dependecies directory nor the update.sql file.

Because they're not .java files, convention is that they're found in src/main/resources. Additionally, because it's a relative path (no leading slash), it will be relative to the modified package path (in this case, the package is com.liferay.wiki.internal.upgrade.v1_0_0, so the modified package path is com/liferay/wiki/internal/upgrade/v1_0_0), as described in the API documentation for Class.getResourceAsStream.

thumbnail
Mirto Silvio Busico, modified 7 Years ago. Regular Member Posts: 240 Join Date: 1/18/12 Recent Posts

Thanks