Question marks appear in WebContent

Message Boards

André Bräkling, modified 3 Years ago.

Question marks appear in WebContent

Junior Member Posts: 30 Join Date: 7/8/13 Recent Posts

Hello everybody.

We currently observe a strange issue in our Liferay 7.1 DXP live system. Sadly, we can't tell you when it started to occur (e.g., if it happens since an update or specific changes in our settings, etc.), and there are no related hints in our log files. But maybe somebody observed similar issues and can give a hint.If new Web Contents are published, suddenly multiple question marks appear within the text.

The behavior seems to be random. Please see the attached image to get an idea on how this looks like.

We did not change any encoding settings or something similar. The question marks appear additional to the existing text, they don't replace other characters.

Any help would be appreciated. Thanks in advance!

André

Christoph Rabel, modified 3 Years ago.

RE: Question marks appear in WebContent

Liferay Legend Posts: 1554 Join Date: 9/24/09 Recent Posts

Could you check the article content? I guess, there are special characters inside. No idea, where they come from, but this looks like characters in some weird encoding which cannot be displayed.

Olaf Kock, modified 3 Years ago.

RE: Question marks appear in WebContent

Liferay Legend Posts: 6400 Join Date: 9/23/08 Recent Posts

Indeed, my first bet is "encoding". If it's the appserver, database, webserver, I can't tell.
Easiest to tackle: Configure UTF-8 all-the-way, everywhere. Everywhere. Anything else is a mess that will sooner or later bite you (or have bitten you, as it seems in this case)

André Bräkling, modified 3 Years ago.

RE: Question marks appear in WebContent

Junior Member Posts: 30 Join Date: 7/8/13 Recent Posts

Good morning everybody.

Thanks for your responses.I just had to wait for the issue to appear again (my content editing colleagues took the screenshot and then removed the corrupted entry), and now I had the chance to check the content.

The question marks are actually stored as question marks - I not just checked the frontend, I also had a look at the database table. No reference to any special characters. The encoding was also my first thought, but it should be UTF-8 everywhere. The database and all tables are set to utf8mb4_unicode_ci. We did not do any (intended) changes there.

Afaik, the question marks don't appear as long as the article is just drafted. They occur if it's published or scheduled for publication. In the database I can see them "appearing" just between the two versions of the journal article. So I also don't think that my colleagues accidentally copy & paste some non printing chars like strange line breaks, tabs or whatever.

It looks like the question marks are added where Liferay adds HTML tags to the article source code (e.g., paragraphs).

I'll play around with articles and try to somehow reproduce it on my own, maybe I can at least figure out what's causing this behavior.

Christoph Rabel, modified 3 Years ago.

RE: Question marks appear in WebContent

Liferay Legend Posts: 1554 Join Date: 9/24/09 Recent Posts

I could be totally off here, but once upon a time I ran into a superweird issue with utf8 and Liferay. I am not sure if this has to do anything with your issue, but maybe it is related to it.
There was (still is?) an issue in MySQL. The utf8 collation was flawed and could actually not support UTF8. I ran into that with some smiley characters (or something like that) somebody copied into the text. It worked with the first save but when the article was saved again, the original unicode character was destroyed and some garbage (maybe ???) was inserted into the content.
I found several articles describing the issue. e.g. here:
https://stackoverflow.com/questions/766809/whats-the-difference-between-utf8-general-ci-and-utf8-unicode-ci
I tried then to change the collation to utf8mb4 but it actually made things worse because Liferay had lots of issues with it (I don't remember details, though). This was several years ago, I just wanted to mention even if I am unsure if it has anything to do with your problem.

André Bräkling, modified 3 Years ago.

RE: Question marks appear in WebContent

Junior Member Posts: 30 Join Date: 7/8/13 Recent Posts

Christoph Rabel:

I tried then to change the collation to utf8mb4 but it actually made things worse because Liferay had lots of issues with it (I don't remember details, though). This was several years ago, I just wanted to mention even if I am unsure if it has anything to do with your problem.

We did the change to utf8mb4 exact for the same reason: Some of our customers were using Emojis and running into problems with UTF8. But we switched to utf8mb4 more than a year ago and never encountered any issues - while the "question mark mystery" appeared the first time just a few weeks ago.
Beside the fact, that the question marks suddenly appear between two versions of an article, my colleagues also told me, that they also appear on "manually written" articles, i.e., texts they create within the WYSIWYG editor directly. I currenlty try to reproduce the issue on one of our test systems which is a clone of our live systems, but I still don't see any regularity after which the problem occurs.Btw. afaik the issue occurs on web contents, blog entries and calendar entries.