Liferay's performance & users in LR databaseLiferay's performance & users in LR databasehttps://liferay.dev/en/c/message_boards/find_thread?p_l_id=119785333&threadId=747247462024-03-28T23:11:49Z2024-03-28T23:11:49ZRE: Liferay's performance & users in LR databaseDavid H Nebingerhttps://liferay.dev/en/c/message_boards/find_message?p_l_id=119785333&messageId=747849872016-05-31T19:34:24Z2016-05-31T19:34:24Z<div class="quote-title">Nick Monteverdi:</div><blockquote>I've been POCcin' Liferay 6.2 CE GA6 with some load testing since I want to investigate that can Liferay handle big enough user database or not.</blockquote><br /><br />Welcome Nick.<br /><br /><blockquote>We are talking about way over 100 000 users that should be added as Liferay users.</blockquote><br /><br />Not a problem. I've been in LR systems with over 10 times as many users, but you must be structured to handle that kind of load.<br /><br /><blockquote>I have been testing in an environment that is not production-like environment but the environment is still quite robust (two webservers, two application servers, an Oracle database server that has quite a lot of beef). I have noticed that when I start adding the users to liferay database; either with the WS-interface (/api/axis/Portal_UserService -> addUser) or by logging in with new user via Liferay UI the performance declines quite rapidly.</blockquote><br /><br />So first thing is whether you have externalized search yet. If you're still using lucene and clusterlink, that's going to be your initial problem. Externalizing to solr will offload the indexing of users out of Liferay and will also ensure that you are not maintaining multiple index copies.<br /><br /><blockquote>When the user_ -table is empty the response times are about 300ms (I am adding new users to Liferay database approx. 3-4 new users / sec). And on 100 000 users the response times with same load are close to 10 seconds.</blockquote><br /><br />This is not normal. I'd fire up Oracle and analyze what's going on, see where the bottlenecks are and eliminate them. Could be something as simple as an index issue or perhaps you need to analyze the tables or ...<br /><br /><blockquote>1a. how many users CAN be added to Liferay database?<br />1b. 1a + what is a "sensible amount" of users? 10? 100? 1000? 10 000?</blockquote><br /><br />I've seen systems with multi-million users.<br /><br /><blockquote>2. Should Liferay be used in this kind of way that there are, for example, 500 000 users in Liferay's user database? Or... should Liferay only keep n amount of "higher level users" and in this case where there are hundreds of thousands users, these should be handled in a different way?</blockquote><br /><br />A user account is necessary if special permissions are required. If your 500k users are just browsing public data, they don't need separate accounts for that. However, if your 500k users are going to have a personalized experience, then they would need separate accounts.<br /><br /><blockquote>3. Is there any easier way to create "bulk data" to Liferay's user database? I have noticed that if you just insert customer with SQL to the USER_ -table it kinda messes everything up. It seems that the user needs to be created in proper way (either with WebService-request (addUser) or by logging in as a new user in Liferay)<br />3b. same goes with deleting users... can't do it with delete from -sql to USER_ -table</blockquote><br /><br />The only way to access Liferay data is through the API. You cannot and should not ever be in the database itself. Users, for example, are also kept in the search index (lucene or solr) and Liferay goes there first; modifying the DB invalidates your indexes and leads to all kinds of bad outcomes.<br /><br /><blockquote>Like I said earlier... I am doing a proof of concept for Liferay and the emphasis on this POC is performance. I know that Liferay 7 is out already and CE should not be used in production but I have to start from somewhere. That somewhere now is LR 6.2 CE GA6.</blockquote><br /><br />Tuning for performance is not a matter of just flipping some switches. You have to know going in what you are going to do and you're going to have to test and tune to get the desired outcome.<br /><br />The first step is architecture. Up front you want to have some sort of caching appliance (such as Varnish) to reduce the hits to tomcat. In the middle, you want to do everything you can to offload work from the app server - leverage external tools such as ghostscript, imagemagick, openoffice and solr. On the back end, you're going to analyze the database to see what it reports where time is being spent and you're going to make necessary changes to eliminate the bottlenecks (likely adding indexes, tunings, etc.).<br /><br /><blockquote>Oh... I have tried quite a lot of JVM tuning based on the Liferay white-papers etc but I think that the bottleneck for adding new users to Liferay database is not there. Would be very interesting to know where the bottleneck is and is this a common issue with Liferay - that the user database cannot support thousands and thousands of users?</blockquote><br /><br />No this is not normal, but you have to peel back all of the layers of your infrastructure to see where the problems lie. Perhaps you have a bit of latency between the app server and the database. Perhaps the database needs to be analyzed to rebalance the indexes. Perhaps lucene is taking some cycles to update the indexes. But look farther than just JVM tuning, that's just the starting point.David H Nebinger2016-05-31T19:34:24ZLiferay's performance & users in LR databaseNick Monteverdihttps://liferay.dev/en/c/message_boards/find_message?p_l_id=119785333&messageId=747247452016-05-30T08:17:11Z2016-05-30T08:17:11ZHi,<br /><br />I've been POCcin' Liferay 6.2 CE GA6 with some load testing since I want to investigate that can Liferay handle big enough user database or not.<br /><br />We are talking about way over 100 000 users that should be added as Liferay users.<br /><br />I have been testing in an environment that is not production-like environment but the environment is still quite robust (two webservers, two application servers, an Oracle database server that has quite a lot of beef). I have noticed that when I start adding the users to liferay database; either with the WS-interface (/api/axis/Portal_UserService -> addUser) or by logging in with new user via Liferay UI the performance declines quite rapidly.<br /><br />When the user_ -table is empty the response times are about 300ms (I am adding new users to Liferay database approx. 3-4 new users / sec). And on 100 000 users the response times with same load are close to 10 seconds.<br /><br />My questions that I want to ask are:<br />1a. how many users CAN be added to Liferay database?<br />1b. 1a + what is a "sensible amount" of users? 10? 100? 1000? 10 000?<br />2. Should Liferay be used in this kind of way that there are, for example, 500 000 users in Liferay's user database? Or... should Liferay only keep n amount of "higher level users" and in this case where there are hundreds of thousands users, these should be handled in a different way?<br />3. Is there any easier way to create "bulk data" to Liferay's user database? I have noticed that if you just insert customer with SQL to the USER_ -table it kinda messes everything up. It seems that the user needs to be created in proper way (either with WebService-request (addUser) or by logging in as a new user in Liferay)<br />3b. same goes with deleting users... can't do it with delete from -sql to USER_ -table<br /><br />Like I said earlier... I am doing a proof of concept for Liferay and the emphasis on this POC is performance. I know that Liferay 7 is out already and CE should not be used in production but I have to start from somewhere. That somewhere now is LR 6.2 CE GA6.<br /><br />Oh... I have tried quite a lot of JVM tuning based on the Liferay white-papers etc but I think that the bottleneck for adding new users to Liferay database is not there. Would be very interesting to know where the bottleneck is and is this a common issue with Liferay