RE: How to keep development environment close to production environment?

thumbnail
Fredi B, modified 5 Years ago. Junior Member Posts: 69 Join Date: 4/1/20 Recent Posts
Hey Liferay-Community,
keeping up with production environment is hard for us.
We, as Developers, are developing with the official Liferay-Bundle locally and our production environment is also using the official Liferay-Bundle and are managed by DevOps.

Time goes by and at some point you have much more content and settings on the production environment than on the multiple developer environments.

What's the best way to keep the developer environments close to the production environments?

I would like to hear back from you guys.
As always: Thank You.

Cheers,
Fredi
thumbnail
Olaf Kock, modified 5 Years ago. Liferay Legend Posts: 6441 Join Date: 9/23/08 Recent Posts
Fredi B:

What's the best way to keep the developer environments close to the production environments?

It depends (TM)
If you refer to code: Typically Dev is ahead of Production
If you refer to content: Typically Dev has a lot of Lorem Ipsum, so it's IMHO rather unrelated (but read on)
If you refer to settings: You mention Devops, the settings there should be obvious, but then there are the little tweaks and switches that you use day by day when editing content. This is where the "it depends" comes in big time.
IMHO Dev-Content is fine with Lorem Ipsum. Test, Integration Test and similar environments might need to be closer to production content.

My favorite vocabulary definition is for Backup:
You're only allowed to call something a Backup, if you've recently demonstrated that you can use it to restore a fully functioning system to a completely different environment. Anything else is called "Some random copy of data that you hope to be useful in case of emergency".

With this in mind, you can cover the "recently demonstrated" part by using a Backup of your environment (that you have anyway, right?) to create a pre-prod environment and use it for testing.
Careful: Some industries have regulations about using production data in test environments - or rather about not using them. And you won't want to send any user of your system update mails coming from a test system - so you'll need to anonymize/pseudonymize or otherwise scramble the user data to make sure that this never happens, in addition to regulatory demands.

Side note: I also don't advocate taking a backup before upgrades, instead I recommend to restore an existing backup to a separate system and upgrade it, while the main site continues to run.

And for the directional flow of content between the different systems, see David's ancient but still relevant article Content Creation is not a Development Activity.
thumbnail
Christoph Rabel, modified 5 Years ago. Liferay Legend Posts: 1555 Join Date: 9/24/09 Recent Posts
We usually start with our dev server and clone everything there and do some cleanup there like removing users and deleting huge folders to save some space. That cleaned instance is then used for the developer machines.
For the database part:
We usually "restore a backup" here. For one or two years now we use docker for the database, it is otherwise annoying if you need local Oracle, MS SQL, MariaDB, ... instances. It is nice to encapsulate them in docker. Same with Elasticsearch.
It is a bit of work to get there, but once you have everything in place, you just download the db dump and execute a few commands. Done.
For Liferay, we just copy the Liferay folder from production, minus logfiles, osgi/state (I usually use rsync, Windows servers are very rare). Then we copy that folder contents to all dev machines into the bundles folder. Fix the paths and connection strings in portal-*.properties and that's it.
And reindex of course.
We do that regularly with integration environments, less often with the dev server and only if it is really necessary on the developer machines.
thumbnail
Krzysztof Gołębiowski, modified 5 Years ago. Liferay Master Posts: 549 Join Date: 6/25/11 Recent Posts
In my case, I usually do periodic synchronization between Production and Staging so the second is content-wise more or less up to date with production. It is a standard dump/restore procedure on database/application and is quite simple as there are usually just a few settings that have to be updated afterwards (site domain in Control Panel, integration endpoints etc.). What's more, I have most of my configuration scripted in Ansible, so if I can quite quickly spin up a full production environment on a local Vagrant machine.
This, of course, applies to sites focused on web content. In the case of portals that mainly run dedicated applications, a more classical development CI/CD process would fit better.
thumbnail
David H Nebinger, modified 5 Years ago. Liferay Legend Posts: 14933 Join Date: 9/2/06 Recent Posts
Actually the best way, IMHO, is to use the Liferay Workspace properly...

In the workspace, there's a configs directory where you'll find a number of subdirs. The common directory will have properties and files that would be shared amongst all environments. I typically create my portal-ext.properties here, then add an include-and-override for a portal-env.properties that will not be in this directory. Portal-ext will have all of the shared properties that will be the same in every environment, regardless of deployment.

Then in each of the appropriate subdirs like dev, uat and prod I'll put my portal-env.properties file. This is for the environment-specific properties such as my DB connection info (unless I'm using JNDI, but I'll still have for example a configs/dev/tomcat-9.0.33/conf/Catalina/localhost/ROOT.xml file where I'll define the dev connection, then repeat in the other environs).

I'll do the same kind of thing for osgi/configs where I'll put shared config files to the common folder but environment-specific ones into the different environ directories.

In this way I can keep all of my environments pretty much the same.

Combine this technique with https://liferay.dev/blogs/-/blogs/creating-a-custom-liferay-tarball for your custom Liferay tarball or, better yet, https://liferay.dev/blogs/-/blogs/liferay-workspace-distribution-bundles where you can build a complete tarball with not just your custom Liferay but also all of your modules deployed and you've got yourself an environment building machine!

Each environment config is used when building the separate tarballs so you get an environment-specific target, but it is based off of the same Liferay tarball and the same custom artifacts, so your custom workspace tarballs are all mirror copies of each other for the most part. You can't get better consistency than that...