Upgrading or moving Madrigal

With the release of Madrigal 2.5, the procedure to upgrade and/or move Madrigal to a new server was made simpler and less risky. To either upgrade Madrigal on your existing server, or to move it to another server, you now do a clean installation first. This clean installation will not effect your present Madrigal server. If the clean installation succeeds, then you migrate your Madrigal data and make the new server active. If the migration fails for any reason, it is simple to switch back to the original Madrigal server.

To determine what release of Madrigal you are presently running, see the title of the main documentation page (that's the page you get to by clicking on Documentation from the home page). The latest release of Madrigal will always be available from the OpenMadrigal web site via the Download/Update link.

This page covers the following administrative tasks:

Upgrading Madrigal (using the same server)
Moving Madrigal to a new server (installs the latest Madrigal version)

Three other less common procedures are also listed:

Moving the Madrigal madroot directory
Direct Madrigal upgrade (the old method - no longer recommended)

Upgrading Madrigal (using the same server)

To upgrade Madrigal to the latest release using the same server:

Create a new MADROOT directory for your new installation on your existing Madrigal server. After installation this will be your permanent MADROOT directory (unless you then follow the moving MADROOT procedure). Set your MADROOT environmental variable to this new directory.
Create two temporary directories on your existing Madrigal server for 1) html files and 2) cgi scripts. For example, if you standard Madrigal server stores its html files under /var/www/html/madrigal and its cgi files under /var/www/cgi-bin/madrigal, you could create the directories /var/www/html/madrigal2 and /var/www/cgi-bin/madrigal2.
From OpenMadrigal, download the Madrigal distribution file, madrigal*.tar.Z to your MADROOT directory.
From OpenMadrigal, download the sample experiment file, experiments.tar.Z to your MADROOT directory.
uncompress experiments.tar.Z and madrigal.tar.Z
tar -xf experiments.tar.
tar -xf madrigal*.tar.
Copy madrigal.cfg from the old MADROOT to the new MADROOT, and edit the following five fields:
1. MADROOT - set to new MADROOT
2. MADSERVERROOT - New, temporary url directory relative to MADSERVER for the temporary html files. This directory should not allow files to be executed. For the "/var/www/html/madrigal2" example, this would be "madrigal2".
3. MADSERVERCGI - New, temporary script directory relative to MADSERVER for temporary cgi files. The web server will need write permission in this directory. This directory should be set to execute only. For the "/var/www/cgi-bin/madrigal2" example , this would be "madrigal2".
4. MADSERVERDOCABS - New, temporary directory (absolute) where html files should be installed. This directory was created in step 3.
5. MADSERVERCGIABS - New, temporary cgi directory. This directory was created in step 3.
In the new MADROOT directory edit metadata/siteTab.txt. This file is comma-delimited. For your site, edit field 4 to be the new MADSERVERROOT set in step 9-2, and field 5 to be the new MADSERVERCGI set in step 9-3.
Be sure to cd to MADROOT before running the following step. Run the following command:
```
bash installMadrigal  &> install.log &
```
There may be a long pause when running updateMaster near the end of the installation since the instParmTab.txt metadata file is being built for the first time by examining every data file, but future calls to updateMaster will be much faster since only new experiments are examined. Help with any installation errors is available from the OpenMadrigal administrator.
If there were no errors, test your madrigal installation at the url given by MADSERVERROOT. You can also test it by running MADROOT/bin/python testMadrigalBuild.py, which will test the ability to create plots.
In the next step you will run a script that will switch the active Madrigal installation to the new installation. By default, it will copy all the experiment data to the new MADROOT directory, so you'll need to make sure there's enough room on the hard drive for two copies of the experiments directory. If you do not have room for that, you may use the --moveExp flag to cause the experiment directory to be moved to the new MADROOT directory, rather than copied. This script will also modify madrigal.cfg to use the web paths of the original Madrigal server, and will then install the new cgi scripts and html documents to those original web directories. In this way users will not need to enter a new url to visit your Madrigal site. If any problem emerges with the new version, instructions are given in the next step for switching back to the original Madrigal version. To make the switch, cd to the new MADROOT and run:
```
bin/switchMadrigal [--moveExp] <original_madroot> <new_madroot>
```
You should now be able to test the new release of Madrigal at the main Madrigal url.
If for any reason you decide you want to switch back to the old version of Madrigal, it is easy to do so. Just export MADROOT to be the old value, cd to the old MADROOT, and run:
```
	  tclsh configureHtml
	  tclsh configureScripts
	  bin/updateMaster
```
You should now see the old version of Madrigal at the main Madrigal url. If you used the --moveExp flag when switching in the previous step, you will need to preceed this procedure by moving the experiments back to the original MADROOT directory and running ./configureExperiments.
There may be reasons you do not want the value of the MADROOT environmental variable to change when you update Madrigal on the same server. For example, you may have written programs using the Madrigal API that hard-coded the MADROOT value. If, after you are done with the procedure above, you decide you want the new version on Madrigal to run using the old MADROOT value, follow the procedure Moving the Madrigal madroot directory.
This update procedure creates two copies of Madrigal and all the data. Once you are satisfied with the new version of Madrigal, free up disk space by removing:
1. The old MADROOT directory
2. The temporary MADSERVERROOT directory created in step 9-2.
3. The temporary MADSERVERCGI directory created in step 9-3.

Moving Madrigal to a new server (installs the latest Madrigal version)

If you want to move Madrigal to a new server, and the main Madrigal url is changing, you should notify the OpenMadrigal administrator so that they can update the Madrigal metadata that lists Madrigal sites. If you are moving to a new server but retaining the old Madrigal url, there is no need to change the metadata file siteTab.txt.

In this procedure you will first install a fresh version of Madrigal, and then run a script to import your Madrigal data from your old server.

Install a new version of Madrigal as decribed in Installing Madrigal for the first time. You may use your old madrigal.cfg file as a guide, but you will need to change it for the fields that have changed for the new server.
Once you are happy with the new installation, run the following script from the MADROOT directory of the new Madrigal server:
```
bin/switchMadrigal.py --newUrl <username>@<server>:<original_madroot> <new_madroot>
```
Check that all the data from the old Madrigal server now appears on the new server.
Once you are satisfied with the new installation, you may remove the old installation by removing the following three directories on the old server:
1. The MADSERVERROOT directory on the old server (specified in old_madroot/madrigal.cfg)
2. The MADSERVERCGI directory on the old server (specified in old_madroot/madrigal.cfg)
3. The old MADROOT directory

Moving the Madrigal madroot directory

There may be reasons you do not want the value of the MADROOT environmental variable to change when you update Madrigal on the same server. For example, you may have written programs using the Madrigal API that hard-coded the MADROOT value. If, after installing Madrigal as described in the Upgrading Madrigal section, you want to have MADROOT return to its original value, do the following:

Rename the old MADROOT directory to a different name
Rename the new, active MADROOT directory to the original MADROOT name.
Set the environmental variable to be the original MADROOT name.
Edit the MADROOT line in madrigal.cfg to be the original MADROOT name.
Next you will rerun the installation with the -n flag so that it only does a small subset of the main installtion's tasks. None of the third party python code is rebuilt, which is a major part of the main Madrigal installation time:
```
bash installMadrigal -n &> install.log &
```
Help with any installation errors is available from the OpenMadrigal administrator.

Direct Upgrade (no longer recommended)

To install the latest Madrigal release using the Direct Upgrade method:

Back up all files in MADROOT/metadata. These backups will only be needed if you have modified siteTab.txt or instTab.txt, and not notified the OpenMadrigal administrator about your changes.
Download the Madrigal distribution file, madrigal*.tar.Z to your MADROOT directory.
uncompress madrigal*.tar.Z
tar -xf madrigal*.tar (This will not overwrite any file meant to be modified by a specific Madrigal installation.)
Compare the new version of madrigal.cfg.template file to your existing madrigal.cfg file found under the MADROOT directory. If any new parameters have been added to the end of the file, add them to you madrigal.cfg file and edit just those entries as described in the installation documentation.
Diff the three backed-up metadata files mentioned in the first step with the installed versions in MADROOT/metadata, and make any changes needed.
Be sure to cd to MADROOT before running the following step. Then you should be able to complete the installation simply by typing
```
bash installMadrigal &> install.log &
```
There may be a long pause when running updateMaster near the end of the installation since the instParmTab.txt metadata file is being built for the first time by examining every data file, but future calls to updateMaster will be much faster since only new experiments are examined. Help with any installation errors is available from the OpenMadrigal administrator.
If there were no errors, your madrigal installation should be running at the url given by MADSERVERROOT. You can also test it by running MADROOT/bin/python testMadrigalBuild.py, which will test the ability to create plots.
If you want to add any documentation pages specific to your site to the Madrigal documentation pages, see the other admin tasks section of this manual.
If you want to add your own rules of the road to the Madrigal experiment page, see the other admin tasks section of this manual.