Managing warm standby disaster recovery

A typical warm standby disaster recovery workflow follows. For more details, see the associated topics below.

Install the active and standby server

Prepare for warm standby installation.
Configure the active Blue Planet server(s).
Bootstrap the active Blue Planet server(s).
Configure the standby Blue Planet server(s).
Bootstrap the standby Blue Planet server(s).

Configure geored tunneling

Configure IP tunnels between active and standby servers.
For a network with multiple clusters, configure all Blue Planet servers (optional).

Verify replication

Verify replication is active by checking Nagios or using API calls to get geored status on all geoaware apps.

In case of failure

Convert standby server to become the active server.
Verify no data is lost.
Configure new standby to newly activated cluster.

Preparing for warm standby installation

To support warm standby geo-redundancy you need:

The specific orchestrate solution that supports georedundancy must be running on the cluster servers. Two Blue Planet clusters composed of at least three nodes with bp2hosttools version 17.06.1 (or greater) installed. One of the clusters acts as the active cluster and the other cluster acts as the standby. There is no support for a single host environments in this release.

For cluster provisioning and bp2hosttools installation instructions, see your Blue Planet Installation Guide.
Only georedundant-aware applications replicate their state from the active to the standby site. For details on which applications are georedundant-aware for warm standby georedundancy, see About warm and cold standby disaster recovery.

System configured so that bpssh commands are successful on all cluster nodes. Geored installation fails if this package is not installed.
SSH must be set up on each of the cluster servers. Verify you can SSH into all the other site hosts.
User account strategies thought out. You can let users use the default geored user and password or set up specific users to have access to geored.
Docker registry configuration is configured properly.
To run the REST API geored calls via Swagger graphical interface, you must have swagger-ui installed and have privileges.
For core ilan config each server has to be set up differently so that the sark engine can figure out who it is responding to.
Network guidelines are 100 Mbps and less than 100 ms latency (NY to Frankfurt).

Installing Blue Planet and warm geored on the active server

For installation instructions, see the Blue Planet installation guide.

Bootstrapping warm geored on the active

This topic describes a procedure to bootstrap your active and standby servers that have already been installed and deployed.

To bootstrap the active site on each host, enter:

curl -u <user>:<password> -X POST https://<active_site_IP>/geored/site/bootstrap -d '
{"site_id":"primary_site_name", "site_state": "active", "remote_sites": [{"address": "<standby_site_IP>", "site_id": "secondary_site_name", "site_state": "standby"}]}'

Substitute your server details above. The keys site_id and site_state are required, but remote_sites may be omitted and added later via another API endpoint.

After entering the command, you receive a config bootstrap completed successfully response. Once you bootstrap the site, active site applications should be running normally and standby site applications should be running in geored standby mode.

At this point if you have made the decision to wait to continue with the full configuration of warm standby geored, you can stop here and complete the rest of the configuration at a later time.

Completing the bootstrapping on the active and standby servers

When you are ready to complete the warm geored on both active and standby servers, do the following:

Bootstrap the active site on each host if you did not include the remote site details. If you have already included the remote sites details, skip this step.

curl -u <user>:<password> -X POST https://<active_site_IP>/geored/site/bootstrap -d '
{"site_id":"primary_site_name", "site_state": "active", "remote_sites": [{"address": "<standby_site_IP>", "site_id": "secondary_site_name", "site_state": "standby"}]}'

Bootstrap the standby site on each host:
```
curl -u <user>:<password> -X POST https://<standby_site_IP>/geored/site/bootstrap -d '
{"site_id": "secondary_site_name","site_state": "standby", "remote_sites": [{"address": "<active_site_IP>", "site_id": "primary_site_name", "site_state": "active"}]}'
```
Substitute your server details above.

After entering each command, you receive a config bootstrap completed successfully response. Once you bootstrap the sites, active site applications should be running normally and standby site applications should be running in geored standby mode.

Configuring geored tunneling

More to come.

Verify replication

Verify that the standby server contains the backup files.

More to come.

In case of failure

More to come.

Managing warm standby disaster recovery