QRIScloud Compute Improvements - Potential Hazard 11 Jan 2016 to 22 Feb 2016 (completed)

Follow

As part of the regular program of QRIScloud / NeCTAR service improvements, QCIF  will be performing hardware upgrades over the next six weeks in the QRIScloud availability zone. Once completed, these upgrades will provide additional compute resources to allow more users to run instances in QRIScloud.

Users are advised of the following potential service interruptions and degradation:

1. Some Instances need to be rebooted and live migrated.

To minimise the impact of the upgrades on existing user services, all virtual machines (Instances) running on the affected hardware will be live migrated to other hardware.

Instances that have been active since the recent NeCTAR Neutron upgrade (i.e. that have not been shut down or hard rebooted since 6:00 PM AEST on 6 Oct 2015) will not successfully live migrate without a reboot.  QRIScloud support will be contacting users individually regarding a shutdown/hard reboot for any instances that need to be migrated.

We would prefer that you perform the instance shutdowns / hard reboots yourself, ahead of the scheduled live migration window:

  • EITHER shut down the instances completely (i.e. the instances need to be listed as Shutdown on the NeCTAR dashboard),
  • OR hard reboot the instances from the NeCTAR dashboard, and check that they are working properly after the reboot.

(Please note: a soft reboot or a reboot from the command line in your virtual machine is NOT sufficient.)

If you don't take the necessary action yourself, we will hard reboot instances prior to the live migration.  The downside is that we cannot check that your Instances' services are working following a reboot.

2. Temporary reduction in QRIScloud compute capacity

To assist in performing the upgrades, new instance scheduling has now been restricted to a limited set of hosts. This step has been taken to allow us perform the upgrades in a timely fashion.

If you attempt to launch instances in the QRIScloud availability zone, you are highly likely to receive a "No available host" error message.  This applies to all flavors and is likely to continue until the upgrade period ends.  Similarly, if you terminate instances during the upgrade period you are highly unlikely to be able to relaunch them in the QRIScloud availability zone.

3. Terminate old or idle QRIScloud Instances

Finally, if you have any old or idle instances that you no longer need in the QRIScloud availability zone, we would appreciate it if you could terminate them after snapshotting, if that is warranted.  (Please be aware that there is a known NeCTAR issue where instances created prior to the Neutron upgrade need to be hard rebooted before snapshots will work.) 

The released capacity will allow us to perform the rolling upgrades more quickly. 

If you have any concerns or questions please contact us at support@qriscloud.org.au.

UPDATE - The "potential hazard" announced above is over, and the reserved QRIScloud compute capacity has been returned to the general NeCTAR pool. Unfortunately, we were only able to perform some of the compute code upgrades that were planned.  A second round of upgrades will be performed in a months time.

Have more questions? Submit a request

Comments

Powered by Zendesk