Section 2: Basic QRIScompute questions
Cloud computing is a metaphor for doing computing tasks on a computer infrastructure run by someone else "on the internet". (The origins of the term are uncertain, and there is no single precise definition.) The difference between cloud computing and the classic IT service model is that the infrastructure you use is typically owned and run by service providers that are external to your organization.
In this case, QCIF is running the Brisbane and Townsville based cloud computing infrastructure on behalf of QCIF member organizations, and we form part of the NeCTAR Research Cloud which is an Australia-wide cloud computing federation with capacity in all States and in the ACT.
Virtual machines (VM's) allow a physical computer to be shared among a number of users, with each user appearing to have exclusive access to the machine.
Virtual machines are typically implemented using software known as a "hypervisor" which mediates each virtual computer's access to the physical computer hardware, and stops the VMs from interfering with each other.
Not really. Typical cloud computing systems are built using standard computing hardware that is optimized for economical performance rather than for speed. By contrast, High Performance Computing (HPC) systems tend to provide high-end processors, providing some combination of large numbers of cores, lots of memory, high-performance inter-processor communication and high performance disk I/O.
Despite this, a lot of computational tasks that run on HPC systems will run just fine on a cloud computing facility. If you want advice on this, please contact QRIScloud support, and we can arrange for an eResearch Analyst to look at your computational problem, and help you figure our the best way to address them.
The NeCTAR Research Cloud is a federation of cloud computing facilities located in each of the Australian State capital cities, and Canberra. The infrastructure is implemented and managed using the OpenStack cloud computing framework.
"OpenStack is a set of software tools for building and managing cloud computing platforms for public and private clouds. Backed by some of the biggest companies in software development and hosting, as well as thousands of individual community members, many think that OpenStack is the future of cloud computing. OpenStack is managed by the OpenStack Foundation, a non-profit which oversees both development and community-building around the project" - source.
A NeCTAR RC project consists of a collection of resources (Instances, Objects, Volumes and so forth) that project members can use. A project is managed by a project manager (who controls who the members are) and has an associate NeCTAR Allocation; see below. (Refer to FAQ 4.1 for explanations of "Instance", "Objects" and "Volumes".)
A Project Trial (PT) is a NeCTAR RC project with limited resources and time-span that is intended to let you try out the cloud before you commit to using it. A PT has the resources for running up to 2 instances using up to 2 VCPUs, and a time limit of 3 months.
Simply visit the NeCTAR RC Dashboard. You will first be directed to your home institution's AAF login page. Then you will be asked to read the NeCTAR terms and conditions. Finally, a PT project will be created automatically for you.
Visit the NeCTAR RC Dashboard (see above), and fill in and submit an application using the Request an Allocation page. You will need to set out your resource requirements and your project duration, and provide a research description and a technical justification for your resource request. NeCTAR RC resources allocated based on the research and technical merit of your application, the resources you are applying for, and resource availability.
We encourage you to contact QRIScloud support if you need help in making the application. We can arrange for a QCIF eResearch analyst to advise and assist you.
A NeCTAR allocation is effectively permission for you and your team to use up to a certain level of NeCTAR cloud resources over a particular period of time. The allocation provides the resource quotas for a NeCTAR project.
The basic computational resources that you need to apply for are Instances, VCPUs and VCPU-hours. The basic computational resources come with a modest amount of disk storage (see Flavor) that will be associated with your virtual machines. In addition, you can apply for VM independent NeCTAR storage in the form of Object Storage and / or Volume Storage.
The terms used above (Instance, VCPU, VCPU-hours, virtual machine, Flavor, Volume Storage, Object Storage, etc) are explained in section 4 of the FAQ.
Unfortunately, no. A NeCTAR allocation gives you quotas for a given number of Instances and VCPUs. However, when you attempt launch an Instance, it can fail with this message:
This can be caused by a variety of things, but a common cause is that OpenStack could not find the required number of free cores or the required amount of memory in the specified Availability Zone. If this happens, you could try launching a smaller Instance, or launching in a different (less full) Availability Zone.
Training is a grey area. The current NeCTAR policy is that the resources can be used for training purposes at the discretion of NeCTAR nodes. We advise the following:
- It is inadvisable expect students or trainees to use their PTs in a training course. The exception is for basic "How to get started with NeCTAR" training, with the proviso the trainer should instruct the users to Terminate their instances.
- If a lecturer or trainer requests an allocation for training purposes, the onus is in him / her to ensure that best practice is followed:
- If trainees are allowed to launch instances, they should be properly advised on how to secure instances, and on the need to Terminated them promptly.
- In either case, the lecturer / trainer (or the staff member who requested the allocation on their behalf) is responsible for the "house keeping".
- If an allocation is required for a student project work, the allocation should ideally be requested by the supervisor. Alternatively, the supervisor should be listed as the Chief Investigator. In either case, the supervisor should take responsibility once the project is completed.
We place no restrictions on the kinds of application that you can run, provided that they meet the general rules set out by NeCTAR.
The simple answer is No.
For legal reasons, we (QCIF) cannot run Microsoft Windows on any of the hardware that runs QRIScloud, and we cannot allow users to do this either.
In principle, yes. However, we cannot give a definite answer to this question without examining what the license conditions are. Please contact QRIScloud support for advice.
Note that software licenses that are tied to specified IP addresses or MAC addresses can be problematic.
Generally speaking, if an application runs on a modern version / distribution of the Linux operating system, it will run on a QRIScompute virtual machine.
Yes, you can. However, the standard NeCTAR Linux images do not have a "desktop environment" installed, so you will typically need to install a lot of related "packages".
NeCTAR has recently started providing service accounts for NeCTAR projects that need them.
If you are a NeCTAR tenant manager, you can request a service account for your project via NeCTAR Support. In the service request, mention that you want a "robot account" and say which NeCTAR project it should be associated with. (A request for a robot account on a PT would be refused.)
If you are running a service on NeCTAR instances, you may need to write scripts to run unattended that need to interact with OpenStack services. For example, you might want your instance's nightly backup script to save a copy of the backup into Swift Object Storage. When a script interacts with OpenStack, it needs to provide an identity and an OpenStack password. If you embed your own NeCTAR identity and password into a script then:
- You are exposing your personal identity information to anyone with root access to the instance, or the ability to gain root access.
- If you reset your personal password, you will break all of the scripts where you have embedded it.
Using a service account ameliorates these problems.
As part of the QRIScloud Stage 2 equipment tranche, we purchased some compute nodes with extra resources. These form the basis of our "Special Compute" services: "big memory", "fast I/O" and "GPU".
You can request access to Special Compute facilities via the QRIScloud Portal Services page.
QRIScloud currently has 4 Big Memory compute nodes which each have 64 cores and 1 Terabyte of RAM. Allowing for overheads, we are able to provide special flavors with the following dimensions:
|Flavor||Memory||VCPUs||Primary disk||Ephemeral disk|
The Fast I/O compute nodes are not available at the moment.
The two GPU compute nodes are ready for "experimental use". While GPU node flavors are not finalized, the sizing is likely to be:
|Flavor||GPU cards||Memory||VCPUs (Intel cores)
|"full size"||2 x Tesla K20m||128Gb||20|
|"half-sized"||1 x Tesla K20m||64Gb||
We are considering moving the Special Compute GPU nodes into Euramoo to simplify management and broaden their availability.
Use the QRIScloud Portal Services page to request access. If you are eligible, and your use-case is a good match for the facility, we will set up an instance for you to use. We normally allocate in blocks of 2 weeks, and we can put your instance "on ice" while some other user gets to use the service.
Nimrod is a framework for running "parameter sweep" computations. Kepler is a tool for designing and running computational workflows. Nimrod and Kepler can be combined in various ways for doing large-scale scientific computations.
The QCIF Nimrod team can arrange to run Nimrod / Kepler based computations on NeCTAR, MASSIV and in other places. You can request access to these resources via the QRIScloud Portal Services page.
The Terrestrial Ecosystems Research Network (TERN) project provided Nimrod and Kepler services; please read "Nimrod and Kepler Services on the CoESRA".
Euramoo is a "virtual" cluster implemented using NeCTAR OpenStack instances that is suitable for running low-end HPC jobs with modest CPU and memory requirements. For more information refer to Section 6 of the FAQs.
Yes, although QCIF's HPC portfolio is currently in flux:
- The QCIF-funded Barrine system has been decommissioned, except as a way to access files stored in Barrine's HSM storage.
- QCIF manages a significant number of "shares" of resources in the NCI HPC facilities.
- The new QCIF-funded Flashlite "data intensive" HPC is now available.
We may also be able to help you with access to member university HPC facilities at UQ, QUT and CQU.
- You can register for access to Flashlite and Euramoo, via the QRIScloud Portal Services page.
- You can request to use QCIF-controlled NCI shares via the QRIScloud Portal Services page.
- For access to other HPC resources, please open a QRIScloud Support request and we will arrange for someone to discuss your needs and put you in contact with the appropriate group.