Tips for building software on Euramoo, Flashlite & Tinaroo

Follow

Background

Unfortunately, due to inadequate resourcing levels, we (QCIF) are generally unable to meet requests from individual users to install applications.  (Feel free to ask, but the answer will typically be "No").  Instead, we recommend that users "self service" and install the applications that they need for themselves.

This page is intended to give some general guidance for people building and installing software.

General instructions

The first two steps to installing software are:

  1. Find the source code bundle for the software you want to install.
  2. Find and read the (correct) build and installation instructions.  You may be able to find them using a Google search.  If not, the source code bundle (e.g. a "tar" or "tar.gz" file) should include instructions in the top directory: look for a README file or similar.

Generally speaking, installing binaries is likely to be problematic. You will get better results by trying to build from source.

Notes: 

  1. You will not have sufficient access to install software into the standard locations; e.g. /usr/bin, /usr/lib, /usr/man and so on. If the instructions tell you to login as "root" or use the "sudo" command, they won't work for you.
  2. If the installation instructions tell you how to install as a non-privileged user (i.e. when you don't have "root" access), follow those instructions.
  3. If the instructions don't explicitly say what to do if you don't have root access, the following sections give some tips on what to do.

Where to install software to

As stated above, you do not have permission to install software into the standard Linux locations, or to the local locations that are used for sharing; e.g. "/opt", "/usr/local" or "/sw".  Instead, you should use one of the following:

  • Your "home" directory (preferred)
  • Your "30days" or "90days" directories ... if you only want to use the software for a relatively short time
  • A "file share" collection.
  • Some shared space that has been set up for you.

Tips for installing packages; i.e. how to handle "apt-get", "yum", "dnf", etc

If the instructions tell you to install a package (for example a library dependency) using "apt-get", "yum" or "dnf". This will typically require root access, which you don't have.

The best / simplest option is find the source code for the packaged software, then download and build it.  Then install it into the directory where you are putting your local installs.

Alternatively, submit a support request to have the package installed. If a package is of general interest / use, and if there is a suitable version available, we may agree to install it for you.

There are other ways that a non-root user can deal with software in packages (e.g. here or here) but they are complicated, and not recommended unless you are a Linux expert.

Tips for "configure" and similar

A lot of C and C++ software uses a "configure" script to analyse the platform dependencies, and generate a Makefile.  You then use the generated Makefile to build and install the software.

If the build instructions say to run something like the following:

    cd some-package
    ./configure      # this is the key step
    make
    make install

you will run into problems in the last step.  The final "make install" step is going to attempt to install into directories where you do not have write permission.  The simple way to get "make install" to install somewhere else is to set a "prefix" when running configure.  For example, the following will install into your home directory.

    cd package
    ./configure --prefix=$HOME
    make
    make install

The prefix can be any directory that you can write to.  The "make install" step will typically create subdirectories called "bin", "lib", "man" and so on in the prefix directory.

Sometimes the instructions may tell you to use "autoconf" to create the "configure" script.  Other times, the script is called "configure.sh" rather than "configure".  In either case, the approach of adding "--prefix=..." is likely to work.  However, read the supplied build documentation.

Tips for hand-written Makefiles

If you have a C or C++ application or library that does not use a configure script, it probably has a hand-written Makefile.  A common convention in hand-written makefiles is to use the DESTDIR variable to specify the installation directory.  (The semantics are typically the same as "--prefix=..."; see above.  Here is a recipe:

  1. Open the top-level Makefile in a text editor
  2. Look to see if it contains references to "$(DESTDIR)"
  3. Make the software using "make" as per the build instruction
  4. Install the software like this: "make DESTDIR=$HOME install".  Note the addition of "DESTDIR=...".

(If you are uncertain, you may want to run the "make" commands with the "-n" option first. That should tell you what running "male: would do ... without doing it.)

Tips for Python

There are three approaches that you can use to installing python applications when you don't have root privilege.

  1. Don't install it at all. Some python scripts do not need to be installed.

Tips for R

Tips for Matlab

Euramoo considerations

As you should be aware (if you use the system) Euramoo is a complicated system that has multiple operating systems and multiple hardware architectures.  You will get the most reliable results and the best performance if you do the following:

  1. Pick one node type that you will use to run the application you are going to install.
  2. Start an interactive job on that node type.
  3. For the node types that use "modules" (all of them apart from BioLinux), use "module avail" and "module load" to load the prerequisites before you build.  (For example the "GCC" module for C / C++ code.)
  4. Build the software and install it into your home directory.
  5. Use the software on that node type only.  Remember to load the same modules that you loaded to build the software.

File space for software

Some people run into problems with file space or file count quotas when building and installing into their HPC home directories. If this happens to you, please raise a support ticket.

If you want to share software you have built with other users, we can provide a "file share" collection to enable this.  Please raise a support ticket if you want to request one.

FAQs:

Q: Can I have root access? 

A: No.  We cannot allow that.  Besides, it is not necessary.

Q: Will you install a Linux package for me? 

A: It depends.  If the version of the package that you require is available via the package manager, then we will consider it.  However, doing this is not as simple as you think, for various reasons.  We may be able to suggest an alternative.

Q: What do I do if I can't get the build to work?

A: Here are some suggestions:

  • Do you have local IT support who could help?  Or an expert in your research group?
  • Raise a QRIScloud (or RCC) support request. If we have time, we may be able to advise, or arrange a consultation.
  • You may be able to get help at your local Hackyhour (UQ and Griffith ... more coming)
  • You may be able to find a solution using Google.
  • Persevere.

 

Have more questions? Submit a request

Comments

Powered by Zendesk