UvA HPC course 2017-02-01
Part B

UvA logo

SURFsara logo

This is part B of the tutorial UvA HPC course 2017-02-01 and covers the following topics:

  1. Persistence
  2. Scale up to a multicore VM
  3. Working with Storage

NOTES:

If you have not completed (and understood) Part A, please do so first.

For the rest of the tutorial we will omit the ubuntu@ip-145...:~$ prompt in the instructions. It allows you to copy-n-paste the commands directly to your terminal.

1. Persistence

Images can be persistent or non-persistent and you can change this mode at will.

Food for brain:

Was the first image that you imported persistent?

In this section you will work with persistent images. You will go through these steps:

Make your image persistent

Start a persistent VM


The Ubuntu Desktop 14.04 App is affected by a problem described in our documentation. To correct it easily, please follow these steps now that you are connected via SSH:

wget https://github.com/sara-nl/clouddocs/raw/gh-pages/UvA-20170201/code/cloudinit_fix.sh
chmod +x cloudinit_fix.sh
sudo ./cloudinit_fix.sh

Food for brain:

  • Is/are the file(s) you created on your VM (part A) there?
  • During the previous run, the VM’s disk was non-persistent. From now on, you can store data in your VM that will be written to the image even if you restart your VM. Try it.

You can now shut your VM down.

2. Scale up to a multicore VM

The HPC Cloud is offered as an Infrastructure as a Service (IaaS). That allows you to give your VMs the form that you need them to have. In this section, you will start a four-core VM, using the exact same image, the Course Image. To scale-up your VM to use multiple cores you will be:

Edit the template

You can customise your VMs by editing the templates you instantiate the VMs from.

From now on, whenever you instantiate this template, you will get a 4-core VM running using the same Course Image.

Food for brain:

After changing the template as described above, how many cores do you expected your VM to have? Does changing a template affects the currently running VM’s that are based on it?

Instantiate the four-core VM

Shut down the four-core VM

3. Working with Storage

The current HPC Cloud offers two storage types: Ceph and SSD. Data stored on Ceph is replicated to protect against data loss in case of hardware failure. Our recommended best practice is to run your operating system on a small SSD image and store your bulk data on Ceph datablock(s).

When you create an image, you must choose where it is stored, under the heading Datastore. You have the choice between local_images_Cursus (SSD) and ceph. When you imported your first App in tutorial Part A, you created an image containing an Ubuntu 14.04 OS on local_images_Cursus (SSD). In this section we’re introducing to you the ceph datastore option. You can create an image in ceph datastore by following these steps:

At this point you should not have any running VMs. If you do, shut them down.

Create a new empty image for bulk data

Let’s create a new image.

NOTE:

A new image will show on the Images list and it will keep in status LOCKED while it is being created. Once it is created it will display status READY. Then you still have to format and mount the disk; you will be doing that just in a moment.

Add the new image to the template

In order to let your VM know about the new datablock, you need to add it to your VM’s template:

Mount the image in the VM

Let’s start using the new disk.

sudo fdisk -l

Note: If above sudo command displays the message sudo: unable to resolve host ip-145-100-... just ignore it.

Food for brain:

You should see (probably, as a last line of the output of the previous command) something like: Disk /dev/vdb doesn't contain a valid partition table. The output should also inform that there is a Disk with approximately of the size you typed in the UI in the previous step. What is that Disk’s name?

sudo mkdir /data  
sudo mkfs -t xfs /dev/vdb  
sudo mount /dev/vdb /data  
sudo chown ubuntu:ubuntu -R /data

Food for brain:

Create new files or folders in your in /data directory. Logout and login again. Are your changes still there? Check with ls /data/. Shut the VM down and start it again. Do you see the files on the datablock? Hint: when you start the VM the datablock is not automatically mounted. You should issue the mount command once again.

Food for brain:

Try to copy a file from your laptop to /data, e.g. with scp myfile ubuntu@145.100.5Q.RST:/data. Log in to the VM and inspect the changes.

BONUS food for brain

This section is meant to propose extra questions we thought would be nice for you to investigate. We invite you to do/think about them even after the workshop has finished.

Bonus: In Parts A & B of the workshop you have explored the UI and learnt how to import an appliance from the Apps Market. However, many people come to the HPC Cloud because they can install and run their own operating system. In particular, Windows is very popular among our users.

NOTE:

Play around, make your checks and shut down all the VMs when you are done. Your running VMs are consuming quota whether they are doing something useful or are idle.

Next: Extras

You have completed part B of the Tutorial UvA HPC course 2017-02-01. If you want more of the HPC Cloud proceed to the Extras part.