VU HPC course 2018-11-16
Part B

VU logo

SURFsara logo

This is part B of the tutorial VU HPC course 2018-11-16 and covers the following topics:

  1. Persistence
  2. Scale up to a multicore VM
  3. Working with Storage

Notes

If you have not completed and understood Part A, please take your time before moving forwards.

Do you have any Virtual Machine(s) running? Terminate them now.

For the rest of the tutorial we will omit the terminal prompt ubuntu@packer-ubuntu-14:~$ in the instructions. It allows you to copy-n-paste the commands directly to your terminal.

1. Persistence

Images can be either persistent or non-persistent and you can change this mode at will.

Take a couple of minutes to read again above statements and fully understand its implications. The persistency concept is crucial at early stages, when you are starting to build a VM that meets your needs, by installing software, pushing data, etc. At this stage, your aim might well be to preserve all your changes! At later stages of your work, you might want to profit from the capability of instantiating multiple Virtual Machines from the same ‘pristine’ image you have prepared so diligently.

In this section you will work with persistent images. We will guide through the following steps:

Food for brain

Was the first image that you imported in Part A persistent?

Making your image persistent

No Virtual Machine should be running at this point and making use of the image we want to change the Persistent attribute. Otherwise, the following procedure will not work.

Starting a persistent VM

You already have the knowledge on how to start a VM. Make use of the template you created before (First Template).

Food for brain

  • Why there was no need to edit the template before instantiating the VM?
  • Is/are the file(s) you created in your VM (in part A) still there?
  • In the previous VM run the disk (image) attached was non-persistent. From now on, you can safely store data in your VM! It will be written to the image attached to the VM, even if you restart it. Try it.

You can now shutdown the VM. Remember, use the Terminate button on the UI.

2. Scale up to a multicore VM

From the course introduction presentation, you gained the insight that SURFsara HPC Cloud operates as an Infrastructure as a Service (IaaS). This cloud computing model brings flexibility to our users, such as the ability to scale-up. In this section, you will start a 4-core VM using the exact same image as before.

To scale-up your VM to use multiple cores you will be:

Editing the template

You can customise your VM(s) by editing the template you instantiate the VM(s) from. Proceed as follows:

From now on, whenever you instantiate this template, you will get a 4-core VM running using the same First Image you have been using so far.

Food for brain

Imagine that a VM is already running instantiated from the First Template. What will happen with that VM if you edit the same template?

Instantiating the four-core VM

Food for brain

Since we are in the VM again … are your files there now? All? None? Can you explain?

Shut down the four-core VM

Before moving forward, please remember to terminate all your VMs. Again, use the Terminate button on the UI.

3. Working with storage types

The current HPC Cloud setup offers two storage types: SSD and Ceph. SSD storage is available as Solid State Drive (SSD) disks in the compute nodes where the VMs run. The available SSD storage per node is limited. On the other hand, Ceph is a robust and flexible storage system that aims for high performance. Ceph can store rather big files, and data placed in our Ceph storage cluster is replicated throughout the cluster to protect against data loss in case of hardware failure. As a guiding note, we advise our users to run the operating system in a small SSD image, while bulk data should be stored in Ceph datablock(s).

For each image in your Cloud environment, you need to choose which storage type you want to use. This is achieved by placing the image on either of the two datastores you have available: Courses_img (which provides the SSD storage type) and ceph.

When you imported your first app in Part A of this tutorial, you created an image which you assigned to the Courses_img datastore. The image has Ubuntu desktop 14.04.5 installed and will be running on the SSD storage of the computing nodes.

In this section you will be exploring the ceph datastore. You can create and make use of an image stored in the ceph datastore, by following these steps:

At this point you should not have any running VMs. If you have, shut them down now.

Creating a new empty image for bulk data

Let’s create a new image via the HPC Cloud UI.

Note

A new image will show on the Images list and it will keep in status LOCKED while it is being created. Once it is created it will display status READY.

Adding the new image to the template

In order to let your VM know about the new datablock, you need to add it to your VM’s template:

Mounting the image in the VM

Let’s start using the new disk.

sudo fdisk -l

Food for brain

From the output of the previous command:

  • can you identify the two disks attached to your VM?
  • can you explain the Disk /dev/vdb doesn't contain a valid partition table statement?

Warning

Creating a file system in a disk (to format) is a dangerous operation. Make sure you format the correct disk device!

sudo mkdir /data  
sudo mkfs -t xfs /dev/vdb
sudo mount /dev/vdb /data  
sudo chown ubuntu:ubuntu -R /data

Food for brain:

  1. Create new files or folders in your /data directory. Logout and login again. Are your changes still there?.
  2. Shut down the VM and start it again. Do you see the files on the Ceph datablock? Hint: when you start the VM the datablock is not automatically mounted.
  3. Copy a file from your laptop to /data, e.g. with scp filename ubuntu@145.100.5Q.RST:/data. Log in to the VM and inspect the changes.

Bonus food for brain

In Parts A & B of the workshop you have explored the UI and learnt how to import an appliance from the Apps Marketplace. You also played with different storage solutions to hold data. Many users come to the HPC Cloud because they can install and run their favorite operating system. For example, Windows is very popular for some of our users.

Next: Extras

You have completed part B of the Tutorial VU HPC course 2018-11-16. If you want more of the HPC Cloud proceed to the Extras part.

Important

Your running VMs get exclusive access to their resources, whether they are doing something useful or not. Because the HPC Cloud is offered on a fair-share basis and other users may actually be needing resources that you may be holding, please remember to terminate all your VMs before you move to the next part of this workshop.