UvA course 2016-06-15 - part B

UvA logo

SURFsara logo

This is part B of the Tutorial UvA course 2016-06-15.

If you have not completed (and understood) Part A, please do so first.

NOTE:

For the rest of the tutorial we will omit the ubuntu@ip-...:~$ prompt in the instructions, in order to allow you to copy-n-paste the commands directly in your terminal.

1. Persistence

Images can be persistent or non-persistent and you can change this mode at will.

Food for brain:

Was the first image that you imported persistent or wasn’t it?

In this section you will work with persistent images. You will go through these steps:

Make your image persistent

Start a persistent VM

Food for brain: During the previous run, the VM’s disk was non-persistent. From now on, you can store data in your VM that will be written on the backing image even if you restart your VM. Try it.

2. Scale up to a multicore VM

The HPC Cloud is offered in an Infrastructure as a Service (IaaS) model (as listed in section Service Models in the The NIST definition of cloud computing paper). That allows you to give your Virtual Machines (VMs) the form that you need them to have. In this section, you will start a four-core VM, using the exact same image, the Course Image. To scale-up your VM to use multiple cores you will be:

Edit the template

You can customise your VMs by editing the templates you instantiate the VMs from.

That was it. From now on you will get a 4-core VM running using the same Course Image.

Instantiate the four-core VM

Food for brain:

  • How/where do you see, in the output of cat /proc/cpuinfo, that your VM actually has 4 cores?
  • What is the status of your image now that it is persistent and your VM is running? Is there any difference in status as compared to when the image was not persistent?

3. Working with Storage

The current HPC Cloud offers two storage types: Ceph and SSD. Ceph is a storage system that we have set up as a shared storage place in its own cluster. As opposed to SSD disks, which form local storage available locally on the physical server that your VM runs on, Ceph is only accessible via the network. The main reason to have Ceph along SSD is that it is very scalable; it allows both to add a lot of storage to the system when needed, while sustaining a lot of concurrent access to many users at the same time. Further, data stored on Ceph is replicated to protect against data loss in case of hardware failure.

With all of this in mind, we advise that you make use of both storage types. You should run your operating system on a small SSD image and store your bulk data on Ceph. You can do so by defining the right datastore when creating your images.

When you create an image, you must choose where it is stored, under the heading Datastore. You have the choice between local_images_Cursus (SSD) and ceph. The first appliance you imported, created the OS image on local_images_Cursus (SSD). In this section you will use the ceph datastore option, by following these steps:

At this point you should not have any running VMs. If you do, shut them down. Let’s create a new image.

Create a new empty image for bulk data

NOTE:

A new image will show on the Images list, and it will keep in status LOCKED while it is being created. When it is created it will come to status READY. Then you still have to format and mount the disk.

Add the new image to the template

In order to let you VM know about the new datablock, you need to add it to your VM’s template:

Mount the image in the VM

Let’s start using the new disk.

sudo fdisk -l

Note: The sudo commands may display the message sudo: unable to resolve host ip-145-100-.... Please, ignore this; sudo is still doing what you are asking it to do.

sudo mkdir /data  
sudo mkfs -t xfs /dev/vdb  
sudo mount /dev/vdb /data  
sudo chown ubuntu:ubuntu -R /data

Food for brain:

Create new files or folders in your in /data directory. Logout and login again. Are your changes still there? Check with ls /data/. Shut the VM down and start it again. Do you see the files on the datablock? Hint: when you start the VM the datablock is not automatically mounted. You should issue the mount command once again.

Food for brain:

Try to copy a file from your laptop to /data, e.g. with scp myfile ubuntu@145.100.58.XYZ:/data. Then log in to the VM and inspect the changes.

BONUS food for brain

This section is meant as extra questions that we thought would be nice for you to investigate, and we invite you to do/think about them even after the workshop is finished.

Bonus: In Parts A & B of the workshop you have explored the UI and learnt how to import an appliance from the AppMarket. However, many people come to the HPC Cloud because they can install and run their own operating system. In particular, Windows is very popular among our users.

Note:

Play around, make your checks and shut down all the VMs when you are done. Your running VMs are consuming quota whether they are doing something useful or are idle.

Next: Extras

You have completed part B of the Tutorial UvA course 2016-06-15. If you want more of the HPC Cloud, see the Extras part.