How to use AWS EC2 GPU instances with BitFusion

Jessica YungData Science, Uncategorized2 Comments

If you want to train neural networks seriously, you need more computational power than the typical laptop has. There are two solutions:

  1. Get (buy or borrow) more computational power (GPUs or servers) or
  2. Rent servers online.

GPUs cost over a hundred dollars each and top models like the NVIDIA TESLA cost thousands, so it’s usually easier and cheaper to rent a server especially if you don’t think you’ll use it much. Fortunately renting and setting up servers is easy with some help from BitFusion.

In this post I’ll cover:

  1. Choosing an AWS EC2 instance to start
  2. Launching your instance
  3. Logging into your instance
  4. Setting up your instance
  5. Cutting costs – making sure you don’t accidentally end up paying hundreds of dollars a month.

If you don’t have an AWS (Amazon Web Services) account, you should register for one here. You’ll need it to rent AWS servers.

I. Choosing an AWS EC2 instance to start

The cheapest GPU-enabled instance on AWS is g2.2xlarge, which costs US$0.767/hr including Bitfusion software costs. I recommend starting out with this if you want to train Tensorflow models. You can rent a Bitfusion-enabled AWS EC2 instance here. You get a free Bitfusion software free trial which lowers hourly costs to US$0.702/h for the first five days.

screenshot.png

Instance details

II. Launching your instance

After clicking ‘Continue’, you will be shown the page below.

  1. Choose your instance type. g2.2xlarge is the cheapest GPU-enabled instance on AWS EC2 (Elastic Cloud, their server renting service). The EC2 Instance Type list is generally arranged in ascending order of price, size and computational power.
  2. Create a new key pair or select an existing key pair you’d like to use (second screenshot). If you create a new key, DOWNLOAD THIS KEY AND DO NOT LOSE IT. This is like your home key to your instance.
  3. You don’t need to change the Security Group settings unless you wish (e.g. change Source to your IP so you can only log on to your instance from your network).
    1. Port 22 is for SSH (command line access to your instance) and
    2. Port 8888 is for browser access to iPython notebooks).
  4. You might be scared at the monthly cost estimate AWS shows you. You won’t have to pay nearly this much to train your models unless you leave your instance running 24/7 for an entire month.

bitfusion-ubuntu-tf-aws.png

screenshot.png

You should receive an email from Amazon about your subscription to Bitfusion Ubuntu 14 TensorFlow (if it’s the first time you’ve used this software).

You should also be shown this screen. Click on ‘AWS Management Console‘ to return to the Console.

aws-post-purchase.png

III. Logging into your instance

It may take your instance a few minutes to set up. Once it’s running, you should see a Public IP available:

aws-ec2-public-ip.png

(I have since rebooted my instance so this is no longer the instance’s Public IP.)

You have two choices: you can

  1. directly access your instance’s Jupyter Notebooks through the browser or
  2. access your instance through the command line, i.e. the black-screen terminal you see in movies. 😛

BitFusion has an excellent guide to help you from this point.

  1. Method 1: If you want to directly access your instance through the browser, type
where you’d usually type google.com or similar. For example, if my Public IP was 1234.5678, I’d type
2. Method 2: If you want to access your instance through the command line, go to Terminal  and type
For example, if my Public IP was 1234.5678 and I saved my key my-key.pem in my user home directory, I’d type
And you should be in your instance! A few quick notes in case you run into problems:

  • To access Terminal on Mac: Go to Search (Command + space) and then type ‘terminal’ and an icon of a black screen should show up. Click that / press enter.
  • Finding your file path:
    • on Mac: You can drag the file from Finder into your Terminal window and it’ll copy the file path for you.
  • If your file path contains spaces:
    • Either use the key ‘/’ before every space to escape it, or surround your file path with quote marks -> "filepath/my name/my-key.pem"
  • ‘Error: Permissions denied(publickey)’ -> Is your key correct? Given you haven’t done anything with your instance yet, worst case you can try terminating your instance and start a new one. This worked for me when I kept getting this error.
  • General debugging: type -v after your ssh ... command to make the command line print more stuff that might tell you where the problem is. Here  v stands for verbose.

IV. Setting up your instance

Again, BitFusion’s guide has good info on this. Here are some points they don’t cover so well:

  1. Move the files you need to the instance.
  • If you’ve saved your code in a GitHub repo (highly recommended), simply type (in the directory you want to save your work in, typically ipynb if your work includes iPython Notebooks because then you’ll be able to access them from the browser)
  • For data files > 100MB that you may have downloaded from the internet:
  1. copy the link you downloaded the file from and type
  2. curl my_link >>> choose_a_file_name.file_extension

For example, I wanted to download traffic sign data so I typed

curl <a href="https://d17h27t6h515a5.cloudfront.net/topher/2016/October/580d53ce_traffic-sign-data/traffic-sign-data.zip">https://d17h27t6h515a5.cloudfront.net/topher/2016/October/580d53ce_traffic-sign-data/traffic-sign-data.zip</a> &gt;&gt; traffic-sign-data.zip

into the console.

2. Change your Python version (Full tutorial from LinuxConfig)

The instance uses Python2.7 by default. If you  want to temporarily change it to use Python 3,

V. Cutting Costs

The key to not being charged hundreds of dollars per month is stopping your instance when you’re done with it. So once you’ve finished training your model, go back to the AWS Console  (browser) > EC2 > Instances (screen shown below).

stop-instance.png

  1. Select your instance.
  2. Click Actions -> to stop your instance.
  3. Check that it has stopped (vs is still ‘running’).

If you’re worried you’ll forget, you can set an AWS budget and ask it to send you email notifications when your monthly bill has exceeded e.g. $50.

screenshot.png

That’s it for now. If you have any questions or suggestions, do leave them in the comments.

Further reading:

2 Comments on “How to use AWS EC2 GPU instances with BitFusion”

  1. Maciej Bajkowski

    great write up Jessica. One small comment, instead of setting an alias for python3, you can simply type python3 instead of python to switch the python version when needed. Same for installing python3 packages, simply use pip3.

  2. Iftimie Alexandru

    Great post Jessica!! Since I found your blog via Udacity Self-Driving-Car ND (just 2 hours earlier) I can truly say that I learned a lot today. I hesitated between buying a laptop or a desktop for machine learning, now I will chose a decent laptop and go with AWS for machine learning.

Leave a Reply