Jessica Yung Navigation
  • Home
  • About
    • What I’m doing now
    • GitHub
  • Projects
    • Hello Motions
    • How I’ve been learning to code
    • Machine Learning Nanodegree
    • Maths Resources
    • Tools
  • Blog
  • Contact
  • Home
  • About
    • What I’m doing now
    • GitHub
  • Projects
    • Hello Motions
    • How I’ve been learning to code
    • Machine Learning Nanodegree
    • Maths Resources
    • Tools
  • Blog
  • Contact

Tag Archive

data preprocessing

Using Bash Scripts to Parallelise Data Preprocessing for Machine Learning Experiments

Jessica Yung10.2018Data Science, ProgrammingLeave a Comment

Parallelising data preprocessing can save you a lot of time. In this post, we’ll go through how to use bash scripts to make parallelising computation easier. The idea is that you split up the data you need to preprocess into different batches, and you run a few batches on each machine. The bash scripts help you loop through batches to … Read More

Top Posts

  • How to use pickle to save and load variables in Python
  • Automate running a script using crontab
  • How to run scripts in the background
  • Python Lists vs Dictionaries: The space-time tradeoff
  • LSTMs for Time Series in PyTorch
  • Numpy Views vs Copies: Avoiding Costly Mistakes
  • MSE as Maximum Likelihood
  • Using generators in Python to train machine learning models
  • How Python implements dictionaries
  • What makes Numpy Arrays Fast: Memory and Strides

Recent Comments

  • Jessica Yung on Explaining Tensorflow Code for a Convolutional Neural Network
  • Jessica Yung on Self-Driving Car Engineer Nanodegree
  • KL Tah on Self-Driving Car Engineer Nanodegree
  • Jessica Yung on Self-Driving Car Engineer Nanodegree Term 1 Review
  • Jessica Yung on Self-Driving Car Engineer Nanodegree Term 1 Review

Archives

  • October 2018
  • September 2018
  • August 2018
  • June 2018
  • March 2018
  • January 2018
  • December 2017
  • October 2017
  • September 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • May 2016
  • April 2016
  • March 2016
  • September 2015
  • July 2015
  • May 2015
  • January 2015
  • November 2014
  • March 2014
  • September 2013
  • December 2012
  • June 2012
  • February 2012
  • June 2010
  • March 2010
  • November 2009
  • October 2009

Categories

  • Artificial Intelligence
  • Careers
  • Data Science
  • Economics
  • Education
  • Engineering
  • Entrepreneurship
  • Highlights
  • Life
  • Machine Learning
  • Mathematics
  • Poetry
  • Poetry 2009
  • Poetry 2010
  • Programming
  • Python
  • Reflections
  • Self-Driving Car ND
  • Statistics
  • Studying
  • Talk Reviews
  • Technology Article Summaries
  • Uncategorized
  • Writing

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org