How to run scripts in the background

Jessica YungProgramming

When running machine learning experiments, you might want to run multiple scripts simultaneously, hide printouts for a script or just do things in a terminal window while running a Jupyter Notebook in the background.

In this post, we will go through how to run scripts in the background, bring them back to the foreground, and check if the scripts are still running.

Running scripts in the background

Suppose you’ve already started running your script, python script.py.  Then:

    1. Press Ctrl+Z to pause the script.
          1. You might see
    2. Type bg to run the script in the background. You should see
    1. OR type fg to run the script in the foreground. You should see
      and the script continuing to run.

You can also run the script in the background directly by typing

in the console. The & symbol instructs the process to run in the background. E.g. I often run jupyter notebook &.

Inspecting processes

Sometimes you may want to check if a process is still running, how long a process has been running or whether it is hanging. (Hanging here means the program is stuck or is not responding to inputs.)

    1. Type ps -x to list all processes (that are executables).
      1. If you’re on your home computer as opposed to a remote server, there may be many processes running, and you may have to run ps -x | grep python or ps -x | grep script.py instead to find your script.
        1. This finds all processes with the word python in them.
          1. | pipes the output of the first command ( ps -x ) to the input of the second command ( grep [word to search] [files to search] ).
          2. grep python files_to_search  finds instances of the string python in files_to_search.
    2. Find the id of your process.
      1. pid stands for process ID.
      2. tty stands for teletype terminals, which were the terminals people used when people first started to use computers.
  1. If you are on Linux, you can run pstack $ID, which should print out the ongoing output of your process. If the process is not hanging, you should see a lot of continuous output which will suggest which part of the program is running. If it is hanging, there likely won’t be many (if any) continuing printouts.
  2. If you want to stop the process, you can type kill $ID.
    1. If you want to stop the process while it is in the foreground, type Ctrl+C.
    2. You can use this to e.g. get rid of an experiment that isn’t responding.

I hope this has been helpful! You can try running a few Python scripts using the same terminal or debugging your Python scripts using this method.

References: