When running machine learning experiments, you might want to run multiple scripts simultaneously, hide printouts for a script or just do things in a terminal window while running a Jupyter Notebook in the background.
In this post, we will go through how to run scripts in the background, bring them back to the foreground, and check if the scripts are still running.
Running scripts in the background
Suppose you’ve already started running your script, python script.py. Then:
Ctrl+Z to pause the script.
Python12^Z+ Stopped python script.py
- You might see
bg to run the script in the background. You should see
Python1+ python script.py &
- Press Ctrl+Z to pause the script.
- OR type
fg to run the script in the foreground. You should see
Python1+ python script.py &
- OR type fg to run the script in the foreground. You should see
You can also run the script in the background directly by typing
python script.py &
in the console. The & symbol instructs the process to run in the background. E.g. I often run jupyter notebook &.
Sometimes you may want to check if a process is still running, how long a process has been running or whether it is hanging. (Hanging here means the program is stuck or is not responding to inputs.)
ps -x to list all processes (that are executables).
- If you’re on your home computer as opposed to a remote server, there may be many processes running, and you may have to run
ps -x | grep python or
ps -x | grep script.py instead to find your script.
- This finds all processes with the word
python in them.
- | pipes the output of the first command ( ps -x ) to the input of the second command ( grep [word to search] [files to search] ).
- grep python files_to_search finds instances of the string python in files_to_search.
- This finds all processes with the word python in them.
- If you’re on your home computer as opposed to a remote server, there may be many processes running, and you may have to run ps -x | grep python or ps -x | grep script.py instead to find your script.
- Find the id of your process.
123[pid] [tty] [time script's been running for] [script name]2939 ttys003 0:01.60 python script.py2949 ttys003 0:00.00 grep python
- pid stands for process ID.
- tty stands for teletype terminals, which were the terminals people used when people first started to use computers.
- Type ps -x to list all processes (that are executables).
- If you are on Linux, you can run pstack $ID, which should print out the ongoing output of your process. If the process is not hanging, you should see a lot of continuous output which will suggest which part of the program is running. If it is hanging, there likely won’t be many (if any) continuing printouts.
- If you want to stop the process, you can type
- If you want to stop the process while it is in the foreground, type Ctrl+C.
- You can use this to e.g. get rid of an experiment that isn’t responding.
I hope this has been helpful! You can try running a few Python scripts using the same terminal or debugging your Python scripts using this method.