First, I want to express my gratitude to my mentor and tech lead “Group” from my 2023 team. Much of the knowledge shared below was learned from him during our time working together.

1. How to run a Python script in background

nohub python script.py >script.log 2>&1 &

output:

[1] 12345

Explanation:

  • nohub: short for no hang up, it’s ignore the hangup signal, keep processes running even after exiting the shell or terminal
  • python: run the script using the Python interpreter
  • script.py: the name of the script to run
  • \>script.log: the output of the script will be redirected to the file script.log
  • 2>&1: the standard error output (stderr) will be redirected to the same file as the standard output (stdout)
    • 0: stdin
    • 1: stdout
    • 2: stderr
    • 2>&1: redirect the stderr to stdout (script.log)
  • &: tell the shell to run the command as a job in the background
  • 12345 is the process ID (PID) of the script

To view the log:

tail -f script.log

Explanation:

  • tail: print the last 10 lines of a file
  • -f: follow, continuously monitor the file for new lines

To see all python processes:

ps -x | grep python

Explanation:

  • ps: print information about processes
  • -x: include processes which do not have a controlling terminal
  • grep python: filter the output to show only lines containing the word “python”

To kill the process:

kill 12345

2. Keep the container stay alive

In the Dockerfile:

CMD ["tail", "-f", "/etc/hosts"]

Explanation:

  • tail: print the last 10 lines of a file
  • -f: follow, continuously monitor the file for new lines
  • /etc/hosts: the file to monitor, which is a file that always exists
  • So the container always runs tail -f /etc/hosts, which keeps the container alive

Or in the yaml file:

["sh", "-c", "tail -f /etc/hosts"]

Explanation:

  • sh: run a shell command
  • -c: execute the command that follows

3. Combine

  1. Using tail -f to keep the container stay alive
  2. Using nohub to run the script, for example with openshift:
oc rsh <pod name>
nohup python script.py >script.log 2>&1 &
sleep 1
tail -f script.log