Ben Chuanlong Du's Blog

And let it direct your passion with reason.

The Right Way to Export PATH in Shell

Some people suggest exporting PATH only in .bash_profile instead of in .bashrc (for Bash). The helps but does not resolve the issue of possible duplicated paths in $PATH. The right way is to check for existence of the path in the $PATH environment variable first, and add it only when …

Hands on the Python Library pexpect

Tips and Traps

  1. The command-line tool of some (e.g., network) applications might be slow to authenticate. If you use pexect to automate such a command-line tool, it is best to wait for sometime after sending password using child.sendline(passwd). If the authentication has ouput on both success and failure, a smart way is to wait for the success or failure message to come out.

IPython Is the Best Shell

IPython is the Best Shell!

  1. Use IPython as much as possible.

    • IPython has virtues of both shell and Python.
    • You should avoid using shell scripts for complicate tasks any way.
  2. If you do want to use a Unix/Linux shell, I'd suggest you stick to Bash unless Linux distributions start …

Change Shell in Mac

You can check the current shell in use by running the following command.

echo $SHELL

However, notice that echo $SHELL does not report the right shell sometimes. A more robust way is to use echo $0.

The default shell can be changed using thee comamnd chsh. For example, thee following …

Spark Issue: Shell Related

Symptom 1

/bin/sh: hdfs: command not found

Possible Causes of Symptom 1

The command hdfs is not on the search path.

Possible Solutions to Symptom 1

  1. Use the full path to the command.
  2. Configure the environment variable PATH before you use the command.
  3. Find other alternatives to the command …