Ben Chuanlong Du's Blog

And let it direct your passion with reason.

Working with Iterators in Python

Iterator vs Generator

Generator is a special case of Iterator. Generator is easy and convenient to use but at additional cost (memory and speed). If you need performance, use plain iterator (with the help of the itertools module). If you need convenience and concise code, use generator.

Please refer to Python Generator vs Iterator for more detailed discussions.

Install Python Packages Using pip

PyPi Statistics

You can check download statistics of Python Packages on PYPI at https://pypistats.org/. This is especially helpful if you want to choose from multiple packages.

Prefer pip

pip is preferred over OS tools (e.g., apt-get, yum, wajig, aptitude, etc.) for managing Python packages. If you are …

The set Collection in Python

General Tips and Traps

  1. The set class is implemented based on hash table which means that its elements must be hashable (has methods __hash__ and __eq__). The set class implements the mathematical concepts of set which means that its elements are unordered and does not perserve insertion order of elements. Notice that this is different from the dict class which is also implemented based on hash table but keeps insertion order of elements! The article Why don't Python sets preserve insertion order?

Tips on pex

Steps to Build a pex Environment File

  1. Start a Python Docker image with the right version of Python interpreter installed. For example,

    docker run -it -v $(pwd):/workdir python:3.5-buster /bin/bash

  2. Install pex.

    pip3 install pex

  3. Build a pex environment file.

    pex --python=python3 -v pyspark findspark -o …

Regular Expression in Python

Online Regular Expression Tester

  1. The Python module re automatically compiles a plain/text pattern using re.compile and caches it, so there's not much benefit to compile plain/text patterns by yourself.

  2. Some regular expression patterns are defined using a single leading backslash, e.g., \s, \b, etc. However, since special characters (e.g., \) need to be escaped in strings in most programming languages, you will need the string "\\s"