Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Simple Use Case¶
The usage of the tqdm module is really simple.
All you need is to import the tqdm function from the tqdm module,
and then wrap the function tqdm over the iterable object that you want iterate over.
This is really convenient if you have a long running task
and want to report the progress of the task.
from tqdm import tqdm
arr = [i**2 for i in tqdm(range(100000))]100%|██████████| 100000/100000 [00:00<00:00, 1151357.70it/s]
You can specify the (guessed) total number of iterations using the argument total.
import time
for i in tqdm(range(100), total=10):
time.sleep(1)100it [01:40, 1.00s/it]
import time
for i in tqdm(range(10), total=100):
time.sleep(1) 10%|█ | 10/100 [00:10<01:30, 1.00s/it]
Notice that trange(i) is a special optimised instance of tqdm(range(i)).
from tqdm import trange
arr = [i**2 for i in trange(0, 100000, 10)]100%|██████████| 10000/10000 [00:00<00:00, 868116.32it/s]
tqdm with multiprocessing¶
The trick is to use multiprocessing.Pool.imap
and use tqdm outside the map (rather than inside it).
Notice that the time reported might not accurate (especially at the beginning and with long-running tasks).
As more tasks get finished,
the ETA is more reliable.
import multiprocessing as mp
from tqdm import tqdm
with mp.Pool(4) as pool:
for _ in tqdm(pool.imap(task, params), total=total):
passCommand-line and Pipe¶
tqdm can also be executed as a module with pipes.
seq 9999999 | tqdm --bytes | wc -l75.2MB [00:00, 140MB/s]
9999999