Ben Chuanlong Du's Blog

It is never too late to learn.

Hands on the tempfile Module in Python

https://docs.python.org/3/library/tempfile.html

This module creates temporary files and directories. It works on all supported platforms. TemporaryFile, NamedTemporaryFile, TemporaryDirectory, and SpooledTemporaryFile are high-level interfaces which provide automatic cleanup and can be used as context managers. mkstemp() and mkdtemp() are lower-level functions which require manual cleanup.

In [1]:
import os
import tempfile
from pathlib import Path

tempfile.TemporaryFile

In [3]:
fp = tempfile.TemporaryFile()
In [6]:
fp.write(b"how are you")
Out[6]:
11
In [7]:
fp.seek(0)
Out[7]:
0
In [8]:
fp.read()
Out[8]:
b'how are you'
In [9]:
fp.close()
In [4]:
type(fp)
Out[4]:
_io.BufferedRandom
In [14]:
fp.name
Out[14]:
58
In [12]:
with tempfile.TemporaryFile() as fp:
    fp.write(b"how are you")
    fp.seek(0)
    print(fp.read())

b'how are you'

tempfile.TemporaryDirectory

Notice that after finish of the with block, the temporary directory is removed automatically.

In [2]:
with tempfile.TemporaryDirectory() as tempdir:
    print(tempdir)

/tmp/tmpwnml8m3t
In [3]:
type(tempdir)
Out[3]:
str
In [9]:
Path(tempdir).exists()
Out[9]:
False

tempfile.mkstemp

  1. The returned file descriptor must be closed (using os.close manually or using a contextual manager) to avoid leak of resource!

  2. The suffix option can be used to create a temporary file with a specific file extension.

Below are examples of using tempfile.mkstemp in correct ways. The first example close the file descriptor manually using os.close. The second example close the file automatically using the with block.

In [2]:
fd, file = tempfile.mkstemp()
print(file)
os.write(fd, b"how are you")
os.close(fd)

/tmp/tmp90pu002w
In [3]:
!cat {file}

how are you
In [4]:
fd, file = tempfile.mkstemp()
print(file)
with os.fdopen(fd, "w") as fout:
    fout.write("I am fine.")

/tmp/tmp4shh3pnx
In [5]:
!cat {file}

I am fine.

Create a temporary SQL file.

In [2]:
tempfile.mkstemp(suffix=".sql")
Out[2]:
(57, '/tmp/tmpzjxy8tg9.sql')

tempfile.mkdtemp

In [3]:
import tempfile

tempfile.mkdtemp()
Out[3]:
'/tmp/tmpwusvuq3c'

Set a Different Temporary Directory

In certain situations, you might want to set a different temporary directory than the default one. For example, if an application leverages temporary files/dirs extensively while there's very limited space in the temporary directory (commonly seen when the application is running in a container environment on cloud), you can run into "OSError: No space left on the device". An easy fix of this issue is to set environment variables to point the temporary directory to an existing directory with enough space which is writable by the user. The code below illustrate how to change the default temporary directory. Notice that tempfile.tempdir = None is added to clear possible caching. For more details, please refer to tempfile.gettempdir .

In [1]:
import tempfile
import os
from pathlib import Path

tempfile.tempdir = None
os.environ["TMPDIR"] = str(Path.home())
tempfile.mkstemp()
Out[1]:
(57, '/Users/dclong/tmptci7lsir')
In [ ]:

Comments