Manipulate pandas DataFrame

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Reshape DataFrame¶

import pandas as pd
import numpy as np

df1 = pd.DataFrame({"x": [1, 2, 3], "y": [5, 4, 3]})
df1

df2 = pd.DataFrame({"x": [0, 0, 3000], "y": [78, 4, 3]})
df2

Melt¶

df = pd.DataFrame(
    {"id": ["a", "b", "c"], "x": [1, 3, 5], "y": [2, 4, 6], "z": [7, 8, 9]}
)
df

dfm = pd.melt(df, id_vars="id", value_vars=["x", "y", "z"])
dfm

Stack¶

df = pd.DataFrame(
    {"id": ["a", "b", "c"], "x": [1, 3, 5], "y": [2, 4, 6], "z": [7, 8, 9]}
)
df

x = df.stack()
x

0  id    a
   x     1
   y     2
   z     7
1  id    b
   x     3
   y     4
   z     8
2  id    c
   x     5
   y     6
   z     9
dtype: object

x.unstack()

type(x)

pandas.core.series.Series

Insert Columns - pandas.DataFrame.insert¶

df = pd.DataFrame(
    {"id": ["a", "b", "c"], "x": [1, 3, 5], "y": [2, 4, 6], "z": [7, 8, 9]}
)
df

df.insert(0, "col", 0)

df

reshape¶

overlap.spot_sample_detail_id.reshape(100, 3)

rename¶

df = pd.DataFrame(
    {"id": ["a", "b", "c"], "x": [1, 3, 5], "y": [2, 4, 6], "z": [7, 8, 9]}
)
df

df.rename(columns=df.iloc[0])

help(df.rename)

Help on method rename in module pandas.core.frame:

rename(index=None, columns=None, **kwargs) method of pandas.core.frame.DataFrame instance
    Alter axes input function or functions. Function / dict values must be
    unique (1-to-1). Labels not contained in a dict / Series will be left
    as-is. Extra labels listed don't throw an error. Alternatively, change
    ``Series.name`` with a scalar value (Series only).
    
    Parameters
    ----------
    index, columns : scalar, list-like, dict-like or function, optional
        Scalar or list-like will alter the ``Series.name`` attribute,
        and raise on DataFrame or Panel.
        dict-like or functions are transformations to apply to
        that axis' values
    copy : boolean, default True
        Also copy underlying data
    inplace : boolean, default False
        Whether to return a new DataFrame. If True then value of copy is
        ignored.
    level : int or level name, default None
        In case of a MultiIndex, only rename labels in the specified
        level.
    
    Returns
    -------
    renamed : DataFrame (new object)
    
    See Also
    --------
    pandas.NDFrame.rename_axis
    
    Examples
    --------
    >>> s = pd.Series([1, 2, 3])
    >>> s
    0    1
    1    2
    2    3
    dtype: int64
    >>> s.rename("my_name") # scalar, changes Series.name
    0    1
    1    2
    2    3
    Name: my_name, dtype: int64
    >>> s.rename(lambda x: x ** 2)  # function, changes labels
    0    1
    1    2
    4    3
    dtype: int64
    >>> s.rename({1: 3, 2: 5})  # mapping, changes labels
    0    1
    3    2
    5    3
    dtype: int64
    >>> df = pd.DataFrame({"A": [1, 2, 3], "B": [4, 5, 6]})
    >>> df.rename(2)
    Traceback (most recent call last):
    ...
    TypeError: 'int' object is not callable
    >>> df.rename(index=str, columns={"A": "a", "B": "c"})
       a  c
    0  1  4
    1  2  5
    2  3  6
    >>> df.rename(index=str, columns={"A": "a", "C": "c"})
       a  B
    0  1  4
    1  2  5
    2  3  6