Btukfyl

I'm aware about the replace function in pandas: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.replace.html

But I've done this simple test and it is not working as expected when I try to replace a float value:

import pandas as pd

import numpy as np



df = pd.DataFrame(np.random.randn(50, 4), columns=list('ABCD'))

print(df.head(n=1))



      A         B        C         D

0  1.437202  1.919894 -1.40674 -0.316737



df = df.replace(1.437202, np.nan)

print(df.head(n=1))



      A         B        C         D

0  1.437202  1.919894 -1.40674 -0.316737

As you see the [[0],[0]] has no change...any idea about what this could be due to?

Problem is float precision, so use function numpy.isclose with mask:

np.random.seed(123)

df = pd.DataFrame(np.random.randn(50, 4), columns=list('ABCD'))

print(df.head(n=1))

          A         B         C         D

0 -1.085631  0.997345  0.282978 -1.506295



df = df.mask(np.isclose(df.values, 0.997345))

Or use numpy.where:

arr = np.where(np.isclose(df.values, 0.997345), np.nan, df.values)

df = pd.DataFrame(arr, index=df.index, columns=df.columns)

print(df.head(n=1))

          A   B         C         D

0 -1.085631 NaN  0.282978 -1.506295

EDIT: You can also get only numeric columns by select_dtypes for filtering by subset with :

np.random.seed(123)

df = pd.DataFrame(np.random.randn(50, 4), columns=list('ABCD')).assign(E='a')



cols = df.select_dtypes(np.number).columns

df[cols] = df[cols].mask(np.isclose(df[cols].values, 0.997345))

print(df.head(n=1))

          A   B         C         D  E

0 -1.085631 NaN  0.282978 -1.506295  a

Just a another trick for specific indices :

>>> print(df.head(n=1))

          A         B         C         D

0 -0.042839  1.701118  0.064779  1.513046



>>> df['A'][0] = np.nan



>>> print(df.head(n=1))

    A         B         C         D

0 NaN  1.701118  0.064779  1.513046