The Difference Between Comparison to Np. Nan and Isnull()
I Supposed That Data[Data. Agefm. Isnull()] and Data[Data. Agefm == Numpy. Nan] Are Equivalent. but No, the First Truly Returns Rows Where Agefm Is Nan, but...
I supposed that
data[data.agefm.isnull()]
and
data[data.agefm == numpy.nan]
are equivalent. But no, the first truly returns rows where agefm is NaN, but the second returns an empty DataFrame. I thank that omitted values are always equal to np.nan, but it seems wrong.
agefm column has float64 type:
(Pdb) data.agefm.describe()
count 2079.000000
mean 20.686388
std 5.002383
min 10.000000
25% 17.000000
50% 20.000000
75% 23.000000
max 46.000000
Name: agefm, dtype: float64
Could you explain me please, what does data[data.agefm == np.nan] mean exactly?
1 Answer
np.nan is not comparable to np.nan... directly.
np.nan == np.nan
False
While
np.isnan(np.nan)
True
Could also do
pd.isnull(np.nan)
True
examples
Filters nothing because nothing is equal to np.nan
s = pd.Series([1., np.nan, 2.])
s[s != np.nan]
0 1.0
1 NaN
2 2.0
dtype: float64
Filters out the null
s = pd.Series([1., np.nan, 2.])
s[s.notnull()]
0 1.0
2 2.0
dtype: float64
Use odd comparison behavior to get what we want anyway. If np.nan != np.nan is True then
s = pd.Series([1., np.nan, 2.])
s[s == s]
0 1.0
2 2.0
dtype: float64
Just dropna
s = pd.Series([1., np.nan, 2.])
s.dropna()
0 1.0
2 2.0
dtype: float64
David Miller brings 15 years of experience in global economics, personal finance strategy, and market dynamics. He specializes in turning complex economic trends into actionable insights for everyday readers.
