Get row-index of the last non-NaN value in each column of a pandas data frame

ahat

How can I return the row index location of the last non-nan value for each column of the pandas data frame and return the locations as a pandas dataframe?

EdChum

Use notnull and specifically idxmax to get the index values of the non NaN values

In [22]:

df = pd.DataFrame({'a':[0,1,2,NaN], 'b':[NaN, 1,NaN, 3]})
df
Out[22]:
    a   b
0   0 NaN
1   1   1
2   2 NaN
3 NaN   3
In [29]:

df[pd.notnull(df)].idxmax()
Out[29]:
a    2
b    3
dtype: int64

EDIT

Actually as correctly pointed out by @Caleb you can use last_valid_index which is designed for this:

In [3]:
df = pd.DataFrame({'a':[3,1,2,np.NaN], 'b':[np.NaN, 1,np.NaN, -1]})
df

Out[3]:
    a   b
0   3 NaN
1   1   1
2   2 NaN
3 NaN  -1

In [6]:
df.apply(pd.Series.last_valid_index)

Out[6]:
a    2
b    3
dtype: int64

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

data.frame: find last index of a value in each row

Get last non-NaN value for each month in pandas

Get column indices for non-zero values in each row in pandas data frame

Get Value of last non-empty column for each row

Python Dataframe Get Value of Last Non Null Column for Each Row

How to get value of last valid index of a different column for each row

Pandas, get first and last column index for row value

Pandas - Get last non Nan value plus count number of non Nan in column on a rolling basis

Get highest date value of each row in a pandas data frame

How to get the index of the mode value of a specific column in a pandas data frame

Random value for each row in pandas data Frame

How to find last index in Pandas Data Frame row and count backwards using column information?

Pandas find last positive value in each column and Replace NaN with that value

Counting changes of value in each column in a data frame in pandas ignoring NaN changes

Pandas data frame : how to check all value of last row if it is non zero

find the set of column indices for non-zero values in each row in pandas' data frame

Pandas: Last time when a column had a non-nan value

How to print index value, column name, and column data for each row of pandas dataframe?

How to fill NaN with a value in same row of the data frame using Pandas

Find the column index of an element for each row of a data frame

Pandas how to get the first and last dates of all non-zero value ranges in a data frame?

How to define conditional for pandas column of several last row in data frame?

for every row find the last column with value 1 in binary data frame

Get value of last non-NA row per column in data.table

For each row return the column index and name of non-NA value

Python: Get last row from Pandas Data Frame

How to get last five consecutive values in pandas data frame row?

How can i get last row in python pandas data frame?

Determine the column with maximum value for each data frame row, accounting for ties