How to fill NaN with a value in same row of the data frame using Pandas

Harry_pb

My data have 20 years of values of carbon emission and a column named average_emission. I want to replace all NaN in a particular row by the corresponding value of average_emission of the same row.

        1990    1991    1992    1993    1994    1995    1996    1997    1998    1999    ...                                                                           2002          2004    2005        2007          2009      2010    average_emission
0   29.620165   29.847608   25.258152   24.430481   22.996441   22.186615   21.641890   21.500661   19.621534   19.652589   ... 24.975515   24.909066   24.510543   24.964531   24.766706   25.613715   24.750133   24.876706   24.182702   24.344568
2   0.221827    0.194971    0.103776    0.092761    0.083184    0.075646    0.068592    0.061814    0.057051    0.043723    ... 0.041641    0.046108    0.040727    0.054855    0.065816    0.088141    0.158962    0.249074    0.302936    0.101080
3   0.398076    0.380695    0.372921    0.473691    0.307601    0.844306    0.781485    0.534831    0.515117    0.627072    ... 0.786223    0.543084    1.086605    1.069417    1.200877    1.311096    1.369425    1.430873    1.401654    0.794954
4   2.278387    1.215677    0.735198    0.726063    0.601347    0.654537    0.636625    0.490365    0.560271    0.960164    ... 1.229541    1.412697    1.376213    1.412498    1.291548    1.507536    1.580113    1.533178    1.515632    1.130908
5   3.203907    3.284094    3.483917    3.715328    3.735031    3.473291    3.364402    3.168027    3.360498    3.339701    ... 3.579922    3.797800    4.094269    4.206789    4.285595    4.181153    4.373573    4.575251    4.764912    3.775633
6   28.711160   29.798843   28.794559   31.007791   32.672833   30.057582   16.638557   16.047240   29.810631   27.173954   ... 24.956551   29.467089   28.481437   25.914587   23.954519   23.195067   23.033600   21.102296   20.120957   26.636164
7   3.440711    3.525384    3.608582    3.477160    3.569366    3.501875    3.648190    3.758387    3.798782    3.969876    ... 3.253313    3.475035    4.032407    4.111635    4.404525    4.496834    4.744178    4.427960    4.342272    3.853951
8      NaN           NaN    1.174674    0.859707    0.901719    1.083089    0.821585    1.044825    1.094343    0.988512    ... 0.998806    1.129174    1.204567    1.443731    1.459637    1.694755    1.868611    1.469961    1.422998    1.207623
10  4.857267    4.641623    4.566841    4.692005    4.683621    4.721298    4.593864    4.670562    4.496901    4.581279    ... 4.536211    4.804484    4.980996    4.974311    5.096290    5.561525    5.628319    5.906292    6.011269    4.895667
11  15.461262   15.133907   15.322589   15.706018   15.581310   15.600983   16.506006   16.516650   16.940007   17.192235   ... 17.372318   16.906198   17.030341   17.174386   17.293874   17.467002   17.704080   17.631833   16.710904   16.628071
DSM

If I understand correctly, you could do:

df = df.where(df.notnull(), df.average_emission, axis=0)

For example:

>>> df
         1990       1991       1992       1993
0   29.620165  29.847608  25.258152  24.430481
2    0.221827   0.194971   0.103776   0.092761
3    0.398076   0.380695   0.372921   0.473691
4    2.278387   1.215677   0.735198   0.726063
5    3.203907   3.284094   3.483917   3.715328
6   28.711160  29.798843  28.794559  31.007791
7    3.440711   3.525384   3.608582   3.477160
8         NaN        NaN   1.174674   0.859707
10   4.857267   4.641623   4.566841   4.692005
11  15.461262  15.133907  15.322589  15.706018
>>> df["average_emission"] = df.mean(axis=1)
>>> df.where(df.notnull(), df.average_emission, axis=0)
         1990       1991       1992       1993  average_emission
0   29.620165  29.847608  25.258152  24.430481         27.289102
2    0.221827   0.194971   0.103776   0.092761          0.153334
3    0.398076   0.380695   0.372921   0.473691          0.406346
4    2.278387   1.215677   0.735198   0.726063          1.238831
5    3.203907   3.284094   3.483917   3.715328          3.421812
6   28.711160  29.798843  28.794559  31.007791         29.578088
7    3.440711   3.525384   3.608582   3.477160          3.512959
8    1.017190   1.017190   1.174674   0.859707          1.017190
10   4.857267   4.641623   4.566841   4.692005          4.689434
11  15.461262  15.133907  15.322589  15.706018         15.405944

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Pandas: How to merge two data frames and fill NaN values using values from the second data frame

How to fill NaN values according to the data type in pandas data frame?

How to forward fill NaN values in pandas data frame?

pandas fill NaN in a data frame column using another data frame column

How to fill the data frame with using the match between columns and column list and value list using pandas?

How to drop row from pandas data frame containing nan

Python/Pandas: if value is NaN or 0 then fill with the value from the next column within the same row

How to go through a pandas data frame and only keep rows that have the same value throughout the entire row?

How do I create a new column in Pandas data frame with the value determined by other columns in the same row

Pandas:How to fill in value from the same column but different row

Pandas data frame: replace nan with the mean of that row

How to fill rows with data from another row with the same reference value?

Pandas: fill nan using previous value and interpolating

How to color (fill in) just one column (whose value in the data frame belongs to a row) of different colors using geom_bar in R?

How to count entries of the same value for each individual row of a data frame?

Fill NaN in second level of multi indexed pandas data frame

how do you concatenated column value for row in pandas data frame?

Get row-index of the last non-NaN value in each column of a pandas data frame

To fit polynomial function to fill the NaN value in my data frame

How to fill in pandas data frame of unknown size

Using Pandas to conditionally remove row and fill value

How to plot percentage of NaN in pandas data frame?

How to create a data frame with a certen row value + the next value on the same row?

How to get the whole row from data frame having maximum value for every 7 records in the pandas data frame?

Random value for each row in pandas data Frame

Split pandas data frame columns by row value

How to subtract value in one row of pandas Data Frame from value in another row without for loop?

check on element in a row in a data frame in pandas is NaN then replace it

Python Check existing value row by row from the Data Frame using Pandas