I have the below spark dataframe.
Name age subject parts
xxxx 21 Maths,Physics I
yyyy 22 English,French I,II
I am trying to explode the above dataframe in both subject and parts like below.
Expected output:
Name age subject parts
xxxx 21 Maths I
xxxx 21 Physics I
yyyy 22 English I
yyyy 22 English II
yyyy 22 French I
yyyy 22 French II
I tried using array.zip for subject and parts and then tried to explode using the temp column, but I am getting null values in the place where there is only one part.
Is there a way to achieve this in Pyspark.
I did this by passing columns as list to a for loop and exploded the dataframe for every element in list
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments