in

Transpose Columns in spark Dataframe using scala


I am would like to find a way to transpose columns in a spark dataframe. I have made a spark scala code that count the number of null values in each column of my dataframe.

My dataframe look like this:

+——+—–+
|name |state|
+——+—–+
| |CA |
|Julia | |
|Robert|null |
|null | |
+——+—–+

with my scala code:

*

val data = Seq(("","CA"), ("Julia",""),("Robert",null),(null,""))
import spark.sqlContext.implicits._
val df = data.toDF("name","state")
def countCols(columns:Array[String]):Array[Column]={
    columns.map(c=>{
      count(when(col(c).isNull,c)).alias(c)
    })
}
df.select(countCols(df.columns):_*).show

I have this result:
+—-+—–+
|name|state|
+—-+—–+
| 1| 1|
+—-+—–+
How is it possible to get the result like this:
+—- +—– +
|name |1 |
+—- +—– +
|state|1 |
+—- +—– +

Thank you for your help



Source: https://stackoverflow.com/questions/70610000/transpose-columns-in-spark-dataframe-using-scala

Details,PoC and patches for CVE-2021-45383 & CVE-2021-45384

Ashers 'gay cake' case: European court rules case inadmissible thumbnail

Ashers ‘gay cake’ case: European court rules case inadmissible