In order to pass the entire row as an additional argument to Spark UDF in Scala I use struct("*"), for example:
df.select(myUDF($"name",struct("*"))) How to do the same in PySpark?
Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about CollectivesStack Internal
Knowledge at work
Bring the best of human thought and AI automation together at your work.
Explore Stack InternalIn order to pass the entire row as an additional argument to Spark UDF in Scala I use struct("*"), for example:
df.select(myUDF($"name",struct("*"))) How to do the same in PySpark?