Forum Discussion
silly_bird
1 year agoRegular Visitor
Help extracting value from dict in a column
Hi all I'm working on API integration in PySpark notebook and there is a column with email & phone that is an array with random order contactMethods = [{'name': 'Email', 'value': 'e...
- 1 year ago
It looks like I managed to figure one solution.
Don't know good or bad, it is my only one
from pyspark.sql.functions import col, udf from pyspark.sql.types import StringType extract_email = udf(lambda cell: str(next(filter(lambda t: t["name"] == "Email", cell), {}).get("value", "")), StringType()) extract_mobile = udf(lambda cell: str(next(filter(lambda t: t["name"] == "Mobile", cell), {}).get("value", "")), StringType()) df = df.withColumn('email', extract_email(col("contactMethods"))).withColumn('mobile', extract_mobile(col("contactMethods"))) display(df)Profies, please advise!
Anonymous
1 year agoNot applicable
Hi silly_bird ,
It looks like you have found a solution. Could you please mark this helpful post as “Answered”?
This will help others in the community to easily find a solution if they are experiencing the same problem as you.
Thank you for your cooperation!
Best Regards,
Yang
Community Support Team
If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!