Forum Discussion

DebbieE's avatar
DebbieE
Icon for Community Champion rankCommunity Champion
2 years ago
Solved

Fabric Pyspark Help. Adding a Min into a group by agg code in Notebooks

It would be really useful if we had a Pyspark forum   Im SQL through and through and learning Pyspark is a NIGHTMARE I have the following code that finds all the contestants with more that one rec...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi DebbieE ,

    Thanks for using Fabric Community.
    As I understand -

     



    Spark SQL Code:

     

    df = spark.sql("SELECT Min(CustomerID), CompanyName, Count(*) FROM gopi_lake_house.customer_table1 group by CompanyName having count(*)>1")
    display(df)

     


    Pyspark Code:

     



    Can you please try below code -

     

     

    from pyspark.sql.functions import *
    
    
    
    result = dfcont.groupBy('CompanyName')\
            .agg(min('CustomerID').alias('minCustomerID'), count('CustomerID').alias('TotalRecords'))\
            .filter(col('TotalRecords') > 1)\
            .show(1000)

     



    Hope this is helpful. Please let me know incase of further queries.