topic sql command in SparkR in Data Engineering

sql command in SparkR

bosho — Thu, 29 Feb 2024 12:37:33 GMT

Hi,

I'm following along with the SparkR demo for Fabric notebooks that's on the microsoft website here:
https://learn.microsoft.com/en-us/fabric/data-science/r-use-sparkr

However, when I try to run any of the sql commands in a cell, instead of actually processing the command my notebook seems to just print out the SQL statement. Any ideas what I might be doing wrong?

Cheers,

Re: sql command in SparkR

HimanshuS-msft — Thu, 29 Feb 2024 21:52:59 GMT

Hello @bosho ,

Can you please share the test code which you are running ?

Thanks
Himanshu

Re: sql command in SparkR

govindarajan_d — Fri, 01 Mar 2024 10:02:07 GMT

Hi @bosho,

Did you use the SQL command like this:

sqlquery <- sql("SELECT * FROM TABLE")

Re: sql command in SparkR

Anonymous — Fri, 01 Mar 2024 11:02:29 GMT

Hi @bosho ,

Thanks for using Fabric Community.
I cannot find issue while I am executing the below code -

Can you please share your test code so I can guide you better?

Re: sql command in SparkR

bosho — Fri, 01 Mar 2024 11:06:28 GMT

Thanks for the quick responses. Yes trying it again today I don't seem to be able to replicate the issue, and I see what you're seeing. Previously it was just printing the SQL code back but today I'm getting what you're getting.

Re: sql command in SparkR

Anonymous — Fri, 01 Mar 2024 11:08:59 GMT

Hi @bosho ,

Glad to know that you no longer see the issue. Please continue using Fabric Community for your further queries.

Re: sql command in SparkR

bosho — Fri, 01 Mar 2024 15:12:04 GMT

Have since seen it's beecause I had loaded tidyverse in that notebook to work on some R stuff, and dplyr's sql() command seems to mask SparkR's. Think that's why it wasn't executing it on my spark dataframe.