Forum Discussion

jccastelblancos's avatar
jccastelblancos
New Member
2 years ago
Solved

implementation of R in notebook

Good evening and greetings to all I come to this forum with the intention of requesting help with the following, load a table in a lakeHouse through a pipeline in the tables section I am reading it...
data engineering
  • jccastelblancos's avatar
    jccastelblancos
    2 years ago

    TomMartens;

    Anonymous 

    I found a solution, which I think may be better, but for now it is my contribution in case anyone else facing the same solution has an alternative.
    Since you are working as a pipeline to get the data to the lakehouse, change the destination so that it is not a table but a file, that is, files/. After this we can load it with readr and work with tidyverse, here the cumbersome thing is that to take it to tables I had no alternative but to write the files/processed table again, then read with SparkR from there to finally be able to use the write procedure which writes it to tables with overwrite or append mode. It is a longer route than I expected but I hope that if anyone knows a better alternative, let us know through this thread. Thanks to everyone who came to my aid.

  • jccastelblancos's avatar
    jccastelblancos
    New Member
    2 years ago

    TomMartens  Anonymous 

    It turned out to be much more complex than I thought, for now what I did was use createOrReplaceTempView to use sql with sql().
    I expected it to be simpler but I have had countless problems with the use of SparkR, I think it is more designed for PySpark than for tidyverse. Still, I'll try a few more things. Thanks for the assistance

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi jccastelblancos ,

      Glad to know that you got some insights over your query. Please continue using Fabric Community on your further queries.