Forum Discussion

jccastelblancos's avatar
jccastelblancos
New Member
2 years ago
Solved

implementation of R in notebook

Good evening and greetings to all I come to this forum with the intention of requesting help with the following, load a table in a lakeHouse through a pipeline in the tables section I am reading it...
data engineering
  • jccastelblancos's avatar
    jccastelblancos
    2 years ago

    TomMartens;

    Anonymous 

    I found a solution, which I think may be better, but for now it is my contribution in case anyone else facing the same solution has an alternative.
    Since you are working as a pipeline to get the data to the lakehouse, change the destination so that it is not a table but a file, that is, files/. After this we can load it with readr and work with tidyverse, here the cumbersome thing is that to take it to tables I had no alternative but to write the files/processed table again, then read with SparkR from there to finally be able to use the write procedure which writes it to tables with overwrite or append mode. It is a longer route than I expected but I hope that if anyone knows a better alternative, let us know through this thread. Thanks to everyone who came to my aid.