Forum Discussion

mrojze's avatar
mrojze
Icon for Helper II rankHelper II
3 years ago
Solved

Loaded a JSON file in OneLake... Now what?

Hi there, I successfully called an API and saved a JSON file in the Files section of the Lakehouse. Now, I am trying to read the file and unpack the datasets inside. I'd love to use the Dataflows ...
Dataflows
Need Help
  • DennesTorres's avatar
    DennesTorres
    3 years ago

    Hi,

    You can do it using a pipeline and the copy activity, you can even using the copy assistant to help you building it.


    The dataflows can't be used for this.

    The pipeline needs to be in the same workspace as the lakehouse, but it will ask you if your source is the "Files" or "Tables" area. The Dataflow Gen 2, on the other hand, doesn't need to be in the same workspace, but only accepts the "Tables" area as source or destination.

    Kind Regards,

     

    Dennes

mrojze's avatar
mrojze
Icon for Helper II rankHelper II
3 years ago

Thanks for your tips.

I now was able to use the Dataflow to access my Files section. I didn't realize there was a Lakehouse connector. I was going directly to the JSON connector instead.

But also, I overlooked that in the Get Data section, there was a OneLake data hub (such a newbie)

Now, to the next issue. You can't pass parameters to a Dataflow yet!!!!

I guess you can save the data somewhere and bring is as a dataset, but you can't pass parameters in a Fabric Dataflow yet. See https://community.fabric.microsoft.com/t5/Data-Pipelines/Pass-data-pipeline-parameter-to-dataflow-gen2/m-p/3264346

 

I am back to build an ADF copy per section. 

But now I have to learn how to create a Table in Delta Lake. So far, I wasn't able to. I tried SQL commands, but CREATE is not available yet.

 

 

DennesTorres's avatar
DennesTorres
Icon for Power Participant rankPower Participant
3 years ago

Hi,

You can use sparkSQL in a notebook to create a table in the lakehouse:


Kind Regards,

 

Dennes