Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
ThiyagarajanLG
Frequent Visitor

Connect ADF Copy Data to Fabric Data Lake House

Hi,

 

I am using my corporate account, can some one help me to show the settings/permissions required to connect ADF CopyData to Fabric Data Lake house.

I tried, but not abl to connect the Fabric account from Azure account.

 

11 REPLIES 11
ThiyagarajanLG
Frequent Visitor

Hi Nikhil,

 

Basically, i would like to read a .csv file from Azure Storage Container using ADF Pipeline and push the data into Fabric Data Lake house.

 

I followed the link How to ingest data into Fabric using the Azure Data Factory Copy activity - Microsoft Fabric | Micro... to do it. Looks like there is some problem in authentication. 

 

ThiyagarajanLG_0-1713776225851.png

 

Hi @ThiyagarajanLG 
Thanks for providing the details.

The error you’re encountering is related to Azure Active Directory authentication. Here’s a breakdown of the issue:

Solutions:

  1. Verify Application Registration:

    • Access the Azure portal and sign in as a global administrator for the "DEVRT" directory.
    • Navigate to Azure Active Directory>App registrations.
    • Search for the application ID "86cdd993-a314-4221-8e6b-bb9a19e5ebc9".
    • If the application isn't found, you'll need to register it in the "DEVRT" directory.
  2. Check Consent (if applicable):

    • If the application is found, ensure it's either a public application (doesn't require user consent) or has been consented to by a user with the necessary permissions in the "DEVRT" directory.
  3. Confirm Tenant Name:

    • Double-check the Azure AD tenant name you're using for authentication. It should be "DEVRT" in this case.

Please refer to these links for more help:

https://sendlayer.com/docs/error-aadsts700016-application-with-identifier-not-found-in-the-directory...

https://learn.microsoft.com/en-us/answers/questions/692461/message-aadsts700016-application-with-ide...

https://stackoverflow.com/questions/57324634/aadsts700016-application-with-identifier-some-id-was-no...

Hope this helps. Please let me know if you have any further issues.

Thank you for the explanation.

 

Do you suggest any other method to transfer the data from ADF to Fabric lake house ? (Using ADF PipleLIne Copy data Activity or any other way ) ??

 

Please let me know.

 

Hi @ThiyagarajanLG 
You can follow the below simple steps:

I have a CSV file named listings in the ADLS Gen2 storage container.

vnikhilanmsft_0-1713779263983.png


I have created a pipeline named Pipeline3 in Fabric workspace. You can use a Copy Assistant.

vnikhilanmsft_1-1713779350559.png

 

vnikhilanmsft_2-1713779407562.png


Create a new connection.

vnikhilanmsft_3-1713779549077.png

You can find the name and key in Access Keys. I have used Account Key as the authentication kind.

vnikhilanmsft_4-1713779603429.png

 

After the connection is successful select the required file.

vnikhilanmsft_5-1713779687803.png


Select the destination as Lakehouse.

vnikhilanmsft_6-1713779747893.png

 

 

vnikhilanmsft_7-1713779829623.png

 

vnikhilanmsft_8-1713779842195.png

 

Click on save+run. The data gets loaded into the lakehouse table.

Hope this helps. Please let me know if you have any further questions.

Thank you again for your help.

 

I believe you are pulling the data from ADF using Fabric pipeline. But i would like to push the data from ADF to Fabric using ADF pipeline.

 

I have one more question, In ADF pipeline, if i use source as XML file in Copy Data Activity, In the preview data the XML format are converted to JSON format. Even if i mentioned source format as XML format.

How do i retain the XML format from Source to Destination in Copy Data ACtivity ??

 

Hi @ThiyagarajanLG 
For your first question, you can use Fabric pipelines as they have more convenient approach. Why do you want to use only ADF pipelines? Just asking to understand the scenario.

Thanks

Hi Nikhilan,

 

My requirement is convert a full xml file from a container and store the full xml file(retaining the xml format) in a single cell in data lake house table.

 

for example: If i have the xml file sample.xml as below

 

<?xml version="1.0"?> <catalog> <book id="bk101"> <author>Gambardella, Matthew</author> <title>XML Developer's Guide</title> </book> </catalog>

 

 

I have read the xml file and store it as below in data lake house. since ADF is more powerful than fabric, i thought of using ADF to Fabric Push mechanism.

RowIDUpdatedbyxmlContent

1

 

 

 

 

393

<?xml version="1.0"?>
<catalog>
   <book id="bk101">
      <author>Gambardella, Matthew</author>
      <title>XML Developer's Guide</title>
   </book>
</catalog>

Hi @ThiyagarajanLG 
Is the data getting converted to JSON in the destination table?
One possible workaround is you can use notebooks in Fabric for this scenario.
Please refer to this link: Solved: Spark XML does not work with pyspark - Microsoft Fabric Community
You can write the data in lakehouse table using Spark code in notebooks.

Hope this helps.

Hi @ThiyagarajanLG 
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

Hi @ThiyagarajanLG 
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

v-nikhilan-msft
Community Support
Community Support

Hi @ThiyagarajanLG 
Thanks for using Fabric Community.
Why do you want to connect Azure Copy Data to Lakehouse? Are you referring to Copy Data Activity here? Apologies I could not understand the query. Can you please provide me some screenshots, so that I can understand better.

Thanks

Helpful resources

Announcements
FabricCarousel_June2024

Fabric Monthly Update - June 2024

Check out the June 2024 Fabric update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.