Forum Discussion

swetadutta7's avatar
swetadutta7
Regular Visitor
1 year ago
Solved

Download data from Web Page

Hello all, is there a step by step guide to be able to read reports from a company web page please. Gone through multiple documents/videos online but still haven't been able to fig it out.
CICD
  • v-lgarikapat's avatar
    v-lgarikapat
    1 year ago

    Hi swetadutta7 ,

     

    Thanks for the details .

     

    Based on the details you provided, here are a few workarounds to download the automated report and move it to Fabric using Dataflow.

     

    Headless browser automation

    Use a tool like Power Automate Desktop

      1. Log into the website with saved credentials.
      2. Navigate to the desired report.
      3. Trigger the download on a schedule (daily).
      4. Save the file (CSV, Excel, etc.) to a local folder or shared drive.

     SFTP (if the site has it)

    If you can convince the team to expose the report via SFTP, that’s a smoother integration. You can use Power Automate or Python scripts to grab the file daily.

     

    Move File into Microsoft Fabric

     

    Once you’ve got the file, here's how to get it into your Fabric Lakehouse via Dataflow Gen2:

    1. Go to Dataflows Gen2 in Microsoft Fabric.
    2. Create a new dataflow and select your source (Excel, CSV, etc.).
    3. Point to the file location (OneDrive, SharePoint, local folder via gateway, etc.).
    4. Use Power Query to transform it as needed.
    5. Choose your destination as a Lakehouse table.
    6. Set the refresh schedule to daily.
    7. If you go with headless browser automation, Power Automate Desktop is beginner-friendly and integrates well with Microsoft 365 tools.
    8. You can test your flow manually first to make sure the download and load steps work before scheduling it.

    Create your first Microsoft Fabric dataflow - Microsoft Fabric | Microsoft Learn

     

    If this post helped resolve your issue, please consider the Accepted Solution. This not only acknowledges the support provided but also helps other community members find relevant solutions more easily.

    We appreciate your engagement and thank you for being an active part of the community.

    Best regards,
    LakshmiNarayana

     

swetadutta7's avatar
swetadutta7
Regular Visitor
1 year ago

Hi, many thanks for the reply..here's few more details that might be useful to mention

- Yes, it is a company web page, requires login id and password to access

- This website has few inbuilt reports, so just trying to download one of them that I can then load it into a fabric lakehouse table using dataflow.

- Need this to be automated, a daily feed

- Unfortunately no API details and the support team take forever to reply back so if absolutely necessary then will go down that route. But, currently I am exploring direct web page access or SFTP file transfer process.

 

Might be worth mentioning that I have only started using Microsoft Fabric dataflow last week, so everything is fairly new to me 🙂 

v-lgarikapat's avatar
v-lgarikapat
Community Support
1 year ago

Hi swetadutta7 ,

 

Thanks for the details .

 

Based on the details you provided, here are a few workarounds to download the automated report and move it to Fabric using Dataflow.

 

Headless browser automation

Use a tool like Power Automate Desktop

    1. Log into the website with saved credentials.
    2. Navigate to the desired report.
    3. Trigger the download on a schedule (daily).
    4. Save the file (CSV, Excel, etc.) to a local folder or shared drive.

 SFTP (if the site has it)

If you can convince the team to expose the report via SFTP, that’s a smoother integration. You can use Power Automate or Python scripts to grab the file daily.

 

Move File into Microsoft Fabric

 

Once you’ve got the file, here's how to get it into your Fabric Lakehouse via Dataflow Gen2:

  1. Go to Dataflows Gen2 in Microsoft Fabric.
  2. Create a new dataflow and select your source (Excel, CSV, etc.).
  3. Point to the file location (OneDrive, SharePoint, local folder via gateway, etc.).
  4. Use Power Query to transform it as needed.
  5. Choose your destination as a Lakehouse table.
  6. Set the refresh schedule to daily.
  7. If you go with headless browser automation, Power Automate Desktop is beginner-friendly and integrates well with Microsoft 365 tools.
  8. You can test your flow manually first to make sure the download and load steps work before scheduling it.

Create your first Microsoft Fabric dataflow - Microsoft Fabric | Microsoft Learn

 

If this post helped resolve your issue, please consider the Accepted Solution. This not only acknowledges the support provided but also helps other community members find relevant solutions more easily.

We appreciate your engagement and thank you for being an active part of the community.

Best regards,
LakshmiNarayana

 

  • v-lgarikapat's avatar
    v-lgarikapat
    Community Support
    1 year ago

    Hi swetadutta7 ,

     

    I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please "Accept as Solution"

    We appreciate your engagement and thank you for being an active part of the community.

    Best regards,
    LakshmiNarayana    .

    • v-lgarikapat's avatar
      v-lgarikapat
      Community Support
      1 year ago

      Hi swetadutta7 ,

       

      If your question has been answered, kindly mark the appropriate response as the Accepted Solution. This small step goes a long way in helping others with similar issues.

      We appreciate your collaboration and support!

      Best regards,
      LakshmiNarayana