Forum Discussion

Tilly01Hill's avatar
Tilly01Hill
Regular Visitor
9 months ago
Solved

Importing Data from a Website

I need to import data from a specific website every day. The only difference to the parameters from day to day is the date. How can I store the import parameters for the API, HTTP Header etc., so tha...
Help
  • v-karpurapud's avatar
    v-karpurapud
    8 months ago

    Hi Tilly01Hill 


    Thank you for reaching out to the Microsoft Fabric Community.

     

    To avoid rebuilding the API request every day, you can set up parameters in Power Query or Microsoft Fabric pipelines. In Power Query (Power BI Desktop or Fabric Dataflows Gen2), create parameters for elements like the base URL, headers, API key, and date. You can generate the date dynamically so it updates automatically with each refresh, such as by using:Date.ToText(Date.From(DateTime.LocalNow()), "yyyy-MM-dd"). Use these parameters in Web.Contents to build the request without needing manual changes. You can securely manage credentials and headers in Power BI Service under Data Source Credentials, and schedule daily refreshes after publishing.

     

    For Microsoft Fabric Data Factory, set pipeline parameters (including a dynamic date) and pass them to an HTTP connector or Copy Activity, then schedule the pipeline to run daily. This method aligns with Microsoft’s recommended practices for parameterized queries and scheduled refreshes. For further details, refer to Web.Contents and Fabric Data Factory pipelines.
     

    I hope this information is helpful. If you have any further questions, please let us know. we can assist you further.

     

    Regards,

    Microsoft Fabric Community Support Team.
     

v-karpurapud's avatar
v-karpurapud
Community Support
8 months ago

Hi Tilly01Hill 


Thank you for reaching out to the Microsoft Fabric Community.

 

To avoid rebuilding the API request every day, you can set up parameters in Power Query or Microsoft Fabric pipelines. In Power Query (Power BI Desktop or Fabric Dataflows Gen2), create parameters for elements like the base URL, headers, API key, and date. You can generate the date dynamically so it updates automatically with each refresh, such as by using:Date.ToText(Date.From(DateTime.LocalNow()), "yyyy-MM-dd"). Use these parameters in Web.Contents to build the request without needing manual changes. You can securely manage credentials and headers in Power BI Service under Data Source Credentials, and schedule daily refreshes after publishing.

 

For Microsoft Fabric Data Factory, set pipeline parameters (including a dynamic date) and pass them to an HTTP connector or Copy Activity, then schedule the pipeline to run daily. This method aligns with Microsoft’s recommended practices for parameterized queries and scheduled refreshes. For further details, refer to Web.Contents and Fabric Data Factory pipelines.
 

I hope this information is helpful. If you have any further questions, please let us know. we can assist you further.

 

Regards,

Microsoft Fabric Community Support Team.
 

Tilly01Hill's avatar
Tilly01Hill
Regular Visitor
8 months ago

Hi. Thanks for your very detailed response. 

I'm very new to Power BI Desktop - how do I find Power Query from my Power BI Homescreen?

 Thanks

  • v-karpurapud's avatar
    v-karpurapud
    Community Support
    8 months ago

    Hi Tilly01Hill 


    In Power BI Desktop, Power Query is accessed through the Transform Data option. From your Home screen, click Home → Transform Data, and this will open the Power Query Editor.
     

    Inside the editor, you can create parameters by selecting Manage Parameters → New Parameter, and then use them in your queries. This is where you’ll define elements like the base URL, headers, and dynamic date for your API call.

     

    Once you’ve set up the query, you can publish the report to the Power BI Service and configure scheduled refresh so it runs automatically every day.

    Here is the official Microsoft Learn documents that will help you

    Query Overview in Power BI Desktop - Power BI | Microsoft Learn
    Parameters - Power Query | Microsoft Learn

    Regards,

    Microsoft Fabric Community Support Team.