Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
MRosk95
Frequent Visitor

Access to Resource Forbidden - Web Contents

Hi there,

 

Until today, I have been accessing a public website through Power Query and pulling content down. All of a sudden I am getting an "Access to Resource is Forbidden" error.

This is my connection code:

let
// Base URL
baseUrl = "https://www.tenders.act.gov.au",

// Query Parameters
Source = Web.Contents(
baseUrl,
[
RelativePath = "/tender/search",
Query = [
#"keywords" = "",
#"tenderCode" = "",
#"tenderTitle" = "",
#"tenderState" = "OPEN",
#"tenderType" = "",
#"prequalificationCategory" = "",
#"procurementCategory" = "",
#"categories[0].fullTitle" = "43000000 Information Technology Broadcasting and Telecommunications",
#"categories[0].id" = "104789",
#"categories[1].fullTitle" = "Components for information technology or broadcasting or telecommunications",
#"categories[1].id" = "107731",
#"categories[2].fullTitle" = "81160000 Information Technology Service Delivery",
#"categories[2].id" = "145204",
#"issuingBusinessId" = "",
#"awardedSupplier.id" = "",
#"awardedSupplier.name" = "",
#"openThisWeek" = "false",
#"openingDateFrom" = "",
#"openingDateTo" = "",
#"ageRestriction" = "",
#"closeThisWeek" = "false",
#"closingDateFrom" = "",
#"closingDateTo" = "",
#"awardedThisWeek" = "false",
#"awardedDateFrom" = "",
#"awardedDateTo" = "",
#"orderBy" = "",
#"groupBy" = "NONE",
#"page" = "",
#"searchTitle" = ""
],
Headers = [
Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
#"User-Agent" = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
]
]
),

 

The website url is: 

https://www.tenders.act.gov.au/tender/search?keywords=&tenderCode=&tenderTitle=&tenderState=OPEN&ten... 

4 REPLIES 4
lbendlin
Super User
Super User

Web.Contents only gets you that far. You need to use Web.BrowserContents and then parse the HTML.

 

You can omit all the query parameters that are blank.

 

let
    Source = Web.BrowserContents("https://www.tenders.act.gov.au/tender/search?tenderState=OPEN&categories%5B0%5D.fullTitle=43000000+Information+Technology+Broadcasting+and+Telecommunications&categories%5B0%5D.id=104789&categories%5B0%5D.percentage=100&openThisWeek=false&closeThisWeek=false&awardedThisWeek=false&groupBy=NONE"),
    #"Extracted Table From Html" = Html.Table(Source, {{"Column1", ".tender-code-state"}, {"Column2", ".tender-code-state .tablesaw-cell-content"}, {"Column3", ".tablesaw-cell-content B"}, {"Column4", ".tender-code-state + *"}, {"Column5", ".strong"}, {"Column6", ".tender-date"}, {"Column7", ".opening_date"}, {"Column8", ".closing_date"}, {"Column9", "DIV:nth-child(2) .line-item-detail"}, {"Column10", "DIV:nth-child(3) .line-item-detail:nth-child(1)"}, {"Column11", ".tender-code-state .tablesaw-cell-label"}, {"Column12", ".tender-row-state"}, {"Column13", ".tender-date .tablesaw-cell-label"}, {"Column14", ".date_label_list:nth-child(1)"}, {"Column15", ".date_label_list:nth-child(5)"}, {"Column16", ".line-item-detail:nth-child(2)"}, {"Column17", ".line-item-detail:nth-child(3)"}}, [RowSelector="TBODY TR"])
in
    #"Extracted Table From Html"

Hey @lbendlin thank you so much for your response.

 

The issue that I continuall come across with Web.BrowserContents() is that I can't get it to refresh in Power Bi Service without installing a on-premises personal gateway? 

 

I could be wrong but why do I need an on-premises gateway to hit a public website? as such Web.Contents is the only option I can seem to use to achieve the outcome I want.

 

If you have a workaround or any ideas it would be a huge help.

 

Thanks,

What you are doing is basically screen scraping. This is frowned upon. The "correct" way would be to use the data API that your data source should provide.  That's what Web.Contents is designed for - for APIs, not for webpage scraping.

Thanks @lbendlin, no API is available and I don't believe the datasource provider is interested in providing one so kind of stuck here but thank you for your help.

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

July PBI25 Carousel

Power BI Monthly Update - July 2025

Check out the July 2025 Power BI update to learn about new features.

Top Kudoed Authors