Forum Discussion
Something went wrong, Dataflow publish failed
Hi, Im getting a strange and vague error when I publish my data flow gen 2.
All I get is "Something went wrong" and can't seem to find out anything else about where it is going wrong.
I can publish without error but then I get:
(Clicking See details just shows me the IDs of the flow and what time the error occurred.)
let
FetchPage = (url) =>
let
Source = Json.Document(Web.Contents(url)),
Data = Source[data],
NextPage = Source[meta][pagination][next],
More = Source[meta][pagination][more],
Result = if More then Data & @FetchPage(NextPage) else Data
in
Result,
StartURL = "https://api.wonde.com/v1.0/schools?page=1",
AllData = FetchPage(StartURL),
...and so on with some more data changes
I managed to solve this, I was changing the base URL of the Web.Contents which meant the flow didn't know what connector to use everytime it changed.
Fixed by using one Base URL and then using RelativePath and Query properly.
So I changed this
let FetchPage = (url) => let Source = Json.Document(Web.Contents(url)), Data = Source[data], NextPage = Source[meta][pagination][next], More = Source[meta][pagination][more], Result = if More then Data & @FetchPage(NextPage) else Data in Result, StartURL = "https://api.wonde.com/v1.0/schools?page=1", AllData = FetchPage(StartURL), ...and so on with some more data changesinto this:
FetchPage = (url) => let baseURL = "https://api.wonde.com/v1.0/", relativePath = "schools", queryList = Uri.Parts(url)[Query], response = Web.Contents(baseURL, [ RelativePath = relativePath, Query = queryList ]), Source = Json.Document(response), Data = Source[data], NextPage = Source[meta][pagination][next], More = Source[meta][pagination][more], Result = if More then Data & @FetchPage(NextPage) else Data in Result, StartURL = "https://api.wonde.com/v1.0/schools?page=1", AllData = FetchPage(StartURL),which means it always uses the same connector and is using the relative path correctly.
8 Replies
Hey!
I'd recommend that you raise a support ticket so an engineer can take a closer look at it.
I managed to solve this, I was changing the base URL of the Web.Contents which meant the flow didn't know what connector to use everytime it changed.
Fixed by using one Base URL and then using RelativePath and Query properly.
So I changed this
let FetchPage = (url) => let Source = Json.Document(Web.Contents(url)), Data = Source[data], NextPage = Source[meta][pagination][next], More = Source[meta][pagination][more], Result = if More then Data & @FetchPage(NextPage) else Data in Result, StartURL = "https://api.wonde.com/v1.0/schools?page=1", AllData = FetchPage(StartURL), ...and so on with some more data changesinto this:
FetchPage = (url) => let baseURL = "https://api.wonde.com/v1.0/", relativePath = "schools", queryList = Uri.Parts(url)[Query], response = Web.Contents(baseURL, [ RelativePath = relativePath, Query = queryList ]), Source = Json.Document(response), Data = Source[data], NextPage = Source[meta][pagination][next], More = Source[meta][pagination][more], Result = if More then Data & @FetchPage(NextPage) else Data in Result, StartURL = "https://api.wonde.com/v1.0/schools?page=1", AllData = FetchPage(StartURL),which means it always uses the same connector and is using the relative path correctly.
Hi danielmccluskey ,
Glad that your issue got resolved. Please continue using Fabric Community for help regarding your queries.
I encountered similar issue recently with Dataflow Gen2 in Microsoft Fabric. I mean nothing really fancy here, only selecting a .csv file uploaded in Lakehouse folder which I had previously created, then define a destination through Add Data Destination to very same Lakehouse with new table name though in different folder. "Dataflow Failed to Publish!" [sigh...]
Okay never mind... solved it! It seems Dataflow Gen2 didn't seem to accept any blank space after a comma... in csv. So I simply removed all blank spaces in original csv file, deleted the wrong formatted csv file version in Lakehouse, then re-uploaded new corrected csv version in same Lakehouse and it worked this time...just as stupid as that... [sigh...]
Interesting.
How did you find out that it was the reason?
Was there some error values in some of the data columns?
