Forum Discussion
SOQL in Dataflow gen2
Hi community,
Is it possible to write an SOQL to extract the data from SalesForce using a Dataflow gen2?
I am aware that it is possible to do it with a Copy Data activity in a pipeline but I was wondering if it is possible to do it with a Dataflow.
My objective is to get the data in the fastest way possible. Spark notebooks are not an option because it takes at least 5 minutes to create the sparksession.
Hi lsabetta,
For large volumes of data, a pipeline will be more efficient than PowerQuery will.
That being said, PowerQuery can connect to Salesforce:
Power Query Salesforce Objects connector - Power Query | Microsoft LearnIf you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
2 Replies
Hi lsabetta,
For large volumes of data, a pipeline will be more efficient than PowerQuery will.
That being said, PowerQuery can connect to Salesforce:
Power Query Salesforce Objects connector - Power Query | Microsoft LearnIf you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
Power Query does not allow you to write custom SOQL. It does include some mild query folding (you can check from the query diagnostics) but that is strictly limited to the object, and does not include child queries.
Apply object filters, check the folding, and then accept that you have to ingest most of the object before throwing away what you don't need.
You will not be able to recreate a Report Type logic with this.
NOTE: While both SFDC and Power Query are case sensitive, Power BI IS NOT. Always use CaseSafeIDs.
