juansgr_'s avatar
juansgr_
New Member
22 days ago
Status:
Abandoned

Expose Power Query as a Standalone Runtime and Command-Line Interface (CLI)

Summary

Power Query has evolved into one of Microsoft's most powerful data transformation technologies and is now used across Excel, Power BI, Fabric, Dataflows, Power Platform, and other products. However, Power Query can only be executed through a host application, despite the existence of a mature M language and execution engine. 

I would like Microsoft to expose the Power Query engine as a first-class standalone runtime and provide an officially supported command-line interface (CLI) and API.

The Problem

Today, Power Query transformations are often embedded inside:

  • Excel workbooks
  • Power BI Desktop files
  • Fabric Dataflows
  • Power Platform Dataflows

While this works well for interactive users, it creates challenges for enterprise-grade automation.

Many organizations would like to:

  • Schedule Power Query transformations without opening Excel.
  • Run Power Query from PowerShell scripts.
  • Integrate Power Query into CI/CD pipelines.
  • Execute transformations on servers without Office dependencies.
  • Reuse Power Query code across multiple solutions.
  • Treat Power Query as a reusable transformation layer rather than a workbook artifact.

Currently, Power Query feels like a language without an officially supported runtime, even though the engine already powers multiple Microsoft products.

2 Comments

  • One practical example from my own work: I use PowerShell to orchestrate a master data backup process where a list of SQL files is executed sequentially through hdbsql.exe, and the results are automatically exported to CSV files for backup and downstream consumption.

    Oversimplified example of an hdbsql.exe execution using server information stored in $HanaServer, a single hardcoded .sql file, and a single output file. In practice, this process can execute multiple SQL files and produce multiple .csv files recursively and sequentially when looped:

    PowerShell
    ↓
    & hdbsql.exe -n $HanaServer -I query1.sql -o example1.csv

    The key benefit is that SQL scripts are treated as first-class, reusable assets. PowerShell does not need to know the query logic itself. It simply invokes hdbsql.exe, passes the .sql file, and handles orchestration, logging, scheduling, and output management.

    I would like to see a similar capability for Power Query. Today, .pq files contain transformation logic but cannot be executed directly as standalone assets in the same way that .sql files can be executed through hdbsql.exe. An official powerquery.exe (or equivalent runtime) would allow organizations to maintain reusable .pq files and execute them from PowerShell, Task Scheduler, Azure Automation, or CI/CD pipelines.

    For example:

    PowerShell
    ↓
    & powerquery.exe -I etl1.pq -o example1.csv

    This runtime could leverage the same authentication mechanisms already supported by Power Query connectors, allowing secure access to sources such as SharePoint, SQL databases, SAP systems, APIs, and other enterprise data platforms.

    The particular use case that made me think about this is: I wanted to use a .pq file to connect directly to a read-only .xlsx file from SharePoint, retrieve a finance reporting schedule from a spreadsheet that currently has a non-table structure, perform all required transformations, and export a normalized dataset (without requiring Excel, Power BI Desktop, or any other host application) to be used as an input downstream in a similar orchestration/automation process as the one I designed for the Backup solution mentioned earlier. This would create the same clean separation of responsibilities that already exists with PowerShell and hdbsql.exe:

    • Power Query for data transformation
    • PowerShell for orchestration and automation
    • SQL for data consumption and reporting


    This approach would make Power Query a true first-class component of enterprise automation and ETL workflows, while preserving the existing user experience within Excel, Power BI, Fabric, and other Microsoft products.

  • miguel's avatar
    miguel
    Icon for Community Admin rankCommunity Admin
    Status changed:
    New
    to
    Abandoned

    This is currently possible via the Dataflow REST API using the endpoint below:
    Query Execution - Execute Query - REST API (Dataflow) | Microsoft Learn

    You can also use the Data Factory MCP to make things a bit more straightforward in wiring the commands, but is fundamentally our path towards the problem / scenario that you've mentioned as to how we enable you to have the full runtime available without the UI or the need for another product (like Excel or Power BI) to be running alongside it.

Recent ideas