Append queries everyday automatically

[Manoharch]

Hello, I have an use case where the transaction tables of same structure from different dates will be loaded everyday in synapse. So I have to load the table and append the new table with existing table. Can it be performed automatically? We don't have access to source to change that.

[bhanu_gautam · Accepted solution]

@Manoharch , Use this m code

let
Source = Sql.Database("your_synapse_server", "your_database"),
NewTable = Source{[Schema="dbo",Item="NewTransactionTable"]}[Data],
ExistingTable = Source{[Schema="dbo",Item="ExistingTransactionTable"]}[Data],
AppendedTable = Table.Combine({ExistingTable, NewTable})
in
AppendedTable

[v-pgoloju]

Hi @Manoharch,

Just a gentle reminder — has your issue been resolved? If so, we’d be grateful if you could mark the solution that worked as Accepted Solution, or feel free to share your own if you found a different fix.

This not only closes the loop on your query but also helps others in the community solve similar issues faster.

Thank you for your time and feedback!

Best,

Prasanna Kumar

[v-pgoloju]

Hi @Manoharch,

We wanted to kindly check in to see if everything is working as expected after trying the suggested solution. If there’s anything else we can assist with, please don’t hesitate to ask.

If the issue is resolved, we’d appreciate it if you could mark the helpful reply as Accepted Solution — it helps others who might face a similar issue.

Warm regards,

Prasanna Kumar

[v-pgoloju]

Hi @Manoharch,

Just following up to see if the solution provided was helpful in resolving your issue. Please feel free to let us know if you need any further assistance.

If the response addressed your query, kindly mark it as Accepted Solution and click Yes if you found it helpful — this will benefit others in the community as well.

Best regards,

Prasanna Kumar

[v-pgoloju · Accepted solution]

Hi @Manoharch,

Thank you for reaching out to the Microsoft Fabric Forum Community.

Also, thanks to @bhanu_gautam  for the prompt and helpful response.

approach to append tables dynamically with the help of Synapse Pipelines, stored procedures and metadata-driven logic to automate the process.

steps to automate: 

Create a Master Table, Use a Metadata Table to Track Loaded Tables, Create a Stored Procedure to Load New Tables then Trigger This Procedure Using Synapse Pipeline.

Alternative approach with M code

let
Source = Sql.Database("your-synapse-server", "your-db"),
TableNames = Table.SelectRows(Source{[Schema="dbo"]}[Data], each Text.StartsWith([Name], "Transactions_")),
CombinedData = Table.Combine(List.Transform(TableNames[Name], each Sql.Database("your-synapse-server", "your-db"){[Schema="dbo", Item=_]}[Data]))
in
CombinedData

**Large number of tables may slow refresh — consider aggregating in Synapse or using incremental refresh.

If you find this response helpful, please consider marking it as the accepted solution and giving it a thumbs-up to support others in the community.

Thank you & Regards,
Prasanna kumar

[bhanu_gautam · Accepted solution]

@Manoharch , Use this m code

let
Source = Sql.Database("your_synapse_server", "your_database"),
NewTable = Source{[Schema="dbo",Item="NewTransactionTable"]}[Data],
ExistingTable = Source{[Schema="dbo",Item="ExistingTransactionTable"]}[Data],
AppendedTable = Table.Combine({ExistingTable, NewTable})
in
AppendedTable