topic Re: consuming a variable in a copy data job? in Pipelines

consuming a variable in a copy data job?

amien — Wed, 30 Oct 2024 16:10:27 GMT

In a pipeline i run a notebook. In the end of this notebook i do a mssparkutils.notebook.exit

After the notebook is done, i have added a 'Set variable' with this setting : @activity('Notebook').output.result.exitvalue

This all works fine.

Now in my next step, i want to do a Copy data, using a query AND using the above mentioned variable. This part doesn't work.

I have this one in my query:

SELECT Week, Value FROM TableName WHERE Week = @variables('Maxweek_')

Is this not correct?

KNP — Wed, 30 Oct 2024 17:41:47 GMT

I think the syntax should be something like...

@concat( 'select week, value from TableName where week = ' , variables('Maxweek_') )

depending on how you've written your variable, you may need to enclose it in quotes, and the escaping of quotes can be confusing...

@concat( 'select week, value from TableName where week = ' , '''' , variables('Maxweek_') , '''' )

So that '''' is surrounding the concat item in '' and then another '' one to escape the other.

amien — Wed, 30 Oct 2024 18:31:36 GMT

it does work (first option) .. only thing is that it will remove the 'edit' button 🙂

KNP — Wed, 30 Oct 2024 18:57:55 GMT

Yeah, you need to use the add dynamic content option for this.

Anonymous — Fri, 01 Nov 2024 08:18:03 GMT

Did KNP reply solve your problem? If so, please mark it as the correct solution, and point out if the problem persists.

Best regards,

Adamk Kong