Forum Discussion
rje_gaiaherbs
2 years agoFrequent Visitor
Pipeline Data Copy Error 2200
I am attempting to copy data from an on prem oracle table to Fabric (tried both wh and lh). This is requiring the use of Azure Data Lake Storage Gen2 for staging. When running the action, the follow...
FabianSchut
1 year agoSolution Sage
This has been a struggle for me for a few weeks too. I've created my own workaround for this problem. It took some time to finetune, but it works for me. Hope it helps you too.
First, I created a lookup that retrieves all the column names from the table in Oracle. The table and schema are pipeline parameters in my case, but can be hardcoded or retrieved in another way too.
The Query for the lookup is (I use XMLAGG instead of other options to mitigate the problem that the returned string was too long) :
@concat('SELECT RTRIM(XMLAGG(XMLELEMENT(E,CASE WHEN DATA_TYPE = ''NUMBER'' AND (CAST (DATA_SCALE AS NUMBER(5,0)) >= CAST (DATA_PRECISION AS NUMBER(5,0)) OR DATA_SCALE IS NULL OR DATA_PRECISION IS NULL) THEN ''CAST('' || COLUMN_NAME || '' AS NUMBER(38,18)) AS '' || COLUMN_NAME ELSE COLUMN_NAME END,'', '').EXTRACT(''//text()'') ORDER BY COLUMN_NAME).GetClobVal(),'', '') AS query_string
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = ''', pipeline().parameters.table_name, '''
AND OWNER = ''', pipeline().parameters.schema, '''')Then I use these retrieved columns in my copy data:
And the query for the source is:
@concat('SELECT ',activity('lookup_columns_from_table').output.firstRow.QUERY_STRING, ' FROM ', pipeline().parameters.schema, '.', pipeline().parameters.table_name)
- jenyfer_duran1 year agoNew Member
Thank you! Works for me too