Forum Discussion

dataengineerv2's avatar
dataengineerv2
Helper I
2 months ago
Solved

got below error when i tried migrate pipelines with power shell module

 

{
"state": "Failed",
"alerts": [
{
"severity": "UnsupportedResource",
"details": "Cannot upgrade Dataset 'ds_adls_ct_parquet' because its Type is 'Parquet'.",
"sourcePipelineName": "pl_ct_incremental_load"
}
],
"result": {},
"resolutions": []
}

for this why powershell cant migrate pipeliens with adls parquet ? i mean ui migration can do this quickly then whats is advantage of poweshell  module ? which option is best for migration and any drawbacks like this if ui migrations also have ?

data engineering
  • ati_puri's avatar
    ati_puri
    2 months ago

    Yes, UI does support and Power shell does not, however, Power Shell is recomended to use when we need to migrate in bulk or scripted migration or in soem CICD scenarios.

    As quoted my MSFT in their article-

    Power Shell migration makes sense when:

    1) You want to script it(CSV/JSON) once and run overnight, UI will be manual.

    2) The script can be parameterized for bulk loads.

    3) For the smaller number of files, use migration assistants for the unsupproted file formats and when we have large number of files, we can use the hybrid approach of using Power shell for supported formats to load in bulk and the unsupported(Parquet/Avro) ones by UI.

     

    Let me know if the above response gives you more insights on your query.

    Please give Kudos if you like the response and mark it as solution if it solves your query/problem.

     

    Thanks

    Ati Puri

5 Replies

    • dataengineerv2's avatar
      dataengineerv2
      Helper I
      2 months ago

      but its supported with ui migration right ? so my question does this powershell migration have any advantage or any other method which better than ui migration ?

       

      • ati_puri's avatar
        ati_puri
        Resolver III
        2 months ago

        Yes, UI does support and Power shell does not, however, Power Shell is recomended to use when we need to migrate in bulk or scripted migration or in soem CICD scenarios.

        As quoted my MSFT in their article-

        Power Shell migration makes sense when:

        1) You want to script it(CSV/JSON) once and run overnight, UI will be manual.

        2) The script can be parameterized for bulk loads.

        3) For the smaller number of files, use migration assistants for the unsupproted file formats and when we have large number of files, we can use the hybrid approach of using Power shell for supported formats to load in bulk and the unsupported(Parquet/Avro) ones by UI.

         

        Let me know if the above response gives you more insights on your query.

        Please give Kudos if you like the response and mark it as solution if it solves your query/problem.

         

        Thanks

        Ati Puri

  • v-abhinavmu's avatar
    v-abhinavmu
    Community Support
    2 months ago

    Hi dataengineerv2,

    Thank you for posting your query in the Microsoft Fabric Community Forum, and thanks to ati_puri for sharing valuable insights.

     

    Could you please confirm if your query has been resolved by the provided solutions? This would be helpful for other members who may encounter similar issues.

     

    Thank you for being part of the Microsoft Fabric Community.

  • v-abhinavmu's avatar
    v-abhinavmu
    Community Support
    1 month ago

    Hi dataengineerv2,

    May I check if this issue has been resolved? If not, Please feel free to contact us if you have any further questions.


    Thank you