topic Re: Copy Job, Database Table Listing Problem in Data Engineering

Copy Job, Database Table Listing Problem

verianalizi — Tue, 31 Dec 2024 08:41:02 GMT

Hello everyone,

I'm trying to copy some tables from SAP R/3 (MS SQL), but when I attempt this, the system tries to list all tables. Due to the large number of tables, it's impossible to display the entire list.

Has anyone encountered a similar issue or have suggestions on how to handle this?

Thanks in advance!

Re: Copy Job, Database Table Listing Problem

v-prasare — Fri, 03 Jan 2025 05:27:01 GMT

Hi @verianalizi,

Thanks for reachoing Microsoft Fabric community Support.

Here are a few potential solutions you could try to handle this issue:

Limit the Table List: instead of attempting to display all tables at once, limit the number of tables shown. You can filter or sort the list based on specific criteria (e.g., table names, schemas, or metadata).

Use SQL queries like:

SELECT TABLE_NAME

FROM INFORMATION_SCHEMA.TABLES

WHERE TABLE_NAME LIKE 'YourPattern%' -- or other filtering conditions

ORDER BY TABLE_NAME

Use Specific Table Names: If you're only interested in certain tables, directly reference the table names in your queries instead of trying to retrieve a list of all tables.

For example: SELECT * FROM YourTableName

Thanks,

Prashanth Are

MS Fabric community support.

Did I answer your question? Mark my post as a solution, this will help others!

If my response(s) assisted you in any way, don't forget to drop me a "Kudos"

Re: Copy Job, Database Table Listing Problem

verianalizi — Thu, 02 Jan 2025 08:40:37 GMT

Actually I want to try the new feature "Copy Job".

Re: Copy Job, Database Table Listing Problem

v-prasare — Fri, 03 Jan 2025 10:33:17 GMT

Use a Query to Limit the Table List

Instead of listing all tables, use a custom query to narrow down the list of tables you need. The "Copy Job" feature should allow you to specify a source query.

Steps:

In the Copy Job wizard, when selecting the source, choose Query Mode instead of browsing the table list.
You can also dynamically filter tables based on table names or schemas:

SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE 'Z%' -- Example to filter custom SAP tables

Thanks,

Prashanth Are

MS Fabric community support.

Did I answer your question? Mark my post as a solution, this will help others!

If my response(s) assisted you in any way, don't forget to drop me a "Kudos"

Re: Copy Job, Database Table Listing Problem

v-prasare — Wed, 08 Jan 2025 07:28:01 GMT

@verianalizi , As we haven’t heard back from you, we wanted to kindly follow up to check if the solution I have provided helps you? or let us know if you need any further assistance here?

Your feedback is important to us, Looking forward to your response.

Thanks,

Prashanth Are

MS Fabric community support.

Did I answer your question? Mark my post as a solution, this will help others!

If my response(s) assisted you in any way, don't forget to drop me a "Kudos"

Re: Copy Job, Database Table Listing Problem

v-prasare — Thu, 16 Jan 2025 10:40:12 GMT

Hi @verianalizi, Hope your doing well.

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution I have provided helps you?

Your feedback is important to us, Looking forward to your response.

Thanks,

Prashanth Are

MS Fabric community support.

Did I answer your question? Mark my post as a solution, this will help others!

If my response(s) assisted you in any way, don't forget to drop me a "Kudos"

Re: Copy Job, Database Table Listing Problem

v-prasare — Mon, 20 Jan 2025 10:40:25 GMT

Hi @verianalizi,

Hope your doing well.

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution I have provided helps you?

Your feedback is important to us, Looking forward to your response.

please find below similar solved threads in community:

Solved: Copy Job (preview) On-Premise to Fabric DW - Microsoft Fabric Community

Thanks,

Prashanth Are

MS Fabric community support.

Did I answer your question? Mark my post as a solution, this will help others!

If my response(s) assisted you in any way, don't forget to drop me a "Kudos"

Re: Copy Job, Database Table Listing Problem

verianalizi — Wed, 22 Jan 2025 13:23:00 GMT

Hello,

As you can see there is no such an option.

Re: Copy Job, Database Table Listing Problem

v-prasare — Mon, 27 Jan 2025 09:15:38 GMT

@verianalizi, my apologies here as above mentioned can be achived from data factory pipelines.

can you please confirm once you have given connection details and click next, it should list all the tables like show below.

as you mentioned issue with listing huge number of tables, this might be due to limitations that is mentioned in Fabric documentation here. PFB screenshot for same.

additionally, below copy job is in preview mode and i request you to please submit your feedback in documentation. your input is valuable and will help other users who might have similar questions. We appreciate your effort in sharing your thoughts

What is Copy job (preview) in Data Factory - Microsoft Fabric | Microsoft Learn

Thanks,

Prashanth Are

MS Fabric community support.

Did I answer your question? Mark my post as a solution, this will help others!

If my response(s) assisted you in any way, don't forget to drop me a "Kudos"