Forum Discussion

Element115's avatar
Element115
Icon for Memorable Member rankMemorable Member
2 years ago

BUG::New table in data warehouse not detected

STEPS 0__create a new data warehouse 1__create a new table in schema [dbo] (with column data types matching or compatible with on-prem SQL Server 2017 source) 2__wait, then refresh until table app...
Dataflows
frlunett's avatar
frlunett
Icon for Microsoft Employee rankMicrosoft Employee
2 years ago

Hello Element115,

 

would you be able to share the SQL statement you used to create the table? What is the name of the table in the Warehouse?

 

Does it help to refresh the navigator view by pressing the refresh button next to "Display options" on the left pane in the above UI?

 

Thanks,

Francesco.

  • Element115's avatar
    Element115
    Icon for Memorable Member rankMemorable Member
    2 years ago

    Sure thing. Here's the DDL:

     

    CREATE TABLE [dbo].[TS60]
(
    ID_TS       varchar(32) NOT NULL
,   NSF         int 
,   _DATE       date
,   _TIME       time(0)
,   INTENSITY   numeric(10, 6)
,   OCCUPANCY   numeric(8, 6)
,   VELOCITY    numeric(9, 6)
,   VOLUME      int
)
GO

     

    Not sure what you are referring to by 'Display options.' If you are talking about refreshing the table, yes, and the table shows up when we open the warehouse in the Data Engineering persona.   I also did a regular browser refresh. None of which matter.

     

    I am currently running a workaround which consists of creating 2 dataflows Gen2.  The first ingests from the on-prem DB but without a destination inside Fabric.  The second DF ingests from the previous DF and this time defines a Fabric data warehouse as destination.  So far, no errors but the refresh has been running for more than 3 hours now.  The total dataset size, which comes from a SQL view, is about 3 million records. It's extremely slow apparently.