topic Copy activity UTC/local time zone conversion problem from On Prem Gateway in Pipelines

Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Wed, 22 May 2024 08:06:41 GMT

Hi,

i cant solve time zone issues, maybe on prem gateway should have some settings or what i should do?

1. Data source is On Prem SQL, all date/time fields are stored in UTC time;

2. Our local time zone is +2;

3. Using copy activity to Lakehouse, all date/time fields are saved these -2 hours. Example, if it is only date (2024-05-22T00:00:00), then in Lakehouse table i have 2024-05-21T:22:00:00, and everything with time is just -2hours back... Until now i was using dataflows gen2, this issue was not ocurring...

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

Anonymous — Wed, 22 May 2024 11:07:17 GMT

Hi matkvaid ,

Thanks for using Fabric Community.
Can you please share the screenshots of your pipeline configuration? Also can you confirm that you did not faced this issue with dataflow gen 2 but observing with copy activity?

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Wed, 22 May 2024 11:17:03 GMT

Hello,

is this enough? Preview is from on prem SQL source, so it has correct date/time (it does not have timezone in source). In Dataflows it was correct, i was testing them from start of Fabric trial and copy date to Warehouse. Now when On Prem gateway connection is possible in Pipelines, we started moving everything to Lakehouse with Pipelines. Attaching screenshot:

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Wed, 22 May 2024 11:18:26 GMT

Oh and i forgot to mention - in bottom of picture, is screenshot from destination lakehouse table

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

Anonymous — Wed, 22 May 2024 11:31:39 GMT

Hi matkvaid ,

We have seen this issue reported by other customers too. It looks like some bug.

Similar Issue - Fabric pipeline copy activity converts time to UTC - Microsoft Fabric Community

I request you to please open a support ticket and have our support team take a closer look at it - Link

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Hope this helps. Please let us know if you have any other queries.

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Wed, 22 May 2024 11:42:34 GMT

Support request ID is 2405220050002839 - it is strange how You already know about the issue, but seems nothing is done about that... 🙂

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

Anonymous — Thu, 23 May 2024 07:27:23 GMT

Hi matkvaid ,

Thanks for sharing the support ticket.

Please allow some time, so team can check and provide a resolution.

In case if you got a resolution please do share with the community as it can be helpful to others .

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Fri, 31 May 2024 07:37:58 GMT

Any news from Microsoft on this issue? I was told by support team, that it is a known bug and fix should have been released yesterday?

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

MarkBurgess — Fri, 31 May 2024 09:02:26 GMT

I'm also eagerly awaiting a fix for this.

Anonymous is there an ETA for the fix? Is it worth me raising my own ticket if it is already a known issue?

Thanks

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

Anonymous — Fri, 31 May 2024 09:07:10 GMT

Hi MarkBurgess , matkvaid

Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this .
We will update you once we hear back from them.

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Mon, 03 Jun 2024 05:13:20 GMT

Hello,

Anonymous maybe any updates on this? We cannot use pipelines, all date time fields are incorrect, all our reports now are wrong... This is a critical issue for business!

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

Anonymous — Mon, 03 Jun 2024 06:20:49 GMT

Hi matkvaid ,

The fix is in progress and estimated deployment time might be around mid june as per the team.
Appreciate your patience.

Thank you

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Mon, 03 Jun 2024 06:26:25 GMT

Anonymous in a response from support i got information that it should be end of May, now it is "around mid june", so it will be or will not be in june, maybe in july? I can say sincerely: my patience is over. I am glad that we are still on Fabric Trial, because if we would pay for capacity that has bug on bug, i would be VERY mad. Now i just think that it is not worth for our company to fully move to Power BI/Fabric from Qlik, because that one at least works without chaning everything completely every year and then not working. First i moved everything to Datamart, we bought Premium per user licences. Then it stopped working from nothing. Support told me - do not use it, it is preview feature that will be deprecated... Overall my experience for now was more negative than positive.

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

datafabricator — Tue, 04 Jun 2024 12:23:36 GMT

Just wanted to say that our team also just tore our hair over this. Found out here that it's a bug.

It's critical to say the least, since it affects incremental loads. Please prioritize it!

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Tue, 04 Jun 2024 12:43:56 GMT

I agree, it affects many things: incremental loads, data in reports... 🙂 It shifts datetime2 columns that have date only, and time is T00:00:00 to - 1 day 😄 I found posts that are more than a month old, and it was also commented that it is a bug. On premise data gateway support for pipelines was released end of March, i believe that it was noticed qute soon. Two months and no fix, no solutions for temporary workarounds - nothing. I understand that we are on Trial, so we can help develop and fix bugs for free, but the trial will end for me, and with current situation i cannot imagine paying for capacity... 😄

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Fri, 14 Jun 2024 06:22:53 GMT

Hello Anonymous i can see that yesterday new on premise data gateway version was released, was this bug fixed?

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Thu, 20 Jun 2024 05:28:04 GMT

Hello, Anonymous any news about this issue? It is already mid-june, still no news about the issue?????

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

matkvaid — Tue, 25 Jun 2024 10:34:07 GMT

Still no information about this?? How is this even possible, did Microsoft decided just to ignore it? Support also does not answer.

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

plochawala — Tue, 23 Jul 2024 17:51:29 GMT

When is this issue going to be resolved? I am also seeing simillar issue.

Re: Copy activity UTC/local time zone conversion problem from On Prem Gateway

Rk_1411 — Wed, 24 Jul 2024 03:38:29 GMT

Any update on this.. It seems to be a known bug with no solution.