Forum Discussion
Dynamic date data retreival
Hello all,
Im trying to alter the SQL query retreiving all of my data to only retreive Today - 4 days and everything after. This is because im using this specific dataflow to refresh without hitting our SQL server too hard.
the SQL querry is LONG and has countless if statements because powerquery. i wouldnt really know where to insert my statement.
Can anyone help me with this specific instance?
Context: This query is used in my source table of my dataflow. This dataflow only needs to show very recent data which we want to have "live". I know its not actually going to be live, but thats not the point, we need it refreshed at specific points of the day so my collegues have more to work off. a few days of data will reduce the time allot in compared to the 1400 days its trying to load now, which takes 25 minutes.
Sql.Database(#"Database Server", #"Database Name", [Query="#(lf)#(lf)select#(lf)#(lf)Leg.""Legnr"", #(lf)Leg.""SalesOrder"",#(lf)ROW_NUMBER() OVER (PARTITION BY Leg.SalesOrder ORDER BY Legnr) AS SalesOrderLine,#(lf)Leg.""LegStatus"", #(lf)Leg.""Planned"",#(lf)Leg.""SalesInvoice"", #(lf)isnull(Leg.EDIProvider,'987654321') as EDIProvider,#(lf)Leg.""Distance"", #(lf)Leg.""OrderLeg"", #(lf)Leg.""PlanGroup"" as 'FK PlanGroup', #(lf)Leg.""FinDate"", #(lf)Leg.""TotalAmount"", #(lf)Leg.""Costs"", #(lf)Leg.""CMRReceived"",#(lf)Leg.[EmailSalesInvoice],#(lf)Leg.[Reference],#(lf)Leg.""CreationUser"",#(lf)Leg.[Cargo],#(lf)--Leg.BeginActivity,#(lf)--leg.EndActivity,#(lf)isnull(B.""Address"",(b.Activitynr + 900000000)) as BeginAddress, #(lf)B.""Date"" as BeginDate, #(lf)B.""DateTill"" as BeginDateTill, #(lf)B.""Time"" as BeginTime, #(lf)B.""TimeTill"" as BeginTimeTill,#(lf)B.""ResourceCombination"" as BeginResourceCombination, #(lf)B.""TripID"" as BeginTripID, #(lf)B.""StartDate"" as BeginStartDate, #(lf)B.""StartTime"" as BeginStartTime, #(lf)B.""EndDate"" as BeginEndDate, #(lf)B.""EndTime"" as BeginEndTime, #(lf)B.""ActivityDuration"" as BeginActivityDuration,#(lf)isnull(E.""Address"",(e.activitynr + 800000000)) as EndAddress, #(lf)E.""Date"" as EndDate, #(lf)E.""DateTill"" as EndDateTill, #(lf)E.""Time"" as EndTime, #(lf)E.""TimeTill"" as EndTimeTill, #(lf)E.""ResourceCombination"" as EndResourceCombination, #(lf)E.""TripID"" as EndTripID, #(lf)E.""StartDate"" as EndStartDate, #(lf)E.""StartTime"" as EndStartTime, #(lf)E.""EndDate"" as EndEndDate, #(lf)E.""EndTime"" as EndEndTime, #(lf)E.""ActivityDuration"" as EndActivityDuration,#(lf)leg.CreationDate,#(lf)Sum(Cargo.""Weight"") as Weight, #(lf)Sum(Cargo.""LoadingMeter"") as LoadingMeter, #(lf)Sum(Cargo.""PalletPlaces"") as PalletPlaces,#(lf)CASE WHEN legnr in (SELECT DISTINCT [leg] AS [pl_orders]#(lf)#(tab)#(tab)#(tab)#(tab)#(tab)FROM SO_Cargo #(lf)#(tab)#(tab)#(tab)#(tab)#(tab)WHERE Unit = 78) THEN 'pl' ELSE NULL END AS [pl order] -- unit 78 = pl#(lf)from SO_Leg as Leg#(lf)left join SO_Activity as B on leg.BeginActivity = B.Activitynr#(lf)left join SO_Activity as E on leg.EndActivity = E.Activitynr#(lf)left join so_cargo as cargo on leg.[legnr] = cargo.Leg#(lf)where FinDate >= DATEADD(year,-3,convert (date,DATEADD(YEAR, DATEDIFF(YEAR,0,GETDATE()) + 1, 0))) #(lf)#(lf)group by #(lf)Leg.""Legnr"", #(lf)Leg.""SalesOrder"",#(lf)Leg.""LegType"", #(lf)Leg.""LegStatus"", #(lf)Leg.""Planned"",#(lf)Leg.""SalesInvoice"", #(lf)Leg.EDIProvider,#(lf)Leg.""Distance"", #(lf)Leg.""OrderLeg"", #(lf)Leg.""PlanGroup"", #(lf)Leg.""FinDate"", #(lf)Leg.""TotalAmount"", #(lf)Leg.""Costs"", #(lf)Leg.""CMRReceived"",#(lf)Leg.[EmailSalesInvoice],#(lf)Leg.[Reference],#(lf)Leg.""CreationUser"",#(lf)Leg.Cargo,#(lf)--Leg.BeginActivity,#(lf)--Leg.EndActivity,#(lf)leg.CreationDate,#(lf)B.Activitynr,#(lf)B.""ActivityKind"", #(lf)B.""Address"", #(lf)B.""Date"", #(lf)B.""DateTill"", #(lf)B.""Time"", #(lf)B.""TimeTill"", #(lf)B.""ResourceCombination"", #(lf)B.""TripID"", #(lf)B.""StartDate"", #(lf)B.""StartTime"", #(lf)B.""EndDate"", #(lf)B.""EndTime"", #(lf)B.""ActivityDuration"",#(lf)E.""ActivityKind"", #(lf)E.""Address"", #(lf)E.""Date"", #(lf)E.""DateTill"", #(lf)E.""Time"", #(lf)E.""TimeTill"",#(lf)E.""ResourceCombination"", #(lf)E.""TripID"", #(lf)E.""StartDate"", #(lf)E.""StartTime"", #(lf)E.""EndDate"", #(lf)E.""EndTime"", #(lf)E.""ActivityDuration"",#(lf)E.Activitynr#(lf)#(lf)#(lf)#(lf)#(lf)"])
Here is your actual SQL statement. You'll need to head over to a SQL forum to get someone to help. Someone here might be able to assist, but a SQL forum as a bunch of people that can assist.
SELECT Leg."legnr", Leg."salesorder", Row_number() OVER ( partition BY Leg.salesorder ORDER BY legnr) AS SalesOrderLine, Leg."legstatus", Leg."planned", Leg."salesinvoice", Isnull(Leg.ediprovider, '987654321') AS EDIProvider, Leg."distance", Leg."orderleg", Leg."plangroup" AS 'FK PlanGroup', Leg."findate", Leg."totalamount", Leg."costs", Leg."cmrreceived", Leg.[emailsalesinvoice], Leg.[reference], Leg."creationuser", Leg.[cargo], --Leg.BeginActivity, --leg.EndActivity, Isnull(B."address", ( b.activitynr + 900000000 )) AS BeginAddress, B."date" AS BeginDate, B."datetill" AS BeginDateTill, B."time" AS BeginTime, B."timetill" AS BeginTimeTill, B."resourcecombination" AS BeginResourceCombination, B."tripid" AS BeginTripID, B."startdate" AS BeginStartDate, B."starttime" AS BeginStartTime, B."enddate" AS BeginEndDate, B."endtime" AS BeginEndTime, B."activityduration" AS BeginActivityDuration, Isnull(E."address", ( e.activitynr + 800000000 )) AS EndAddress, E."date" AS EndDate, E."datetill" AS EndDateTill, E."time" AS EndTime, E."timetill" AS EndTimeTill, E."resourcecombination" AS EndResourceCombination, E."tripid" AS EndTripID, E."startdate" AS EndStartDate, E."starttime" AS EndStartTime, E."enddate" AS EndEndDate, E."endtime" AS EndEndTime, E."activityduration" AS EndActivityDuration, leg.creationdate, Sum(Cargo."weight") AS Weight, Sum(Cargo."loadingmeter") AS LoadingMeter, Sum(Cargo."palletplaces") AS PalletPlaces, CASE WHEN legnr IN (SELECT DISTINCT [leg] AS [pl_orders] FROM so_cargo WHERE unit = 78) THEN 'pl' ELSE NULL END AS [pl order] -- unit 78 = pl FROM so_leg AS Leg LEFT JOIN so_activity AS B ON leg.beginactivity = B.activitynr LEFT JOIN so_activity AS E ON leg.endactivity = E.activitynr LEFT JOIN so_cargo AS cargo ON leg.[legnr] = cargo.leg WHERE findate >= Dateadd(year, -3, CONVERT (DATE, Dateadd(year, Datediff(year, 0, Getdate()) + 1, 0))) GROUP BY Leg."legnr", Leg."salesorder", Leg."legtype", Leg."legstatus", Leg."planned", Leg."salesinvoice", Leg.ediprovider, Leg."distance", Leg."orderleg", Leg."plangroup", Leg."findate", Leg."totalamount", Leg."costs", Leg."cmrreceived", Leg.[emailsalesinvoice], Leg.[reference], Leg."creationuser", Leg.cargo, --Leg.BeginActivity, --Leg.EndActivity, leg.creationdate, B.activitynr, B."activitykind", B."address", B."date", B."datetill", B."time", B."timetill", B."resourcecombination", B."tripid", B."startdate", B."starttime", B."enddate", B."endtime", B."activityduration", E."activitykind", E."address", E."date", E."datetill", E."time", E."timetill", E."resourcecombination", E."tripid", E."startdate", E."starttime", E."enddate", E."endtime", E."activityduration", E.activitynrOr, post some sample data and tell us what you are trying to do and we can start from scratch with Power Query.
You should rarely/never use that advanced SQL dialog box. Either create a View on the SQL server, or do it in Power Query. Doing it via an advanced SQL statement can cause permissions issue and prevents simple edits. I could adjust an M statement to only get the last 4 days of info in 2-3 lines that would fold back to the server. Nothing after an advanced SQL statement entry will allow folding.
3 Replies
- Greg_DecklerCommunity Champion
Anonymous - It looks like you are using a SQL Statement in your query. This is done in the Advanced portion of the Source step. I think that you would want to paste this SQL code instead and modify it to only get the last 4 days.
- AnonymousNot applicable
Greg_Deckler wrote:Anonymous - It looks like you are using a SQL Statement in your query. This is done in the Advanced portion of the Source step. I think that you would want to paste this SQL code instead and modify it to only get the last 4 days.
This was done by a previous engineer, ive been looking for a while now, is there a way to get this query up again?
- edhansCommunity Champion
Here is your actual SQL statement. You'll need to head over to a SQL forum to get someone to help. Someone here might be able to assist, but a SQL forum as a bunch of people that can assist.
SELECT Leg."legnr", Leg."salesorder", Row_number() OVER ( partition BY Leg.salesorder ORDER BY legnr) AS SalesOrderLine, Leg."legstatus", Leg."planned", Leg."salesinvoice", Isnull(Leg.ediprovider, '987654321') AS EDIProvider, Leg."distance", Leg."orderleg", Leg."plangroup" AS 'FK PlanGroup', Leg."findate", Leg."totalamount", Leg."costs", Leg."cmrreceived", Leg.[emailsalesinvoice], Leg.[reference], Leg."creationuser", Leg.[cargo], --Leg.BeginActivity, --leg.EndActivity, Isnull(B."address", ( b.activitynr + 900000000 )) AS BeginAddress, B."date" AS BeginDate, B."datetill" AS BeginDateTill, B."time" AS BeginTime, B."timetill" AS BeginTimeTill, B."resourcecombination" AS BeginResourceCombination, B."tripid" AS BeginTripID, B."startdate" AS BeginStartDate, B."starttime" AS BeginStartTime, B."enddate" AS BeginEndDate, B."endtime" AS BeginEndTime, B."activityduration" AS BeginActivityDuration, Isnull(E."address", ( e.activitynr + 800000000 )) AS EndAddress, E."date" AS EndDate, E."datetill" AS EndDateTill, E."time" AS EndTime, E."timetill" AS EndTimeTill, E."resourcecombination" AS EndResourceCombination, E."tripid" AS EndTripID, E."startdate" AS EndStartDate, E."starttime" AS EndStartTime, E."enddate" AS EndEndDate, E."endtime" AS EndEndTime, E."activityduration" AS EndActivityDuration, leg.creationdate, Sum(Cargo."weight") AS Weight, Sum(Cargo."loadingmeter") AS LoadingMeter, Sum(Cargo."palletplaces") AS PalletPlaces, CASE WHEN legnr IN (SELECT DISTINCT [leg] AS [pl_orders] FROM so_cargo WHERE unit = 78) THEN 'pl' ELSE NULL END AS [pl order] -- unit 78 = pl FROM so_leg AS Leg LEFT JOIN so_activity AS B ON leg.beginactivity = B.activitynr LEFT JOIN so_activity AS E ON leg.endactivity = E.activitynr LEFT JOIN so_cargo AS cargo ON leg.[legnr] = cargo.leg WHERE findate >= Dateadd(year, -3, CONVERT (DATE, Dateadd(year, Datediff(year, 0, Getdate()) + 1, 0))) GROUP BY Leg."legnr", Leg."salesorder", Leg."legtype", Leg."legstatus", Leg."planned", Leg."salesinvoice", Leg.ediprovider, Leg."distance", Leg."orderleg", Leg."plangroup", Leg."findate", Leg."totalamount", Leg."costs", Leg."cmrreceived", Leg.[emailsalesinvoice], Leg.[reference], Leg."creationuser", Leg.cargo, --Leg.BeginActivity, --Leg.EndActivity, leg.creationdate, B.activitynr, B."activitykind", B."address", B."date", B."datetill", B."time", B."timetill", B."resourcecombination", B."tripid", B."startdate", B."starttime", B."enddate", B."endtime", B."activityduration", E."activitykind", E."address", E."date", E."datetill", E."time", E."timetill", E."resourcecombination", E."tripid", E."startdate", E."starttime", E."enddate", E."endtime", E."activityduration", E.activitynrOr, post some sample data and tell us what you are trying to do and we can start from scratch with Power Query.
You should rarely/never use that advanced SQL dialog box. Either create a View on the SQL server, or do it in Power Query. Doing it via an advanced SQL statement can cause permissions issue and prevents simple edits. I could adjust an M statement to only get the last 4 days of info in 2-3 lines that would fold back to the server. Nothing after an advanced SQL statement entry will allow folding.