Filter out uneeded Data from extremely large SQL DB so I don't have to refresh it all.

[Anonymous]

Hi All,

I have a very large SQL data base, some tables (the parameters tables) are over 10 million rows deep (and growing) it is basically all the information from all different projects (which I have project ID's and project fileid's for) I want to build reports for only one of the many projects (which have several files) and don't want to have to reload all the data that I don't need.

I have very little SQL script experience, I beleive I can filter certain things in the SQL Statement section or with the use of Parameters.

Can anyone help me out.

Thanks

[Anonymous]

Hi @Anonymous ,

Do you mean you want to write a sql to select the part of data that you want?

like :

select * from your_table where project_id = your_project_id

Aiolos Zhao

[Anonymous]

Thanks for the reply Aiolos,

Yeah pretty much, but there will be a few different things i'll need to filter as not all tables have the Project id

do I just keep repeating the script?

select * from your_table where project_id = your_project_id

select * from your_table where project_id = your_project_id

select * from your_table where project_id = your_project_id

etc?

 And should I do this when I initally load the sql db or as a parameter, so I can add a remove things if needed?

Sorry a bit of a noob here

[Anonymous]

Seems like you have many tables,right?

actually I don't know the final result that you want.

So basically you can repeat to use the sql or you can do some "join" if there are some dimension tables.

If you don't need other project data, you can use the sql when you load the data to power bi, then you don't need to filter project it in power bi.

Well, it's better to give a small example, so I can know what exactly you want.

Aiolos Zhao

[Anonymous]

Hi Aiolos

This is the table and I want to use Project 15

Would I write -  select * from projects where projectId = 15

I think this will be the only filter I need.

Thanks

[Anonymous]

yes, right.

If you also want other tables only include projectid = 15, I think you can use table relationship to limit other tables.

Aiolos Zhao

[Anonymous]

So I tried "select * from projects where projectId = 15" but I only get the onl floder that split into a single Query.

I'm not entirely sure how to get the rest of the folders based off the relationships

These are all the tables I need to filter out all the projects except 1