Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
7 years ago

Filter out uneeded Data from extremely large SQL DB so I don't have to refresh it all.

Hi All,

 

I have a very large SQL data base, some tables (the parameters tables) are over 10 million rows deep (and growing) it is basically all the information from all different projects (which I have project ID's and project fileid's for) I want to build reports for only one of the many projects (which have several files) and don't want to have to reload all the data that I don't need.

 

I have very little SQL script experience, I beleive I can filter certain things in the SQL Statement section or with the use of Parameters.

 

Can anyone help me out.

Thanks

Need Help

6 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    7 years ago

    Hi Anonymous ,

     

    Do you mean you want to write a sql to select the part of data that you want?

     

    like :

     

    select * from your_table where project_id = your_project_id

     

    Aiolos Zhao

    • Anonymous's avatar
      Anonymous
      Not applicable
      7 years ago

      Thanks for the reply Aiolos,

       

      Yeah pretty much, but there will be a few different things i'll need to filter as not all tables have the Project id

      do I just keep repeating the script?

      select * from your_table where project_id = your_project_id

      select * from your_table where project_id = your_project_id

      select * from your_table where project_id = your_project_id

       

      etc?

       And should I do this when I initally load the sql db or as a parameter, so I can add a remove things if needed?

       

      Sorry a bit of a noob here

      • Anonymous's avatar
        Anonymous
        Not applicable
        7 years ago

        Seems like you have many tables,right?

        actually I don't know the final result that you want.

        So basically you can repeat to use the sql or you can do some "join" if there are some dimension tables.

         

        If you don't need other project data, you can use the sql when you load the data to power bi, then you don't need to filter project it in power bi.

         

        Well, it's better to give a small example, so I can know what exactly you want.

         

        Aiolos Zhao