Forum Discussion

Alex_Pierse's avatar
Alex_Pierse
Frequent Visitor
3 years ago
Solved

Getting data from multiple google sheets in one query

Hello community) Before me was the task of getting data from 10 Google tables that are in the same folder on Google Drive. Unfortunately, there is no technical possibility to deoat this through the ...
  • Alex_Pierse's avatar
    3 years ago

    Hello. The problem was solved a little differently. Below I throw off everything that I could find on the Internet. Connectors are not all adequate, and do not work as you want.
    The problem was solved through the Google API and a Python script. Below I throw off 2 links to Russian-language videos with a solution to the problem. But his script is a little tricky and a lot of superfluous,
    a little reworking, we get a good result

    Google Drive 
    Google Sheets 

    My example working state python script

    import pandas as pd
    import glob

    from pydrive.auth import GoogleAuth

    from pydrive.drive import GoogleDrive

    from oauth2client.service_account import ServiceAccountCredentials

    gauth = GoogleAuth()
    scope = ["https://www.googleapis.com/auth/drive"]

    gauth.credentials = ServiceAccountCredentials.from_json_keyfile_name(r"Let to the key file API Google", scope)
    drive = GoogleDrive(gauth)
    folder_id = ['ID Google Folder ']
    for files in folder_id:
    file_list = drive.ListFile({'q': "'{}' in parents and trashed=false".format(files)}).GetList()
    for file in file_list:
    test_id=file['id']
    test_title=file['title']
    mime='application/vnd.google-apps.spreadsheet'
    if mime in file.metadata['mimeType']:
    test_file=drive.CreateFile({'id': test_id })
    content=test_file.GetContentFile(fr'C:\Users\**\Desktop\xls\{test_title}.xlsx', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
    path = r"C:\Users\**\Desktop\xls"
    file_list = glob.glob(path + "/*.xlsx")
    excl_list = []
    for file in file_list:
    excl_list.append(pd.read_excel(file))
    excl_merged = pd.concat(excl_list, ignore_index=True)
    excl_merged.to_excel(r'C:\Users\**\Desktop\combined.xlsx')

    The script version is dry, but since I'm not a programmer, this is a hint for you where to look for the convenience of work.
    I worked through Anaconda. And yes, PowerBi needs to be run through Anaconda, because you will have to download a bunch of packages like pandas