Forum Discussion

jaryszek's avatar
jaryszek
Icon for Super User rankSuper User
1 year ago
Solved

How to get table directly from GitHUB using web connector and outh2?

Hi Guys, I created ustom connector like here: [DataSource.Kind="GitHub", Publish="GitHub.Publish"] shared GitHub.Contents = (repositoryName as text, optional folderPath as text) => let ...
  • johnbasha33's avatar
    1 year ago

    Hi jaryszek 
    You're doing great so far — building your custom GitHub connector is not easy stuff.
    I'll help you extend it nicely so you can also fetch CSVs directly from a GitHub repo and load them as a table.

    What you want:

    Add a function to your connector that:

    • Downloads a .csv file from GitHub

    • Reads it into a Power Query table

    Exactly like GithubSample.PagedTable in the Microsoft example.

    Here's how you can extend your connector:

    You add a new function like this:

    [DataSource.Kind="GitHub", Publish="GitHub.Publish"]
    shared GitHub.LoadCsvTable = (repositoryName as text, filePath as text) =>
    let
    baseUrl = "https://raw.githubusercontent.com/xxx/" & repositoryName & "/main/" & filePath,
    csvContent = Web.Contents(baseUrl, [
    Headers = [
    #"Authorization" = "Bearer " & Extension.CurrentCredential()[access_token],
    #"User-Agent" = "PowerQuery"
    ]
    ]),
    csvTable = Csv.Document(csvContent, [
    Delimiter = ",",
    Columns = 10, // You can change based on your file
    Encoding = 65001, // UTF-8
    QuoteStyle = QuoteStyle.Csv
    ])
    in
    csvTable;
    shared GitHub.LoadCsvTable = (repositoryName as text, filePath as text) =>
    let
    baseUrl = "https://raw.githubusercontent.com/xxx/" & repositoryName & "/main/" & filePath,
    csvContent = Web.Contents(baseUrl, [
    Headers = [
    #"Authorization" = "Bearer " & Extension.CurrentCredential()[access_token],
    #"User-Agent" = "PowerQuery"
    ]
    ]),
    csvTable = Csv.Document(csvContent, [
    Delimiter = ",",
    Encoding = 65001,
    QuoteStyle = QuoteStyle.Csv
    ]),
    output = Table.PromoteHeaders(csvTable)
    in
    output;

    Then inside Power BI or Power Query UI:

    You can call:
    GitHub.LoadCsvTable("your-repo-name", "folder/subfolder/yourfile.csv")

    Did I answer your question? Mark my post as a solution! Appreciate your Kudos !!