Forum Discussion

JoelDucharme's avatar
JoelDucharme
Helper I
1 year ago
Solved

Sharepoint list import using Implementation 2.0 gives error for a number over 1000.

If I connect the list with version 1.0, it works, but it's really slow. So I'm trying to convert my Powerquery using version 2.0, but, in 3 columns, that are a "personalised" type in the list, numbers of value over 3000 (which is formatted 3 000,00 - fr-CA) gives me an error. The screenshot is in French. It says. "Sorry... We cannot proceed to convert in a number." 

 

Unfoturnately, I cannot change the column type since it's being fed by a PowerApp that needs it to be formatted this way. 

 

I tried to change the regional settings to fr-CA. I tried to change the column type in PowerQuery with using the regional function. I tried English US, International, Canada.... nothing works. Always the same error. 

 

I even tried to convert it to text, even if it would have been a useless column. Even that didn't work.

 

Any ideas for what I can do to make it work?

 

  • v-kpoloju-msft's avatar
    v-kpoloju-msft
    1 year ago

    Hi JoelDucharme,

    Thanks again for providing the detailed code and context you’re making great progress!

    I noticed a likely issue with the line that expands a lookup column.

    Your original line:

    fieldselect = "&$top=5000&$select = Id,fournisseur,approbation_superviseurId/Titre&$expand = People"

    is likely causing problems because:

    • "People" isn't a valid field in your list.
    • The correct field name to expand is usually the same as the lookup column (e.g., approbation_superviseurId), and SharePoint expects "Title" (not "Titre") in the $select even if the list is in French.

    I've provided a corrected version of the full M code above with this fix applied. Please give it a try and let us know how it works out for you.

    M Code:

    let
        sitename = "finances-applications",
        listname = "demande_paiement",
        baseurl = "https://lescompagnonsdesfrancslo871.sharepoint.com/sites/" & sitename & "/_api/web/lists/GetByTitle('" & listname & "')/",
        itemcount = Json.Document(Web.Contents(baseurl & "ItemCount", [Headers=[Accept="application/json"]]))[value],
        skiplist = List.Numbers(0, Number.RoundUp(itemcount / 5000), 5000),
        #"Converted to Table" = Table.FromList(skiplist, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
        #"Renamed Columns" = Table.RenameColumns(#"Converted to Table",{{"Column1", "Skip"}}),
        #"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"Skip", type text}}),
    
        //  Fixed select and expand clause
        fieldselect = "&$top=5000&$select=Id,fournisseur,approbation_superviseurId/Title&$expand=approbation_superviseurId",
    
        Custom1 = Table.AddColumn(#"Changed Type", "Items", each 
            Json.Document(Web.Contents(baseurl & "/items?$skipToken=Paged=TRUE%26p_ID=" & [Skip] & fieldselect, 
            [Headers=[Accept="application/json"]]))),
        
        #"Expanded Items" = Table.ExpandRecordColumn(Custom1, "Items", {"value"}, {"value"}),
        #"Expanded value" = Table.ExpandListColumn(#"Expanded Items", "value"),
        #"value développé" = Table.ExpandRecordColumn(#"Expanded value", "value", {
            "Id", "fournisseur", "facture_date", "facture_numero", "facture_details", "AuthorId", "EditorId", 
            "Created", "Modified", "approbateur_superviseurId", "approbateur_cadreId", "l1_projet", "l1_no_compte", 
            "l1_activite", "l1_subvention", "l1_total", "l1_taxes_1", "total_grand", "total_taxes_1", "total_taxes_2", 
            "approbation_sup_statut", "approbation_sup_date", "approbation_sup_notes", "approbation_cadre_statut", 
            "approbation_cadre_date", "approbation_cadre_utilisateurId", "approbation_cadre_notes", 
            "approbation_sup_utilisateurId", "approbation_tresorie_statut", "approbation_tresorie_date", 
            "approbation_tresorie_utilisateurId", "approbation_tresorie_notes", "approbation_president_statut", 
            "approbation_president_date", "approbation_president_utilisateuId", "approbation_president_notes", 
            "finances_methode_paiement", "finances_numero_methode_paiement", "finances_no_fichier_tfe", 
            "finances_notes", "finances_verification_statut", "finances_verification_date", 
            "finances_verification_utilisateuId", "finances_inscription_statut", "finances_inscription_date", 
            "finances_inscription_utilisateurId", "statut", "est_urgent", "type_demande", "membre_du_personnelId", 
            "carte_credit", "magasin_vendeur", "est_devise_etrangere", "finances_non_approuve", 
            "finances_non_approuve_note", "est_retire", "annee_fiscale", "Attachments"
        }),
    
        #"Type modifié" = Table.TransformColumnTypes(#"value développé",{
            {"facture_date", type datetime}, 
            {"Modified", type datetime}, 
            {"approbation_sup_date", type datetime}, 
            {"approbation_cadre_date", type datetime}, 
            {"approbation_tresorie_date", type datetime}, 
            {"approbation_president_date", type datetime}, 
            {"finances_verification_date", type datetime}, 
            {"finances_inscription_date", type datetime}
        })
    in
        #"Type modifié"
    

     

    If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.

    Thank you for using Microsoft Community Forum.

18 Replies

  • v-kpoloju-msft's avatar
    v-kpoloju-msft
    Community Support

    Hi JoelDucharme,
    Thank you for reaching out to the Microsoft fabric community forum. Also Jai-Rathinavel, for his inputs on this thread. I have identified few workarounds that may help resolve the issue.

    Thank you for providing the detailed context and troubleshooting steps you've already taken. This information is very helpful in understanding the situation.

    From your description, it appears the issue is with how the SharePoint Online List connector (Implementation 2.0) parses locale-specific number formats. Specifically, numbers formatted in fr-CA (e.g., 3 000,00) use a space as the thousand’s separator and a comma as the decimal separator. The 2.0 connector currently does not handle these formats well, resulting in a DataFormat.Error when attempting to parse these values as numbers, even if the column is later converted or treated as text.

    To continue using the faster 2.0 connector while avoiding conversion errors, you can manually clean and convert these values in Power Query using a custom transformation. Please follow these steps:

    • In Power Query, identify the affected column(s).
    • Create a custom column using this transformation:
    let
    
        CleanedText = Text.Replace(Text.Replace([YourColumnName], " ", ""), " ", ""),
    
        DotDecimal = Text.Replace(CleanedText, ",", "."),
    
        ParsedNumber = try Number.From(DotDecimal) otherwise null
    
    in
    
        ParsedNumber

    This removes both regular and non-breaking spaces, replaces the comma with a dot, and then converts the value to a number in a culture-invariant format.

    If multiple columns are affected, you can use Table.TransformColumns:

    let
    
        Source = [YourSourceStepHere],
    
        CleanedTable = Table.TransformColumns(
    
            Source,
    
            {
    
                {"Column1", each try Number.From(Text.Replace(Text.Replace(Text.Replace(_, " ", ""), " ", ""), ",", ".")) otherwise null, type number},
    
                {"Column2", each try Number.From(Text.Replace(Text.Replace(Text.Replace(_, " ", ""), " ", ""), ",", ".")) otherwise null, type number}
    
            }
    
        )
    
    in
    
        CleanedTable

    Be sure to replace "Column1"/"Column2" with your actual column names.

    For your reference, here are a few related Microsoft Learn articles:

    Unfortunately, locale aware number parsing from custom SharePoint columns isn’t currently handled automatically by Implementation 2.0. This workaround allows you to preprocess and convert the values manually while still benefiting from the improved performance of the 2.0 connector.

    If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.

    Thank you for using Microsoft Community Forum.

    • v-kpoloju-msft's avatar
      v-kpoloju-msft
      Community Support

      Hi JoelDucharme,

      May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

      Thank you.

      • JoelDucharme's avatar
        JoelDucharme
        Helper I

        Unfortunately, my time is limited this week. I have tried Jai-Rathinavel's idea with the custom code and direct dynamic access. It's almost working. I have a few bugs to figure out, when I get the time.

    • JoelDucharme's avatar
      JoelDucharme
      Helper I

      Both. It takes over 1 hour to sometimes close to 2 hours for refresh the semantic model in power bi service. When I make a change in the desktop version, every step is so teadious.

       

      I know that my power query model (sorry, I'm not sure of the terms...) is not very streamlined and I'm working on it, but there has to be something wrong with the way it's working. That's why I'm exploring the 2.0 implementation, since, from what I've seen in a few places, it should be much quicker.