The ultimate Fabric, Power BI, SQL, and AI community-led learning event. Save €200 with code FABCOMM.
Get registeredEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
Good morning everyone,
I have a table with a series of invoice rows imported from XML. Basically, each row in the table corresponds to a line item in an invoice. I have performed a series of operations and reached this table:
id_doc_r | Idx | id-doc | SommaRighe | CodiceFiscale | Fornitore | TipoDoc | DataDoc | NumeroDoc | ImportoTotaleDocumento | Causale | CodiceCIG | NumeroLinea | Descrizione | RigaFT-NC | StatoVerifica | MeseCompetenza | AnnoCompetenza
where id_doc_r uniquely identifies the invoice line.
I also have a mapping table like this:
Cod_Fornitore | NomeFornitore | ParolaChiaveServizio | ParolaChiaveServizio2 | ParolaChiaveBeneficiario | Servizio | Beneficiario
Cod_Fornitore | NomeFornitore | ParolaChiaveServizio | ParolaChiaveServizio2 | ParolaChiaveBeneficiario | Servizio | Beneficiario |
11612350153 | COOPERATIVA X | Spazio | Neutro | Spazio Neutro | ||
92020910152 | ASSOCIAZIONE CIELO | Accoglienza | B.G. | Poverta | Bianchi | |
92003770159 | ASSOCIAZIONE TERRA | DDN | ||||
09319650966 | Azienda Sanitaria Z | CDX | Milano | I.R. | Servizio CDX | Rossi |
09319650966 | Azienda Sanitaria Z | CDX | P.D. | Servizio CDX | Verdi | |
09319650966 | Azienda Sanitaria Z | CDX | Servizio CDX | |||
02840030121 | BETA COOPERATIVA SOCIALE | AES Baobab | ||||
01014660432 | ESPRESSO ITALIA Srl | SMART | ||||
11062930151 | XXX Onlus Soc. Coop. Soc. | Accoglienza | H.E. | Comunità | Hotel |
I would like to assign a service and possibly a beneficiary to each row in the invoices table based on the following rules:
I hope I was clear. I tried with ChatGPT but got poor results... I guess I should first study Power Query properly...
Thanks for now,
DebianG
Solved! Go to Solution.
hello.. thanks i solved in this way:
let
// Choose whether to refer directly to the table (query) or the result in the worksheet (Excel table).
// This is only for query speed reasons since the previous table only needs to be updated once or twice a day.
Origine = Excel.CurrentWorkbook(){[Name="competenza_riferimento"]}[Content],
// Origine = #"competenza-riferimento",
// Join with the mapping table
UnioneMapServizi = Table.NestedJoin(Origine, {"CodiceFiscale"}, MappingServizi, {"Cod_Fornitore"}, "MatchServizio", JoinKind.LeftOuter),
// Add the "ServizioFinale" column without expanding the subtable
AggiungiServizioFinale = Table.AddColumn(
UnioneMapServizi,
"ServizioFinale",
each
let
descr = [Descrizione], // Text to analyze
subtab = [MatchServizio], // Subtable with keywords and service
// Filters subtable rows based on the presence of keywords
// - if none of the keywords are found but the supplier (matching tax code) has a default service, assign that one;
// otherwise, search for service and beneficiary keywords.
righeMatch = Table.SelectRows(
subtab,
// Anonymous function that searches within the description field
(r) =>
(r[ParolaChiaveServizio] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio]))) and
(r[ParolaChiaveServizio2] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio2]))) and
(r[ParolaChiaveBeneficiario] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveBeneficiario])))
)
in
if Table.IsEmpty(righeMatch) // If the result is empty, assign null
then null
else righeMatch{0}[Servizio], // Take the first valid service
type text
),
// Group by invoice
RaggruppaPerFattura = Table.Group(
AggiungiServizioFinale,
{"id-doc"},
{{"Dati", each _, type table}}
),
// Assign the service (the first valid one) to all invoice rows if present in at least one row; otherwise, remains null
PropagaServizioFattura = Table.AddColumn(
RaggruppaPerFattura,
"ServizioFattura",
each
let
ListaServizi = List.RemoveNulls(List.Distinct(Table.Column([Dati], "ServizioFinale")))
in
if List.Count(ListaServizi) > 0 then List.First(ListaServizi) else null,
type text
),
// Expand the "ServiziFattura" column
EspandiServizioFattura = Table.ExpandTableColumn(
Table.NestedJoin(
AggiungiServizioFinale,
"id-doc",
PropagaServizioFattura,
"id-doc",
"DatiServizio",
JoinKind.LeftOuter
),
"DatiServizio",
{"ServizioFattura"}
),
// Add the "BeneficiarioFinale" column without expanding the subtable
AggiungiBeneficiarioFinale = Table.AddColumn(
EspandiServizioFattura,
"Beneficiario",
each
let
descr = [Descrizione], // Text to analyze
subtab = [MatchServizio], // Subtable with keywords and beneficiary
// Filter subtable rows based on the presence of keywords
righeMatch = Table.SelectRows(
subtab,
(r) =>
(r[ParolaChiaveServizio] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio]))) and
(r[ParolaChiaveServizio2] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio2]))) and
(r[ParolaChiaveBeneficiario] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveBeneficiario])))
)
in
if Table.IsEmpty(righeMatch)
then null
else righeMatch{0}[Beneficiario], // Take the first valid match
type text
),
// Remove auxiliary columns
#"Rimosse colonne1" = Table.RemoveColumns(AggiungiBeneficiarioFinale,{"MatchServizio", "ServizioFinale"}),
#"Rinominate colonne1" = Table.RenameColumns(#"Rimosse colonne1",{{"ServizioFattura", "Servizio"}}),
// Add a value of 1 to rows where a beneficiary is present, otherwise 0.
// The idea is to assign 1 to all rows of that invoice by grouping them.
Verifica_presenza_Ben = Table.AddColumn(#"Rinominate colonne1", "Presente_Beneficiario", each if [Beneficiario] <> null then 1 else 0),
// Group and assign 1 to all rows of the invoice where at least one row contains a beneficiary
RaggruppateRigheFattBen = Table.Group(Verifica_presenza_Ben, {"id-doc"}, {{"Pres_Ben_Fattura", each List.Max([Presente_Beneficiario]), type number}, {"table", each _, type table [id_doc_r=text, Idx=number, #"id-doc"=text, SommaRighe=number, CodiceFiscale=text, Fornitore=text, TipoDoc=text, DataDoc=datetime, NumeroDoc=text, ImportoTotaleDocumento=number, Causale=nullable text, CodiceCIG=nullable text, NumeroLinea=number, Descrizione=text, #"RigaFT-NC"=number, StatoVerifica=text, MeseCompetenza=text, AnnoCompetenza=text, Servizio=nullable text, Beneficiario=text, Presente_Beneficiario=number]}}),
// Expand the table, order columns, and remove "Presente_Beneficiario" column (which refers to a single row), keeping "Pres_Ben_Fattura"
EspandiBenFattura = Table.ExpandTableColumn(RaggruppateRigheFattBen, "table", {"id_doc_r", "Idx", "SommaRighe", "CodiceFiscale", "Fornitore", "TipoDoc", "DataDoc", "NumeroDoc", "ImportoTotaleDocumento", "Causale", "CodiceCIG", "NumeroLinea", "Descrizione", "RigaFT-NC", "StatoVerifica", "MeseCompetenza", "AnnoCompetenza", "Servizio", "Beneficiario", "Presente_Beneficiario"}, {"id_doc_r", "Idx", "SommaRighe", "CodiceFiscale", "Fornitore", "TipoDoc", "DataDoc", "NumeroDoc", "ImportoTotaleDocumento", "Causale", "CodiceCIG", "NumeroLinea", "Descrizione", "RigaFT-NC", "StatoVerifica", "MeseCompetenza", "AnnoCompetenza", "Servizio", "Beneficiario", "Presente_Beneficiario"}),
// Order and remove column
OrdinaColonne = Table.ReorderColumns(EspandiBenFattura,{"id-doc", "id_doc_r", "Idx", "SommaRighe", "CodiceFiscale", "Fornitore", "TipoDoc", "DataDoc", "NumeroDoc", "ImportoTotaleDocumento", "Causale", "CodiceCIG", "NumeroLinea", "Descrizione", "RigaFT-NC", "StatoVerifica", "MeseCompetenza", "AnnoCompetenza", "Servizio", "Beneficiario", "Presente_Beneficiario", "Pres_Ben_Fattura"}),
RimuoviColonnaBen = Table.RemoveColumns(OrdinaColonne,{"Presente_Beneficiario"}),
// Filter only invoices with a beneficiary; this will be a table to be appended later.
// Now we group the remaining rows.
FattureConBeneficiario = Table.SelectRows(RimuoviColonnaBen, each ([Pres_Ben_Fattura] = 1)),
// Reference to two previous steps (before the last filter)
NuovoFiltro = RimuoviColonnaBen,
// Filter only invoices where the beneficiary is not present
FattureSenzaBeneficiaio = Table.SelectRows(NuovoFiltro, each ([Pres_Ben_Fattura] = 0)),
// The idea is to consolidate multiple invoice rows into a single row,
// concatenating the 'Descrizione' field and removing unnecessary fields
// (which are not included in the expansion).
Of course.. sorry!
https://drive.google.com/file/d/1MmyDuoJOjoM3w4zemIpwlo-vUzptq-mn/view?usp=share_link
this is my file, there is a table 'mapping', another table 'fatture' and expeted results 'ris_atteso' in this table i would like to match the service and eventually the beneficiary (last two columns)
Hope you will be able to get the file... thanks for now !
debianG
A) Rewrite your rules from the 1st post please. You are using ENG column names, but in file there are column names in ITA language.
B) for your point number 1. How you can assign the service if there is no keyword defined for such supplier?
Be more precise in description please.
I'm so sorry dufoq3 !
thanks i try to explain better than before
I have two tables 'mapping' and 'fatture'.
'fatture' is a table of rows of invoices, each row correspond to a row of an invoice.
this is my new mappin table (different from that of my first post) and same of inside the attached file:
Cod_Fornitore | NomeFornitore | ParolaChiaveServizio | ParolaChiaveServizio2 | ParolaChiaveBeneficiario | Servizio | Beneficiario |
11612370845 | cooperativa theta | diurna | Ca.Ma. | Diurna | Paperone | |
11612370845 | cooperativa theta | Neutro | Spazio N | |||
92021009711 | associazione delta | Accoglienza | B.G. | Poverta | Paperino | |
92003818788 | società futura | DDN | ||||
9319730774 | cooperativa alfa | CDD | I.R. | Servizio CDD | Pluto | |
9319734127 | cooperativa alfa | CDD | P.D. | Servizio CDD | Pippo | |
9319683628 | cooperativa alfa | CDD | C.D. | Servizio CDD | Topolino | |
2840121395 | società gamma | AES Baobab | ||||
1014685988 | cooperativa omega | SMART | ||||
11063000570 | associazione gamma | Accoglienza | H.E. | Comunità H |
I try to rewrite the rules based on file attached:
I would like to assign a service (coloumn "Servizio") and possibly a beneficiary (column "Beneficiario") to each row in the "fatture" table based on the following rules:
1) If there is no keyword defined for a given supplier (column "NomeFornitore" (e.g., "cooperativa omega"), I directly assign the "Servizio" (this can be done with a simple inner join, as far as I know) to all "fatture" rows for that supplier ("nome fornitore"). The field to match the 'mapping' table with the 'fatture' table is Cod_Fornitore in 'mapping' and CodiceFiscale in 'fatture'. In this case, it is sufficient for **Cod_Fornitore** and **CodiceFiscale** to match, and the *Servizio* is assigned.
2) If there is no Beneficiary Keyword "ParolachiaveBeneficiario", but at least one or both Service Keywords ("ParolaChiaveServizio" anda "ParolaChiaveServizio2") are defined (simultaneously) (e.g., associazione gamma), I assign the service to all "fatture" rows only if one or both keywords appear in the description field (column "Descrizione").
3) If a Beneficiary Keyword ("ParolaChiaveBeneficiario") is defined, I assign the service to the entire invoice and the beneficiary only to the specific row where both the beneficiary keyword and other service keywords (if present "ParolaChiaveServizio" and "ParolaChiaveServizio2") are found in the description field (column "Descrizione") (e.g., cooperativa alfa).
I was thinking of performing the matching with this strategy: first, I assign the *Servizio* to the invoices where no keyword is needed, only the match between **Cod_Fornitore** and **CodiceFiscale**, and I create a table. Then, I will perform a left anti join to identify the rows that are still unmatched and apply the other two rules.
The rows that have not been matched will have the value "NON ASSEGNATO" in the *Servizio* column.
hope is clear than before.. thanks for now
hello.. thanks i solved in this way:
let
// Choose whether to refer directly to the table (query) or the result in the worksheet (Excel table).
// This is only for query speed reasons since the previous table only needs to be updated once or twice a day.
Origine = Excel.CurrentWorkbook(){[Name="competenza_riferimento"]}[Content],
// Origine = #"competenza-riferimento",
// Join with the mapping table
UnioneMapServizi = Table.NestedJoin(Origine, {"CodiceFiscale"}, MappingServizi, {"Cod_Fornitore"}, "MatchServizio", JoinKind.LeftOuter),
// Add the "ServizioFinale" column without expanding the subtable
AggiungiServizioFinale = Table.AddColumn(
UnioneMapServizi,
"ServizioFinale",
each
let
descr = [Descrizione], // Text to analyze
subtab = [MatchServizio], // Subtable with keywords and service
// Filters subtable rows based on the presence of keywords
// - if none of the keywords are found but the supplier (matching tax code) has a default service, assign that one;
// otherwise, search for service and beneficiary keywords.
righeMatch = Table.SelectRows(
subtab,
// Anonymous function that searches within the description field
(r) =>
(r[ParolaChiaveServizio] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio]))) and
(r[ParolaChiaveServizio2] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio2]))) and
(r[ParolaChiaveBeneficiario] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveBeneficiario])))
)
in
if Table.IsEmpty(righeMatch) // If the result is empty, assign null
then null
else righeMatch{0}[Servizio], // Take the first valid service
type text
),
// Group by invoice
RaggruppaPerFattura = Table.Group(
AggiungiServizioFinale,
{"id-doc"},
{{"Dati", each _, type table}}
),
// Assign the service (the first valid one) to all invoice rows if present in at least one row; otherwise, remains null
PropagaServizioFattura = Table.AddColumn(
RaggruppaPerFattura,
"ServizioFattura",
each
let
ListaServizi = List.RemoveNulls(List.Distinct(Table.Column([Dati], "ServizioFinale")))
in
if List.Count(ListaServizi) > 0 then List.First(ListaServizi) else null,
type text
),
// Expand the "ServiziFattura" column
EspandiServizioFattura = Table.ExpandTableColumn(
Table.NestedJoin(
AggiungiServizioFinale,
"id-doc",
PropagaServizioFattura,
"id-doc",
"DatiServizio",
JoinKind.LeftOuter
),
"DatiServizio",
{"ServizioFattura"}
),
// Add the "BeneficiarioFinale" column without expanding the subtable
AggiungiBeneficiarioFinale = Table.AddColumn(
EspandiServizioFattura,
"Beneficiario",
each
let
descr = [Descrizione], // Text to analyze
subtab = [MatchServizio], // Subtable with keywords and beneficiary
// Filter subtable rows based on the presence of keywords
righeMatch = Table.SelectRows(
subtab,
(r) =>
(r[ParolaChiaveServizio] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio]))) and
(r[ParolaChiaveServizio2] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveServizio2]))) and
(r[ParolaChiaveBeneficiario] = null or Text.Contains(Text.Lower(descr), Text.Lower(r[ParolaChiaveBeneficiario])))
)
in
if Table.IsEmpty(righeMatch)
then null
else righeMatch{0}[Beneficiario], // Take the first valid match
type text
),
// Remove auxiliary columns
#"Rimosse colonne1" = Table.RemoveColumns(AggiungiBeneficiarioFinale,{"MatchServizio", "ServizioFinale"}),
#"Rinominate colonne1" = Table.RenameColumns(#"Rimosse colonne1",{{"ServizioFattura", "Servizio"}}),
// Add a value of 1 to rows where a beneficiary is present, otherwise 0.
// The idea is to assign 1 to all rows of that invoice by grouping them.
Verifica_presenza_Ben = Table.AddColumn(#"Rinominate colonne1", "Presente_Beneficiario", each if [Beneficiario] <> null then 1 else 0),
// Group and assign 1 to all rows of the invoice where at least one row contains a beneficiary
RaggruppateRigheFattBen = Table.Group(Verifica_presenza_Ben, {"id-doc"}, {{"Pres_Ben_Fattura", each List.Max([Presente_Beneficiario]), type number}, {"table", each _, type table [id_doc_r=text, Idx=number, #"id-doc"=text, SommaRighe=number, CodiceFiscale=text, Fornitore=text, TipoDoc=text, DataDoc=datetime, NumeroDoc=text, ImportoTotaleDocumento=number, Causale=nullable text, CodiceCIG=nullable text, NumeroLinea=number, Descrizione=text, #"RigaFT-NC"=number, StatoVerifica=text, MeseCompetenza=text, AnnoCompetenza=text, Servizio=nullable text, Beneficiario=text, Presente_Beneficiario=number]}}),
// Expand the table, order columns, and remove "Presente_Beneficiario" column (which refers to a single row), keeping "Pres_Ben_Fattura"
EspandiBenFattura = Table.ExpandTableColumn(RaggruppateRigheFattBen, "table", {"id_doc_r", "Idx", "SommaRighe", "CodiceFiscale", "Fornitore", "TipoDoc", "DataDoc", "NumeroDoc", "ImportoTotaleDocumento", "Causale", "CodiceCIG", "NumeroLinea", "Descrizione", "RigaFT-NC", "StatoVerifica", "MeseCompetenza", "AnnoCompetenza", "Servizio", "Beneficiario", "Presente_Beneficiario"}, {"id_doc_r", "Idx", "SommaRighe", "CodiceFiscale", "Fornitore", "TipoDoc", "DataDoc", "NumeroDoc", "ImportoTotaleDocumento", "Causale", "CodiceCIG", "NumeroLinea", "Descrizione", "RigaFT-NC", "StatoVerifica", "MeseCompetenza", "AnnoCompetenza", "Servizio", "Beneficiario", "Presente_Beneficiario"}),
// Order and remove column
OrdinaColonne = Table.ReorderColumns(EspandiBenFattura,{"id-doc", "id_doc_r", "Idx", "SommaRighe", "CodiceFiscale", "Fornitore", "TipoDoc", "DataDoc", "NumeroDoc", "ImportoTotaleDocumento", "Causale", "CodiceCIG", "NumeroLinea", "Descrizione", "RigaFT-NC", "StatoVerifica", "MeseCompetenza", "AnnoCompetenza", "Servizio", "Beneficiario", "Presente_Beneficiario", "Pres_Ben_Fattura"}),
RimuoviColonnaBen = Table.RemoveColumns(OrdinaColonne,{"Presente_Beneficiario"}),
// Filter only invoices with a beneficiary; this will be a table to be appended later.
// Now we group the remaining rows.
FattureConBeneficiario = Table.SelectRows(RimuoviColonnaBen, each ([Pres_Ben_Fattura] = 1)),
// Reference to two previous steps (before the last filter)
NuovoFiltro = RimuoviColonnaBen,
// Filter only invoices where the beneficiary is not present
FattureSenzaBeneficiaio = Table.SelectRows(NuovoFiltro, each ([Pres_Ben_Fattura] = 0)),
// The idea is to consolidate multiple invoice rows into a single row,
// concatenating the 'Descrizione' field and removing unnecessary fields
// (which are not included in the expansion).
Hi @debianG,
If your issue was resolved, please mark your own post as the solution so this will be helpful for other community members who have similar problems to solve it faster.
Thank you.