'Post in Group' API using Nodejs

[nfadili]

Hello. I am implementing a feature that allows a user to upload a .pbix file from their local machine through our webpage and have it published into a Power BI workspace by our web app's server. The only way I was able to get this to work was by creating a temp file of the user's .pbix on the server, calculating it's length, and then sending the POST /imports request with a `Content-Length` header with that length along with a stream of the temp file.

My main question is this: does the POST /imports endpoint require the 'Content-Length' to be calculated before the request is sent? I am trying to avoid storing potentially large temp files on the server (which is how I can get the length). The ideal scenario is that the file streams from the browser to the nodejs server, and then directly to the Power BI API endpoint. I have tried various http libraries and various combinations of headers and received a few different errors from Power BI. Sometimes I get 500 Internal Server Error responses with no detail, and sometimes I get 400 InvalidFileSizeError responses like this:

{"error":{"code":"InvalidFileSizeError","pbi.error":{"code":"InvalidFileSizeError","parameters":{"FileSize":"9223372036854775807"},"details":[],"exceptionCulprit":1}}}

It seems that with nodejs if a 'Content-Length' header is not specified when sending a stream it might default to using `Infinity`. Not sure if that is pertinent or not.

Here's a rough idea of the code that prints responses from Power BI:

const fetch = require('node-fetch');
const FormData = require('form-data');


const form: FormData = new FormData();

// readStream is a stream from the post request to the node server from the browser and is made with `createReadStream` from the 'streams' module
form.append('', readStream); 

const url = `https://api.powerbi.com/v1.0/myorg/groups/${workspaceId}/imports?datasetDisplayName=${displayName}&nameConflict=Abort`;
const options = {
    method: 'POST',
    body: form,
    headers: {
        Authorization: `Bearer <token>`,
        ...form.getHeaders()
    }
};

return fetch(url, options)
    .then((res) => {
        console.log(res.ok);
        console.log(res.status);
        console.log(res.statusText);
        console.log(res.headers.raw());
        return res.text();
    })
    .then((data) => console.log(data))
    .catch((err) => {
        console.log(err);
    });

If anyone knows the reason for these errors or has any experience with streaming forms to Power BI, any help would be very very much appreciated. Thank you!

Helpful links

- https://docs.microsoft.com/en-us/rest/api/power-bi/imports/postimportingroup

- https://github.com/form-data/form-data#readme

- https://github.com/node-fetch/node-fetch

[dpiret]

Hi @nfadili 

I made it work, but I'm unsure whether it will help you.

I used form-data's async getLength() function, which returns the length of the stream. This length is not exactly the file's lenght you obtain from the disk:

- fs.statSync(__basedir +'/pbix/workspace-master.pbix').size: 199460

- from getLenght: 199672

A crude working version looks like this

try {
 let AccessToken = await auth.getMicrosoftToken();
 var formData = new FormData();
 formData.append(datasetDisplayName,fs.readFileSync(__basedir +'/pbix/workspace-master.pbix'));

 var options={};
 options.method= 'POST';
 options.data= formData;
 options.url= Mustache.render("https://api.powerbi.com/v1.0/myorg/groups/{{groupId}}/imports?datasetDisplayName={{datasetDisplayName}}", { groupId: groupId, datasetDisplayName: datasetDisplayName });

 var headers = formData.getHeaders();
 headers['Authorization'] = `Bearer ${AccessToken}`;
 headers['content-type'] = 'multipart/form-data'

 formData.getLength(async function(err, len) {
   headers['Content-Length']=len;
   options.headers= headers;
   console.log(`uploading PIX ${datasetDisplayName} to the workspace ${groupId}`)
   let response = await http.axios(options)
   console.log(`It may have worked\n${JSON.stringify(response.data)}`);
   return response.data
 });

 I guess I'll see you around 😉

d

[dpiret]

Hi @nfadili , did you sort this out. 

I'm having the exact same message issue using Axios (I guess FileSize: '9223372036854775807' means infinite?). Mi pbix file size i s just 194KB

My code is quite similar to yours

async postImportInGroup(groupId, datasetDisplayName) {
try {
   let AccessToken = await auth.getMicrosoftToken();
   var formData = new FormData();
   formData.append(datasetDisplayName, fs.createReadStream(__basedir +'/pbix/workspace-master.pbix'));
   var headers = formData.getHeaders();
   headers['Authorization'] = `Bearer ${AccessToken}`;
   headers['content-type'] = 'multipart/form-data'
   const options = {
      method: 'POST',
      headers: headers,
      data: formData,
      url: Mustache.render("https://api.powerbi.com/v1.0/myorg/groups/{{groupId}}/imports?datasetDisplayName={{datasetDisplayName}}", { groupId: groupId, datasetDisplayName: datasetDisplayName }),
   };
   let response = await axios(options)
   return response.data
   }

cheers

[nfadili]

Hey @dpiret,

I ended up contacting MSFT support about this issue shortly after posting here. They told me that the Content-Length header is required but is not listed in their documentation https://docs.microsoft.com/en-us/rest/api/power-bi/imports/post-import-in-group - they also told me that they would update the documentation to include this detail but have not done so.

I am still using the method I described above (write the file to disk, calculate length, submit, delete from disk) but I would prefer not to. If you figure anything else out please share 🙂