Forum Discussion

nfadili's avatar
nfadili
Frequent Visitor
6 years ago

'Post in Group' API using Nodejs

Hello. I am implementing a feature that allows a user to upload a .pbix file from their local machine through our webpage and have it published into a Power BI workspace by our web app's server. The only way I was able to get this to work was by creating a temp file of the user's .pbix on the server, calculating it's length, and then sending the POST /imports request with a `Content-Length` header with that length along with a stream of the temp file.

 

My main question is this: does the POST /imports endpoint require the 'Content-Length' to be calculated before the request is sent? I am trying to avoid storing potentially large temp files on the server (which is how I can get the length). The ideal scenario is that the file streams from the browser to the nodejs server, and then directly to the Power BI API endpoint. I have tried various http libraries and various combinations of headers and received a few different errors from Power BI. Sometimes I get 500 Internal Server Error responses with no detail, and sometimes I get 400 InvalidFileSizeError responses like this:

 

 

{"error":{"code":"InvalidFileSizeError","pbi.error":{"code":"InvalidFileSizeError","parameters":{"FileSize":"9223372036854775807"},"details":[],"exceptionCulprit":1}}}

 

 

 

It seems that with nodejs if a 'Content-Length' header is not specified when sending a stream it might default to using `Infinity`. Not sure if that is pertinent or not.

 

Here's a rough idea of the code that prints responses from Power BI:

 

 

 

const fetch = require('node-fetch');
const FormData = require('form-data');


const form: FormData = new FormData();

// readStream is a stream from the post request to the node server from the browser and is made with `createReadStream` from the 'streams' module
form.append('', readStream); 

const url = `https://api.powerbi.com/v1.0/myorg/groups/${workspaceId}/imports?datasetDisplayName=${displayName}&nameConflict=Abort`;
const options = {
    method: 'POST',
    body: form,
    headers: {
        Authorization: `Bearer <token>`,
        ...form.getHeaders()
    }
};

return fetch(url, options)
    .then((res) => {
        console.log(res.ok);
        console.log(res.status);
        console.log(res.statusText);
        console.log(res.headers.raw());
        return res.text();
    })
    .then((data) => console.log(data))
    .catch((err) => {
        console.log(err);
    });

 

 

 

If anyone knows the reason for these errors or has any experience with streaming forms to Power BI, any help would be very very much appreciated. Thank you!

 

Helpful links

- https://docs.microsoft.com/en-us/rest/api/power-bi/imports/postimportingroup

- https://github.com/form-data/form-data#readme

- https://github.com/node-fetch/node-fetch

3 Replies

  • Hi nfadili , did you sort this out. 

    I'm having the exact same message issue using Axios (I guess FileSize: '9223372036854775807' means infinite?). Mi pbix file size i s just 194KB

    My code is quite similar to yours

    async postImportInGroup(groupId, datasetDisplayName) {
    try {
       let AccessToken = await auth.getMicrosoftToken();
       var formData = new FormData();
       formData.append(datasetDisplayName, fs.createReadStream(__basedir +'/pbix/workspace-master.pbix'));
       var headers = formData.getHeaders();
       headers['Authorization'] = `Bearer ${AccessToken}`;
       headers['content-type'] = 'multipart/form-data'
       const options = {
          method: 'POST',
          headers: headers,
          data: formData,
          url: Mustache.render("https://api.powerbi.com/v1.0/myorg/groups/{{groupId}}/imports?datasetDisplayName={{datasetDisplayName}}", { groupId: groupId, datasetDisplayName: datasetDisplayName }),
       };
       let response = await axios(options)
       return response.data
       }

    cheers

     

    • nfadili's avatar
      nfadili
      Frequent Visitor

      Hey dpiret,

       

      I ended up contacting MSFT support about this issue shortly after posting here. They told me that the Content-Length header is required but is not listed in their documentation https://docs.microsoft.com/en-us/rest/api/power-bi/imports/post-import-in-group - they also told me that they would update the documentation to include this detail but have not done so.

       

      I am still using the method I described above (write the file to disk, calculate length, submit, delete from disk) but I would prefer not to. If you figure anything else out please share 🙂

  • Hi nfadili 

     

    I made it work, but I'm unsure whether it will help you.

     

    I used form-data's async getLength() function, which returns the length of the stream. This length is not exactly the file's lenght you obtain from the disk:

    fs.statSync(__basedir +'/pbix/workspace-master.pbix').size: 199460

    - from getLenght: 199672

     

    A crude working version looks like this

     

    try {
     let AccessToken = await auth.getMicrosoftToken();
     var formData = new FormData();
     formData.append(datasetDisplayName,fs.readFileSync(__basedir +'/pbix/workspace-master.pbix'));
    
     var options={};
     options.method= 'POST';
     options.data= formData;
     options.url= Mustache.render("https://api.powerbi.com/v1.0/myorg/groups/{{groupId}}/imports?datasetDisplayName={{datasetDisplayName}}", { groupId: groupId, datasetDisplayName: datasetDisplayName });
    
     var headers = formData.getHeaders();
     headers['Authorization'] = `Bearer ${AccessToken}`;
     headers['content-type'] = 'multipart/form-data'
    
     formData.getLength(async function(err, len) {
       headers['Content-Length']=len;
       options.headers= headers;
       console.log(`uploading PIX ${datasetDisplayName} to the workspace ${groupId}`)
       let response = await http.axios(options)
       console.log(`It may have worked\n${JSON.stringify(response.data)}`);
       return response.data
     });

     

     

     I guess I'll see you around ğŸ˜‰

    d