Using composite model PBI dataset content not loading

EDIT: 
I have found a solution - added Read, Build access to user groups to related dataset.


Hi

I have a workspace for PBI datasets - data models, with one place, rather than copying to multible locations.
I want to use these datasets in different reports (another workspace). I can create a connection and ceate visuals, but somehow if end-user does not have somekind of workspace right's where datasets are located, the visual will not load up.. is there a workaround or am i missing something?

I gave a user "Contributor" rights to datasets workspace and after that visuals loaded up..