Cortana integration

Hi all, I've been following the great video posts from the Guys(s) in a cube and recently implemented the one on Cortana integration. This invovled activating the Cortana services, enabling Q&A on my app after uploading it to my workspace and running the utterance check on (https://app.powerbi.com/cortana/test). After that was done the app I created was available in my start menu, perfect!

Now this worked the first time with one app quite quickly, I tried it on a second app and followed the saem steps but ran into some issues. Ran through the exact same steps but the app does not appear in the start menu. It appears in the last step which is to check the Cortana utterance section but just does not appear in the start menu. I have tried uploading it again and giving the page a different name but to no avail.

Are there some sort of limits that I'm not aware of or has anyone else run into this error and fixed it?