Notebook Resources | Spark not recognized in local machine | File not recognized in local machine

Hi QixiaoWang !

No worries about the late reply. I’m glad I can contribute for the VS code experience in Fabric and, thanks for the time you put into trying to figure out these issues.

Regarding Spark session invocation:

As a quick test, I tried to invoke the spark session inside my notebook, before even going into my modules, but apparently the init_lighter.py module is not being recognized, as you can see in image below:

although it was initialized when I opened the notebook in my local machine, as you can see in the output console in above image, and also it does exist in my local machine as below image demonstrates:

Maybe, at this point, it’s just me being bad with Python, but you happen to have any clue on how to solve this one?

Regarding the logging file:

I understand what you mean, but it is still a bit odd to me that the Tables are recognized but the Files are not. I am just wondering, right now, what would be the best/advised practice to have in this case. Perhaps instead of a logging file, I should have a logging table, so I don’t have to replicate too many artifacts in my local machine, in order to avoid mistakes or maintenance complexity/overhead.

Again, thanks for the input and for your time. 🙂