Forum Discussion

JP8991's avatar
JP8991
Kudo Commander
1 year ago

Really strange issue - os.listdir

Hey all,   Could you kindly look at the below, I have a really weird one where I am referencing a location within my files area of a Lakehouse however sporadically it does not work, I have no idea ...
data engineering
notebook
frithjof_v's avatar
frithjof_v
Community Champion
1 year ago

Is Xplor a file or directory?

 

If Xplor is a file, not a directory, then I guess the error message you're seeing makes sense.

  • JP8991's avatar
    JP8991
    Kudo Commander
    1 year ago

    Xplor is a directory.

    • Brunovs79_cimen's avatar
      Brunovs79_cimen
      New Member
      1 year ago
      In this example, I use the notebookutils to loop through the files in a dir and get only the name without the extension.
      from       notebookutils.fs import ls

      import os

      application_path = 'abfss://#########@onelake.dfs.fabric.microsoft.com/000000000000/Files/folder/'

      files_list = []

      for file in ls(os.path.join(application_path,'excel_files_from_anp')):

              file_name = file.name
              if file_name.endswith('.xlsx'😞
                  files_list.append(file_name.split('.')[0])

      for some reason os.listdir doesn't work in a pipeline or with abfss path.
      Also ls is not allowed to return or anything or become an object.
      print the dir of file and you will see some properties like name, size...