Error DeltaTableIsInfrequentlyCheckpointed when accessing lakehouse table

Thanks very much for this code. I was able to do some testing my end with a dataflow that I have running hourly.

Anonymous It looks to me like the _last_checkpoint" file isn't being updated when the automatic checkpoints are being created.

However, running the code from kblackburn is updating the "_last_checkpoint" file to point to the newly created checkpoint.

So, it would seem that regularly running the below code in a notebook (which will update all tables for all lakehouses in the same workspace as the notebook's default lakehouse) is the workaround until this is resolved.

%%spark

import org.apache.spark.sql.delta.DeltaLog

val lakehouses = spark.catalog.listDatabases()

lakehouses.collect().sortWith(_.name < _.name).foreach { lakehouse =>

  if (lakehouse.name != "DataflowsStagingLakehouse") {

    val tables = spark.catalog.listTables(lakehouse.name)

    tables.collect().sortWith(_.name < _.name).foreach { table =>

      DeltaLog.forTable(spark, s"${lakehouse.locationUri}/${table.name}").checkpoint()
      println(s"Completed code run for lakehouse: ${lakehouse.name}, table: ${table.name}")

    }

  }

}

This worked for me. 

For anyone getting an error looking for the org.apache.spark.sql.delta.DeltaLog import, make sure you include the first line "%%spark" indicating that this is Scala code (not pyspark).

kblackburn I have hundreds of tables with this issue, is-there an easy to to have this code run for every table in my lakehouse?  I am not very familiar with Scala, so your help would be appreciated.

Thanks kblackburn ! This is a good workaround. Thanks for sharing! I hope they'll fix this soon. Perhaps any news on a fix?