Forum Discussion

aseagull's avatar
aseagull
Helper IV
4 years ago
Solved

Undocumented Option to Lines.FromText or File.Contents to cap input?

Hi. I am reading large text files in daily, and will never need more than the first 20k rows (which get updated every night).  This file will grow very large. Is there a way to limit the rows read in *before* they take up M's memory, and get trimmed away?

Need Help
Tips and Tricks
  • ImkeF's avatar
    ImkeF
    4 years ago

    No, that's not possible with text files, unfortunately.
    Only with sources that support query folding.

     

3 Replies

  • ImkeF's avatar
    ImkeF
    Community Champion
    4 years ago

    No, that's not possible with text files, unfortunately.
    Only with sources that support query folding.

     

    • aseagull's avatar
      aseagull
      Helper IV
      4 years ago

      Yeah, I would make this a feature request, but probably it's not worth MS's time to expand support for reading text files. 😉

  • Anonymous's avatar
    Anonymous
    Not applicable
    4 years ago

    But since the preview is only 1000 lines, and you set your query for the top 20,000 lines, would it still commit more than 20,000 lines to memory?

     

    --Nate