Keep/drop a random sample of rows from text (.csv) files?

Keep/drop a random sample of rows from text (.csv) files?

3
NewbieNewbie
3

    Aug 18, 2009#1

    Hi,

    I'm not sure if this is the right place to ask the question, but I was wondering if there is a way to randomly keep or drop rows in a text file using Ultraedit? For instance, randomly drop 10% of the rows, or 1000 of the rows throughout the text file?

    If anyone can give me some suggestions on how to do this, I'd really appreciate it.

    Thanks! :)
    262
    MasterMaster
    262

      Oct 05, 2009#2

      Hi! Needed a small challenge for a coffee break :-) - and maybe you have already solved this in a different way, since you wrote your request in August.

      You will need a script for this and thus use a newer version of UE supporting scripts. And here is an example script which will delete x percent of rows (lines) in a csv file. See the variable g_nDropPercentage near the top. I hope you can understand the script from the embedded comments:

      Code: Select all

      // Define percentage of rows to be dropped. Integer Number between 0 and 100.
var g_nDropPercentage = 10;

// Call as function:
dropRandomRows();

function dropRandomRows() {
  // is is a csv file - if not quit this function
  if(! UltraEdit.activeDocument.isExt("csv")) {
    return;
  }

  // Go to bottom of active document
  UltraEdit.activeDocument.bottom();

  // Ensure last line has line ending
  if (UltraEdit.activeDocument.isColNumGt(1)) {
    UltraEdit.activeDocument.insertLine();
    if (UltraEdit.activeDocument.isColNumGt(1)) UltraEdit.activeDocument.deleteToStartOfLine();
    UltraEdit.activeDocument.bottom();
  }

  // Line after the last line endning is not counted, hence minus 1
  var nTotalNumberOfLines = UltraEdit.activeDocument.currentLineNum - 1;

  // Drop further action on empty files
  if(nTotalNumberOfLines < 1) {
    return;
  }

  // Calculate the number of lines to be dropped:
  var nNumberOfLinesToBeDropped = Math.floor( (nTotalNumberOfLines / 100) * g_nDropPercentage) + 1;

  // If to few lines - drop further action:
  if(nNumberOfLinesToBeDropped<1) {
    return;
  }

  // while there are still lines to be dropped - iterate:
  while(nNumberOfLinesToBeDropped>0) {
    // calculate a random line number:
    var nRandomLineNum = Math.floor( Math.random() * nTotalNumberOfLines);

    // Goto random line and delete it:
    UltraEdit.activeDocument.gotoLine(nRandomLineNum,0);
    UltraEdit.activeDocument.deleteLine();

    // Decrease total number of lines by 1:
    nTotalNumberOfLines--;

    // Decrease number of lines to be dropped by 1:
    nNumberOfLinesToBeDropped--;
  }
}