IP List duplicate IP removal

IP List duplicate IP removal

12
Basic UserBasic User
12

    Jun 02, 2021#1

    Hi, 

    I have a file with 1000s of IPs, is there a way I can strip out all the duplicate IPs to only 1 entry of it?

    Thanks

    Darcey
    Web Developer by day, DAW Tinkerer by night.
    6,686585
    Grand MasterGrand Master
    6,686585

      Jun 02, 2021#2

      Run a sort with removing duplicate lines.
      Best regards from an UC/UE/UES for Windows user from Austria
      12
      Basic UserBasic User
      12

        Jun 02, 2021#3

        Thanks for the info, will use that in future. However I just wrote a node script to do this for me and dump it to a json file.
        Web Developer by day, DAW Tinkerer by night.
        19476
        MasterMaster
        19476

          Jun 02, 2021#4

          Hi,

          Perl regexp, probably not very performing in a big file. Just for fun :)

          F: ^(.+)$\r\n(?=(?s:.*)^\1$)
          R: empty

          BTW I've just found a bug in UE regexp engine. You can try the regexp but it can skip some matches. I am going to report the bug to IDM.

          BR, Fleggy