Tapatalk

IP List duplicate IP removal

IP List duplicate IP removal

11
Basic UserBasic User
11

PostJun 02, 2021#1

Hi, 

I have a file with 1000s of IPs, is there a way I can strip out all the duplicate IPs to only 1 entry of it?

Thanks

Darcey
6,824625
Grand MasterGrand Master
6,824625

PostJun 02, 2021#2

Run a sort with removing duplicate lines.
11
Basic UserBasic User
11

PostJun 02, 2021#3

Thanks for the info, will use that in future. However I just wrote a node script to do this for me and dump it to a json file.
19276
MasterMaster
19276

PostJun 02, 2021#4

Hi,

Perl regexp, probably not very performing in a big file. Just for fun :)

F: ^(.+)$\r\n(?=(?s:.*)^\1$)
R: empty

BTW I've just found a bug in UE regexp engine. You can try the regexp but it can skip some matches. I am going to report the bug to IDM.

BR, Fleggy