• Post Reply Bookmark Topic Watch Topic
  • New Topic

Check duplication in a very large excel file  RSS feed

 
Vince Hon
Ranch Hand
Posts: 117
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have an excel file (or csv) that contains many records (e.g. e-mail address).
aaa@aaa.com
bbb@bbb.com
fff@fff.com
ccc@ccc.com
.
.
.
.
Some of the records may be duplicated and I want to delete the duplicate one and provide a new list that without duplications.
Personally, I first sorted the record alphabetically and then comparing each record one by one.
As the excel file contains thousounds of records, the processing time is critical.
Is my algorithm work ? or are there any better solutions ?
Thanks
Vince
 
Peter den Haan
author
Ranch Hand
Posts: 3252
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Please refer to replies given in the Performance forum -- and please don't post the same question in more than one forum.
- Peter
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!