Uniq commnd in unix is used to eliminate the duplicate lines from file.
Read this post on informaticapoints.com