Java 8 count duplicates in List

1. Overview

In this short tutorial, we'll look at some different ways to count the duplicated elements in an ArrayList.

Table of Contents Show

1. Overview
1. Stream.distinct() – To Remove Duplicates
1.1. Remove Duplicate Strings
1.2. Remove Duplicate Custom Objects
How to find duplicate elements in a Stream in Java
How to find duplicates in Java array?
Solution 1 :
Solution 2 :
Solution 3 :
“find duplicate values in list java 8” Code Answer
Java answers related to “find duplicate values in list java 8”
Java queries related to “find duplicate values in list java 8”

1. Stream.distinct() – To Remove Duplicates

1.1. Remove Duplicate Strings

The distinct() method returns a Stream consisting of the distinct elements of the given stream. The object equality is checked according to the object’s equals() method.

List<String> list = Arrays.asList("A", "B", "C", "D", "A", "B", "C"); // Get list without duplicates List<String> distinctItems = list.stream() .distinct() .collect(Collectors.toList()); // Let's verify distinct elements System.out.println(distinctItems);

Program output:

[1, 2, 3, 4, 5, 6, 7, 8]

1.2. Remove Duplicate Custom Objects

The same syntax can be used to remove the duplicate objects from List. To do so, we need to be very careful about the object’s equals() method, because it will decide if an object is duplicate or unique.

Consider the below example where two Person instances are considered equal if both have the same id value.

public class Person { public Person(Integer id, String fname, String lname) { super(); this.id = id; this.fname = fname; this.lname = lname; } private Integer id; private String fname; private String lname; //Getters and Setters are hidden for brevity @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; Person other = (Person) obj; return Objects.equals(id, other.id); } @Override public String toString() { return "Person [id=" + id + ", fname=" + fname + ", lname=" + lname + "]"; } }

Let us see an example of how we can remove duplicate Person objects from a List.

//Add some random persons Collection<Person> list = Arrays.asList(p1, p2, p3, p4, p5, p6); // Get distinct people by id List<Person> distinctElements = list.stream() .distinct() .collect( Collectors.toList() );

To find all unique objects using a different equality condition, we can take the help of the following distinctByKey() method. For example, we are finding all unique objects by Person’s full name.

//Add some random persons List<Person> list = Arrays.asList(p1, p2, p3, p4, p5, p6); // Get distinct people by full name List<Person> distinctPeople = list.stream() .filter( distinctByKey(p -> p.getFname() + " " + p.getLname()) ) .collect( Collectors.toList() ); //********The distinctByKey() method need to be created********** public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) { Map<Object, Boolean> map = new ConcurrentHashMap<>(); return t -> map.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null; }

How to find duplicate elements in a Stream in Java

Given a stream containing some elements, the task is to find the duplicate elements in this stream in Java.

Examples:

Input: Stream = {5, 13, 4, 21, 13, 27, 2, 59, 59, 34}
Output: [59, 13]
Explanation:
The only duplicate elements in the given stream are 59 and 13.

Input: Stream = {5, 13, 4, 21, 27, 2, 59, 34}
Output: []
Explanation:
There are no duplicate elements in the given stream, hence the output is empty.

How to find duplicates in Java array?

In the first paragraph, I have given you a brief overview of three ways to find duplicate elements from Java array. Now, let's understand the logic behind each of those solutions in little more detail.

Solution 1 :

Our first solution is very simple. All we are doing here is to loop over an array and comparing each element to every other element. For doing this, we are using two loops, inner loop, and outer loop. We are also making sure that we are ignoring comparing of elements to itself by checking for i != j before printing duplicates. Since we are comparing every element to every other element, this solution has quadratic time complexity i.e. O(n^2). This solution has the worst complexity in all three solutions.
for (int i = 0; i < names.length; i++) { for (int j = i + 1 ; j < names.length; j++) { if (names[i].equals(names[j])) { // got the duplicate element } } }

This question is also very popular in programming interviews and if you are preparing for them, I also suggest you solve problems fromCracking the Coding Interview: 150 Programming Questions and Solutions. One of the best books to prepare for software developer interviews.

Solution 2 :

The second solution is even simpler than this. All you need to know is that Set doesn't allow duplicates in Java. Which means if you have added an element into Set and trying to insert duplicate element again, it will not be allowed. In Java, you can use the HashSet class to solve this problem. Just loop over array elements, insert them into HashSet using add() method, and check the return value.

If add() returns false it means that element is not allowed in the Set and that is your duplicate. Here is the code sample to do this :

for (String name : names) { if (set.add(name) == false) { // your duplicate element } }
The complexity of this solution is O(n) because you are only going through the array one time, but it also has a space complexity of O(n) because of the HashSet data structure, which contains your unique elements. So if an array contains 1 million elements, in the worst case you would need a HashSet to store those 1 million elements.

Solution 3 :

Our third solution takes advantage of another useful data structure, hash table. All you need to do is loop through the array using enhanced for loop and insert each element and its count into hash table. You can use HashMap class of JDK to solve this problem. It is the general purpose hash table implementation in Java. In order to build table, you check if hash table contains the elements or not, if it is then increment the count otherwise insert element with count 1. Once you have this table ready, you can iterate over hashtable and print all those keys which has values greater than one. These are your duplicate elements. This is in fact a very good solution because you can extend it to found count of duplicates as well. If you remember, I have used this approach tofind duplicate characters in Stringearlier. Here is how you code will look like :
// build hash table with count for (String name : names) { Integer count = nameAndCount.get(name); if (count == null) { nameAndCount.put(name, 1); } else { nameAndCount.put(name, ++count); } } // Print duplicate elements from array in Java Set<Entry<String, Integer>> entrySet = nameAndCount.entrySet(); for (Entry<String, Integer> entry : entrySet) { if (entry.getValue() > 1) { System.out.printf("duplicate element '%s' and count '%d' :", entry.getKey(), entry.getValue()); } } Time complexity of this solution is O(2n) because we are iterating over array twice and space complexity is same as previous solution O(n). In worst case you would need a hash table with size of array itself.