TL;DR: A ConcurrentModificationException has nothing to do with threading!
The "concurrent" collections you mention in java.util.concurrent are designed to provide fast and robust alternatives to Collections.synchronizedXXX. They are for use with multiple threads at the same time.
A ConcurrentModificationException, on the other hand, happens when you alter a collection whilst iterating it in the same Thread:
final List<String> list = ...
for(final String s : list) {
list.remove(s);
}
This code will throw a ConcurrentModificationException on the second iteration every time. Accessing a non-threadsafe Collection from multiple threads at the same time is Undefined Behaviour and may never produce a ConcurrentModificationException. Or it might just catch fire.
In order to avoid such behaviour, use an Iterator and ask it to modify the Collection:
final List<String> list = ...
final Iterator<String> iter = list.iterator();
while(iter.hashNext()) {
final String s = iter.next();
iter.remove();
}
To answer your completely unrelated question, in order to create a thread safe Set simply use the following:
final Set<String> threadSafe = Collections.newSetFromMap(new ConcurrentHashMap<>());
With the addition of Collections.newSetFromMap the JDK designers did not see the point in creating a ConcurrentHashSet class.
In summary
Do not use a concurrent Collection to get around the improper use of a Collection in a single threaded environment. It will at best be slower and at worst make your code behave in unexpected ways as the consistency guarantees provided by iterators for concurrent collections are much more relaxed.