The general rule is that const is multiple-reader safe, and other operations are not.
But a number of exceptions are carved out. Basically, the non-const operations returning iterators or references to existing objects (no object is created) are also considered "reader" operations, and are multiple-reader safe. Using the non-const_iterators they return to modify the underlying data often has issues, but in many containers such modifications only cause contention if done on the same element that another operation is accessing.
So
if (map.find(foo) == map.end())
is safe to use with other operations that only read from the map object.
There are still good reasons to call the const operations. In c++17 there is std::as_const, permitting
if (std::as_const(map).find(foo) == map.cend())
which only calls const methods on map. You can write your own as_const easily:
template<class T>
std::add_const_t<T>& as_const( T& t ) { return t; }
(in c++11 you'll need to expand add_const_t). The std version adds
template<class T>
void as_const( T const&& t ) = delete;
to block some relatively pathological cases.
...
When thinking about thread safety, realize that it is a relational property. Two operations are relatively thread safe.
In the case of std containers, you have to think about how the operation reads or writes the container itself, which elements it reads or writes. While the standard is more technical, that will give you an intuitive understanding of what is allowed.
Methods that only read the container and return iterators or references to elements are sometimes non-const, but they themselves only read the container.
Operations that mutate or read elements are often only exclusive with other operations that mutate that element. In a vector, you are free to v[0] = 77; while another thread reads v[7] with no synchronization (outside of vector<bool>). But the moment you .push_back with insufficient capacity, you need to be synchronized with every other read.
This can be counter intuitive. Be careful, and read the documentation, if you are doing synchonization free access to a container. The intuition is only a first step.