As documented here, the signature for the std::priority_queue template is:
template<
class T,
class Container = std::vector<T>,
class Compare = std::less<typename Container::value_type>
> class priority_queue;
So you need to specify the underlying container as a template argument. You can't rely on the default template argument, as you're trying to specify the comparator (third template argument). Also, I'm not sure that the way you specify your comparator is correct... You might want to specialize std::less for vertex to use your custom comparison function instead, and rely on the default template parameters.
You may want to try this:
class vertex;
bool compareVertex(vertex *v1, vertex *v2);
namespace std {
template<>
struct less <vertex*> {
bool operator() (vertex* lhs, vertex* rhs) const {
return compareVertex(lhs, rhs);
}
};
}
std::priority_queue<vertex*> pq;
Note that your less specialization has to be defined inside the global or std namespace.
EDIT:
Apparently, from the comments, your compareVertex function is actually a member function of some graph class.
In that case, you probably want to do something like this:
struct VertexComparator {
explicit VertexComparator(graph& g)
: graph_(g) {
}
bool operator() (vertex* lhs, vertex* rhs) const {
return graph_.compareVertex(lhs, rhs);
}
private:
graph& graph_;
};
typedef priority_queue<vertex*, vector<vertex*>, VertexComparator> tVertexPriorityQueue;
graph aGraphInstance;
tVertexPriorityQueue pq(VertexComparator(aGraphInstance));