16

Consider an input iterator like join_iterator: it iterates over the concatenation of other ranges. Calling ++i repeatedly can be much slower than a simple i += n.

Nevertheless, most C++ code that requires advancing an iterator by an arbitrary amount uses std::advance, which automatically resorts to calling ++i when the iterator isn't random-access.

(Sadly, most people use std::advance(i, n) instead of using std::advance; advance(i, n), so I can't just supply advance for my iterator and rely on ADL.)

On the other hand, I can't use + or += because input iterators don't have to implement them.

So the question is: how would I go about supporting such a scenario, when:

  • Implementing such an iterator?

  • Using an input iterator which might have an optimized operator +=?

(Note that advance and + isn't the only scenario in which this matters -- distance and - has the same problem.)

Jack V.
  • 1,381
  • 6
  • 20
user541686
  • 205,094
  • 128
  • 528
  • 886
  • Sorry, can you clarify the point about ADL? Why, exactly, does advance() not work for you? – Michael Aaron Safyan May 21 '13 at 06:35
  • 1
    @MichaelAaronSafyan: Well, it's because everyone seems to use `std::advance` instead of `advance`, so even if I provide it, it won't actually be used. – user541686 May 21 '13 at 06:37
  • Oh, so you don't actually need this fast iteration, it is just a client of yours that needs it? How exactly are you planning to use this / do you actually know that ++ is too slow? – Michael Aaron Safyan May 21 '13 at 06:39
  • 1
    @MichaelAaronSafyan: Yes, I **do** need this fast iteration. (Yes, I *actually know* `++` *is* too slow.) I've come across *both* situations myself -- writing the iterator itself, as well as writing algorithms that would benefit from such an optimized iterator. I'd like to know how to properly handle **both** cases: i.e., both how to **provide** and **consume** such an iterator. – user541686 May 21 '13 at 06:42
  • Gotcha. So, I would personally recommend using a different interface that makes this more explicit (for the uses where you care to be fast), but provide an adaptor to the iterator interface when you need to be compatible with iterator-based functions. For example, having a Next() method that requires you to specify a number to advance. – Michael Aaron Safyan May 21 '13 at 06:45
  • Wouldn't it be legal in this case to define a customized version of `std::advance` itself (i.e. inside namespace `std`)? – jogojapan May 21 '13 at 06:48
  • @jogojapan: You're not allowed to overload anything inside `std`, as far as I know. :( But if there's no other solution I just might have to do that... – user541686 May 21 '13 at 06:48
  • 1
    @Mehrdad True, but we would specialize a template, not overload a function (I think). Cf. http://stackoverflow.com/a/14403772/777186 – jogojapan May 21 '13 at 06:50
  • 1
    @jogojapan: Ah, partial function specialization works for C++11 though, and most iterators are templated so they would need partial specialization instead of full specialization, right? What would I do about C++03? – user541686 May 21 '13 at 06:51
  • @MichaelAaronSafyan: Uhm, "don't use iterators" isn't exactly a solution in any shape or form. The whole question is about how to **interoperate** properly with standard C++ code that uses iterators (which is basically all C++ code)... so obviously tossing away the entire notion of iterators isn't exactly going to help when I'm probably not going to be *both* the producer *and* the consumer in the same program, is it?! Your solution is basically the problem itself, which prompted the question... – user541686 May 21 '13 at 06:52
  • 1
    @Mehrdad You are right. My solution would only work if you specialize `std::advance` for both template parameters, i.e. specialize it explicitly. And that's no good because you can't (or wouldn't want to) anticipate all possible integer types for the second parameter. – jogojapan May 21 '13 at 07:01

1 Answers1

3

According to C++11 §24.4.4,

Since only random access iterators provide + and - operators, the library provides two function templates advance and distance. These function templates use + and - for random access iterators (and are, therefore, constant time for them); for input, forward and bidirectional iterators they use ++ to provide linear time implementations.

You should only have to define + and -, and specify std::random_access_iterator_tag. There is no need to specialize or overload std::advance.

Potatoswatter
  • 134,909
  • 25
  • 265
  • 421
  • 3
    That would be lying though -- the iterator isn't random-access. – user541686 May 21 '13 at 07:25
  • Huh? Where did I say constant time? o_O – user541686 May 21 '13 at 07:25
  • @Mehrdad Ah, I see. Still, if there's any potential gain, some underlying subranges must be random-access. I'd suggest simply telling that lie. The complexity guarantee is violated in mathematical terms, but it's the only way to get client code to pay attention to the "random-access" operators. – Potatoswatter May 21 '13 at 07:29
  • 2
    The problem with lying is that that leaves collateral damage! For example, making it random-access requires me to implement `operator--` too -- but I usually can't. Which means that any existing code that has different (e.g. optimized) behavior for random-access iterators (perhaps by using `operator-=` or `operator+=` with a negative argument) now completely fails to work with my iterators at all... and lest you think that's far-fetched, Visual C++ uses a different algorithm for `std::rotate` depending on whether or not the iterator is random-access. – user541686 May 21 '13 at 07:32
  • 1
    @Mehrdad That's unfortunate… I personally rewrote GCC's (libstdc++-v3) `std::rotate`, back in 2009, because the random-access style is universally slower than the forward-only style. The next better way to lie would be a derived or proxy iterator to add the random access tag to the "truthful" class, so the hack can be judiciously applied. I can't think of another solution without performance and storage penalties. – Potatoswatter May 21 '13 at 08:18
  • This is the sort of thing Microsoft does. Please don't. – Lightness Races in Orbit May 21 '13 at 08:29
  • @LightnessRacesinOrbit: Microsoft didn't write it, Dinkumware did. – user541686 May 21 '13 at 09:00
  • @Mehrdad: I wasn't referring to their C++ standard library implementation; I was referring to everything they do, ever (and, in particular, early versions of Internet Explorer). – Lightness Races in Orbit May 21 '13 at 12:53
  • @Potatoswatter: I don't think it's required that `+` and `-` be constant time, I have a pair I use that are logarithmic time to the size number of entries in the parent data structure. – Mooing Duck May 21 '14 at 18:53
  • @MooingDuck Reviewing the conversation, that sounds similar to Mehrdad's situation. The Standard Library guarantees constant time for its RAIs, but the templates will work regardless of the complexity of the operations you plug in. However, as Mehrdad points out, you need to be careful about performance even for logarithmic time. – Potatoswatter May 22 '14 at 00:13