How can OrderedDict know about the element order of an already instantiated dict?

Question

I was playing around with the OrderedDict type in Python 3.6 and was surprised by its behaviour. When I create a simple dict like this in IPython:

d = dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])

I get:

{'guido': 4127, 'jack': 4098, 'sape': 4139}

as an output, which doesn't preserve the order of elements at instantiation for some reason. Now, when I create an OrderedDict from d like this:

od = OrderedDict(d)

the output is:

OrderedDict([('sape', 4139), ('guido', 4127), ('jack', 4098)])

Now I ask myself, how can the OrderedDict-constructor know about the order of elements at instantiation of d? And does it always behave the same, such that I can rely on the order of elements in the OrderedDict?

I was already reading the Python docs about dictionaries and OrderedDicts but I didn't find an answer to my question.

The output from (sys.version):

In[22]: sys.version
Out[22]: '3.6.1 (default, Apr  4 2017, 09:40:21) \n[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)]'

Answer 1

It's now obvious that the custom hook (sys.displayhook) that IPython uses to display output is pretty printing things (using it's own pretty printer).

By directly calling displayhook you can see how it ruins the insertion order:

In [1]: from sys import displayhook
   ...: displayhook({'1': 0, '0': 1})
Out[1]: {'0': 1, '1': 0}

In addition, if you grabbed the dictionary str instead (sending a string to be displayed instead of a dict object) you'd get the correct and expected order:

In [2]: d = dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
   ...: d
Out[2]: {'guido': 4127, 'jack': 4098, 'sape': 4139}

In [3]: str(dict(t))
Out[3]: "{'sape': 4139, 'guido': 4127, 'jack': 4098}"

similarly by printing it.

I'm not sure why IPython does this with 3.6, it was quite confusing (edit: see relevant issue on GitHub). In your standard Python REPL, this behavior won't manifest since sys.displayhook isn't implemented to do any pretty printing.

---

The dict d you've created does maintain insertion order, that's why the OrderedDict is maintaining that same order.

The fact that it does is, of course, an implementation detail. Until that is changed (and it does appear that it will) you should stick to using OrderedDict to reliably maintain order across implementations.

---

By the way, if you want this disabled, you could start IPython with the --no-pprint option which disables its pretty printer:

➜ ipython --no-banner --no-pprint 

In [1]: dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
Out[1]: {'sape': 4139, 'guido': 4127, 'jack': 4098}

Answer 2

In 3.6, as an implementation detail, all dicts are ordered. You're being fooled by IPython: Before 3.6, the order of keys was arbitrary, so for user-friendliness, IPython's interactive output for dict and set (where normal Python would just print the repr) sorts the keys. That's why your dict appears to be in alphabetical order. It's possible IPython might eventually drop that behavior when running on 3.6+, since as you've noticed, it is quite confusing.

If you explicitly print, rather than relying on ipython to output the results of the previous expression for you, you'll bypass ipython's REPL magic and see the "natural" order. Same goes for just about any other means of interacting with the dict, since iteration will proceed in the expected order.

Answer 3

As you probably know, dictionaries in Python are not ordered according to the language specification. They do have an inherent order but that order is arbitrary.

So when you pass a standard dictionary to the constructor of an OrderedDict, the new OrderedDict will be filled from the values of the original dictionary by iterating its values. That way, the inherent order of the dictionary will be used, and that will be what you will be seeing in the final OrderedDict.

Now, with Python 3.6, there was a change in the implementation of the default dictionary. As discussed and explained on this question, standard dictionaries now preserve the insertion order. That’s why when you created the OrderedDict from the Python 3.6 dict, the original order is preserved as well.

Does this mean that OrderedDict becomes obsolete in Python 3.6+? No, as the order preserving of standard dictionaries is an implementation detail. Instead of the arbitrary order of previous implementations, the new dictionary just happens to have the “correct” order. But this is in no way guaranteed by the language specification, and may or may not be the case for other implementations. As such you cannot and should not rely on it.

Btw. note that Python 3.6 (the language, not just the implementation) does guarantee that the order of keyword arguments to OrderedDict is preserved. E.g. this preserves the order:

>>> OrderedDict(sape=4139, guido=4127, jack=4098)
OrderedDict([('sape', 4139), ('guido', 4127), ('jack', 4098)])