How to convert complex lists into strings and back in Python 3.7?

Question

I'm trying to save Python lists to a file and read them back again when I re-launch the program. The problem is my lists are complex: i.e. different amounts of tuples within tuples.

The best I could come up with is turning my list into a string initially (which works) but there's no way I can think of to revert the changes.

with open(filename, 'w') as f:
            f.write(str(objs))
            f.close()

This works but how do I return this to a list?

Just to clarify what my definition of a complex list is, here's an example:

[(((0.0, 0.0, 0.0), (1000.0, 0.0, 0.0), (0.0, 2.0, 0.0), (1000.0, 2.0, 
0.0), (0.0, 0.0, 1000.0), (1000.0, 0.0, 1000.0), (0.0, 2.0, 1000.0), 
(1000.0, 2.0, 1000.0)), ((0, 2, 3, 1), (4, 6, 7, 5), (1, 3, 7, 5), (4, 6, 
2, 0), (2, 6, 7, 3), (4, 0, 1, 5)), ((255, 0, 0), (255, 128, 0), (255, 255, 
0), (255, 255, 255), (0, 0, 255), (0, 255, 0)))]

You could save the list in a [pickle](https://docs.python.org/3/library/pickle.html) file, or save it as [json](https://docs.python.org/3/library/json.html) (though this will convert the tuples to lists). — snakecharmerb, Feb 19 '19 at 13:58
"The best I could come up with is turning my list into a string initially" don't do that. Use a supported serialization format, like `pickle` for arbitrary objects, or even just `json` will work for a simple list — juanpa.arrivillaga, Feb 19 '19 at 16:02

trsvchn · Accepted Answer · 2019-02-21T00:44:57.247

As @snakecharmerb already mentioned you can use json or pickle. Here is an example:

Code:

my_list = [(((0.0, 0.0, 0.0), (1000.0, 0.0, 0.0), (0.0, 2.0, 0.0), (1000.0, 2.0, 
0.0), (0.0, 0.0, 1000.0), (1000.0, 0.0, 1000.0), (0.0, 2.0, 1000.0), 
(1000.0, 2.0, 1000.0)), ((0, 2, 3, 1), (4, 6, 7, 5), (1, 3, 7, 5), (4, 6, 
2, 0), (2, 6, 7, 3), (4, 0, 1, 5)), ((255, 0, 0), (255, 128, 0), (255, 255, 
0), (255, 255, 255), (0, 0, 255), (0, 255, 0)))]

import json

with open('my_list.json', 'w') as f:
    json.dump(my_list, f)

with open('my_list.json','r') as f:
    loaded_list = json.load(f)

print('Using json:')
print(loaded_list)

import pickle

with open('my_list.pkl', 'wb') as f:
    pickle.dump(my_list, f)

with open('my_list.pkl', 'rb') as f:
    loaded_list = pickle.load(f)

print('Using pickle:')
print(loaded_list)

Output:

Using json:
[[[[0.0, 0.0, 0.0], [1000.0, 0.0, 0.0], [0.0, 2.0, 0.0], [1000.0, 2.0, 0.0], [0.0, 0.0, 1000.0], [1000.0, 0.0, 1000.0], [0.0, 2.0, 1000.0], [1000.0, 2.0, 1000.0]], [[0, 2, 3, 1], [4, 6, 7, 5], [1, 3, 7, 5], [4, 6, 2, 0], [2, 6, 7, 3], [4, 0, 1, 5]], [[255, 0, 0], [255, 128, 0], [255, 255, 0], [255, 255, 255], [0, 0, 255], [0, 255, 0]]]]

Using pickle:
[(((0.0, 0.0, 0.0), (1000.0, 0.0, 0.0), (0.0, 2.0, 0.0), (1000.0, 2.0, 0.0), (0.0, 0.0, 1000.0), (1000.0, 0.0, 1000.0), (0.0, 2.0, 1000.0), (1000.0, 2.0, 1000.0)), ((0, 2, 3, 1), (4, 6, 7, 5), (1, 3, 7, 5), (4, 6, 2, 0), (2, 6, 7, 3), (4, 0, 1, 5)), ((255, 0, 0), (255, 128, 0), (255, 255, 0), (255, 255, 255), (0, 0, 255), (0, 255, 0)))]

As you can see json converts tuples to lists.

Yes, this is spot on. So, instead of initially converting your list to string and writing to a file, you can save this in a pickle file. fileName = "my_list" fileObject = open(fileName,'wb') pickle.dump(my_list,fileObject) fileObject.close() — Pallavi Roy, Feb 19 '19 at 15:31
You could use cpickle instead of pickle, cpickle is optimized and much-much faster than pickle especially if the size of data is large. the above code can be changed like "import cpickle as pickle" — Ravishankar Sivasubramaniam, Apr 10 '19 at 03:17

Varun Chhangani · Answer 2 · 2019-02-19T15:08:18.863

str function will convert your complex list / nested lists and tuple to string Further, eval converts any string to an actual code snippet

However as mentioned by Taras Savchyn, eval can lead to SQL injections and more. So instead use ast.literal_eval

Hence:

>>>import ast

>>> mylist = [(((0.0, 0.0, 0.0), (1000.0, 0.0, 0.0), (0.0, 2.0, 0.0), (1000.0, 2.0, 0.0), (0.0, 0.0, 1000.0), (1000.0, 0.0, 1000.0), (0.0, 2.0, 1000.0), (1000.0, 2.0, 1000.0)), ((0, 2, 3, 1), (4, 6, 7, 5), (1, 3, 7, 5), (4, 6, 2, 0), (2, 6, 7, 3), (4, 0, 1, 5)), ((255, 0, 0), (255, 128, 0), (255, 255, 0), (255, 255, 255), (0, 0, 255), (0, 255, 0)))]

>>> mylist
[(((0.0, 0.0, 0.0), (1000.0, 0.0, 0.0), (0.0, 2.0, 0.0), (1000.0, 2.0, 0.0), (0.0, 0.0, 1000.0), (1000.0, 0.0, 1000.0), (0.0, 2.0, 1000.0), (1000.0, 2.0, 1000.0)), ((0, 2, 3, 1), (4, 6, 7, 5), (1, 3, 7, 5), (4, 6, 2, 0), (2, 6, 7, 3), (4, 0, 1, 5)), ((255, 0, 0), (255, 128, 0), (255, 255, 0), (255, 255, 255), (0, 0, 255), (0, 255, 0)))]

>>> mystring = str(mylist)

>>> print(mystring)
'[(((0.0, 0.0, 0.0), (1000.0, 0.0, 0.0), (0.0, 2.0, 0.0), (1000.0, 2.0, 0.0), (0.0, 0.0, 1000.0), (1000.0, 0.0, 1000.0), (0.0, 2.0, 1000.0), (1000.0, 2.0, 1000.0)), ((0, 2, 3, 1), (4, 6, 7, 5), (1, 3, 7, 5), (4, 6, 2, 0), (2, 6, 7, 3), (4, 0, 1, 5)), ((255, 0, 0), (255, 128, 0), (255, 255, 0), (255, 255, 255), (0, 0, 255), (0, 255, 0)))]'

>>> type(mystring)
<class 'str'>

>>> print(ast.literal_eval(mystring))
[(((0.0, 0.0, 0.0), (1000.0, 0.0, 0.0), (0.0, 2.0, 0.0), (1000.0, 2.0, 0.0), (0.0, 0.0, 1000.0), (1000.0, 0.0, 1000.0), (0.0, 2.0, 1000.0), (1000.0, 2.0, 1000.0)), ((0, 2, 3, 1), (4, 6, 7, 5), (1, 3, 7, 5), (4, 6, 2, 0), (2, 6, 7, 3), (4, 0, 1, 5)), ((255, 0, 0), (255, 128, 0), (255, 255, 0), (255, 255, 255), (0, 0, 255), (0, 255, 0)))]

>>> type(ast.literal_eval(mystring))
<class 'list'>

Hope this solves your problem. You can comment the answer to ask any further queries

Using eval is a bad practice. https://stackoverflow.com/questions/1832940/why-is-using-eval-a-bad-practice — trsvchn, Feb 19 '19 at 14:41

How to convert complex lists into strings and back in Python 3.7?

2 Answers2