1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
Q: I used the word Pickle in a conversation, and (Glyph and/or JP) shotgunned a handful of Prozac and washed it down with whiskey. Is he / are they okay? A: Pickle is baaaad news, kids. It's *great* if your tiny little application needs to save some data in a hurry. It generates some BIIIIG problems if you use it on a large scale. Your data tends to calcify along with your application code. It's very difficult to find the links between them in either direction; you don't know what application code your data references, you don't know what data your application code is producing. The fact that it's easy to manage this on a framework level is deceptive - once you start saving real data the rules all change. Explicit is better than implicit. *EVERYTHING* in Pickle is implicit. It is possible - nay, *easy*, to have some totally random part of your application stick temporary data to an in-database objects. Once objects 'get dirty' in this fashion, it's nearly impossible to find them, and when you do, the usual way to detect the problem is by having a 'load' operation explode! As a corollary to that, since there are no explicit schemas, if you write an otherwise valid upgrader which does not delete an obsoleted attribute, that attribute will silently sit around forever, bloating your database until the end of time (or until objects you THOUGHT were gone from your data are deleted from your code, only to surprise you by then not loading) cPickle can be coerced to coredump for certain inputs, both on store and on load. Some of these inputs are valid, some are the result of programming bugs. Heaven help you if you ever create a bug in a reduce method. Also, 'regular' Pickle is too slow even for a joke. There are no tools, besides a raw Python prompt, to investigate the contents of a pickle, or to profile the disk or memory usage of different parts of it. They are completely opaque blobs. We have experienced all of these issues in production and some of them still give us nightmares. While we've never lost any user data, it has certainly made the process of upgrading and enhancing our production server... challenging.
This paste never expires. View raw. Pasted through import.