Tortured pixels is a term/concept that refers to editing that damages the pixel quality of an image. One of the ways it shows up is in "banding," which is when a gradual gradation of color becomes distinct bands. You can see the banding in the above image on her forehead and cheek; it becomes abruptly white rather than gradually. Really, anytime you adjust an image, you're torturing the pixels, which is why Photoshop gurus recommend non-destructive editing (which at its best is keeping edit adjustments localized rather than global-- for example using a mask so that some elements are left untouched).
Sometimes, you can't tell that you've over-tortured a pixel until you see the print. Your software program's histogram is an easy way to see if you're damaging the image integrity. The image above I blew out in Photoshop for a high key look (specifically for this post; it's not something I'd do in life). Below is the histogram. Notice the broken lines? That indicates pixel torture. This image may look fine on screen, but print it anything larger than 4x6 and you'll see the image degradation.