Saturday, May 14, 2005

application-sensitive image-bending

update: go to my new image databending gallery, the new home for all my databent image work!

i touched upon this briefly in my previous post, as well as back in march, but i definitely think this demands further study.

if you don't want to read a bunch of technical databending crap, please scroll down because i have an experiment for you.

different image formats behave differently when they are "databent" (that is, when they are edited or manipulated in an "incorrect" environment), depending on the level of complexity of the image data and how it's stored. in my image editing experience (which is far more limited than i'd like), i can classify the image formats i've worked with into two general categories: simple formats, which are very resilient to bending (so long as you don't damage the header, which tends to be very short) and yield fairly predictable results, and volatile formats, which tend to be fairly fragile but produce peculiar results if you manage to bend them without completely breaking them.

BMP and TIF are simple formats, and their general structure quickly becomes obvious after a few bending experiments. files are structured with a short header and a data block, and that's about it. there might be a footer, but if so then altering it doesn't seem to corrupt the file. the data block for these formats is a basic stream of pixel data, which is read left-to-right, top-down. there is no equivalent to a "line break"; the size of the image is established in the header and the pixels simply load in, in order. each pixel is defined as an RGB (red-green-blue) color value, so the basic structure looks something like this:


if you were to move or delete the first byte of the data block (the first "R"), the entire image would "color shift" to green (because every pixel's G value would be interpreted as R, etc). this effect can be clearly seen in bends such as this one:

similarly, if you were to run some kind of algorhythmic processing on the data block (i'm thinking audio FX because i prefer to bend in an audio editing environment, but the same principles would apply to other environments), you would simply "corrupt" or alter the color values of the pixel data itself. for example, here is the bollywood vanilla coke ad, converted to BMP, with sound forge's "fuzz 1" distortion preset effect run on the pixel data:

(i haven't experimented much with using FX on simple formats, but others have... sorry, i don't have any links right this second)

PSD and JPG are both volatile formats. they don't behave anything like simple formats. it's very easy to alter the wrong byte of data and corrupt the whole file. pixel data is definitely not stored in a simple stream; it appears to be stored in two- or three-dimensional arrays (how many arrays are used varies depending on options used). the structure or even the location of these arrays is not transparent to the databender: after dozens of times editing PSD files in a wav editor, i have a good idea of which areas not to edit, but locating the exact boundaries of those areas is beyond me).

editing the data in these arrays corrupts the pixels in a structural sense rather than simply color and location. bizarre horizontal banding effects are common (as seen in the wallpaper i posted on sunday and the bulk of my image bends).

anyway, on to the inspiration for this post:

it has become increasingly apparent that when bending volatile image files, the end result varies substantially depending on the end application and OS used to open the bent file. i suspected it was true back in march, and now i'm dead certain.

the proof came today when i posted an actual bent jpg file. i bent the file using goldwave, then opened it in mspaint to verify that it would open. it did, so i posted it, only to realize that the same file looked different when i viewed it in firefox (using win2k). so i opened it again in paint and copied the image to a new file, to create a permanent version of how paint interpreted the image, and posted that as well. i also tried to capture a version of how it looked in firefox, but while taking the screenshot was easy i didn't have any good cropping tools, so i didn't end up posting it (i should've emailed myself the screenshot... stupid).

later browser testing revealed that the image wouldn't even open in internet explorer. and when i got home and checked on my computer at home (again using firefox, using xp sp1), the image looked different still!

this is a powerful reminder of one of the core concepts of databending: you never really working directly with data; you work with the data as interpreted by the editing environment you use to access that data. there is at least one built-in layer of abstraction present at all times. with image-bending, you could say that the "real" bending happens not when you edit the data, but when that data is reinterpreted back into its original format. the same exact file will bend differently when opened in different applications on the same computer/OS, and will also look different when viewed in the same application but on a different computer/OS.

this raises some fundamental questions: just how important is the "computer" part of the equation? will two machines running the same OS and application versions interpret a file in the same way? or do hardware variations, etc factor in? could it be that a bent volatile file will render differently on every computer you open it in? or does OS even factor in at all?

maybe i'm giving away too many of my "trade secrets" in this post (as if anyone other than syntax is reading by this point), but i desperately want your participation in the following experiment:

stop scrolling now. here's the experiment:

look at this image:

if you don't see an image at all (just a red X or whatever), that's fine. you probably won't see one if you're using internet explorer (though if you're using some other browser and can't see it, i'd like to know).

if you can see an image above, compare it to the following images:

if the image above looks like one of those images, please tell me which one it looks like, and what OS and browser you're using, including browser version (you can generally find this in any application's help menu under "about [application]").

if the image above looks different than any of these, please email me a screen capture of what it looks like. this is extremely important for the experiment to yield any useful results: it will only take at most a couple minutes and i would really appreciate your help. i will post your submitted image, your name/handle, and maybe even a shout-out or two if you like. if you are comfortable with cropping and want to crop it, great, but you sure don't have to. (if you don't know how to take a screenshot, see "tech help" below)

extra credit: if you really want to go above and beyond to help out the experiment and get major brownie points too, here's what else you can do: save the bent image (the one above) to your hard drive, and open it up in microsoft paint (or other image editors; i can't open it in photoshop, but maybe someone could on a mac, not to mention other application on other platforms). if it also looks different than any of these, select all, copy to the clipboard, and paste it to a new document (should be 333x485 pixels), then email me the document.

edits/updates: much thanks everyone for responding, and especially syntax, erroneousrex, ian page-echols, jack smiley, and ken goudswaard for sending in their screenshots (the fifth image is syntax's; the 6th, 7th, & 8th are rex, ian, and jack's mac os x submissions, respectively; and the 9th is ken's). i'll print a full list of everyone who responded (who i have names for) when i post all the results, probably in a few days.

tech help: if you know how to take a screenshot, you can stop reading now.

how to take a basic screen capture (windows): view the image and try to make sure you have the whole image visible. then press the print screen key on your keyboard (it might be abbreviated "prt scr" or similar). this will take a "screen shot" of whatever is visible on your monitor, and place it on your windows clipboard. then you simply need to go into an image editor, create a new document that's sufficiently large, and the "paste" the screenshot from the clipboard into a new document. save the file, email it to me, and i'll love you forever.

if you've never used an image editor (or don't think you have one): go to your start menu, choose the "accessories" menu, and then choose "paint". the microsoft paint application will open. in paint, go to the "image" menu and select "attributes". set the "units" to "pixels" and make sure the image size is at least as big as your screen (800x600, 1024x768, 1280x960, etc). now you're ready to paste in your screenshot. go to the "edit" menu and select "paste". then save it up and email it to me. JPG format is better than BMP, etc.

mac users: i'm not really qualified on this subject, but in mac os x, you can create a pdf screenshot by pressing Command+Shift+3. i will gladly accept these pdf files, though i would prefer something like jpg if you are a screenshot wiz.