Monday, January 07, 2013

summoning ghost frames

this is a write-up of the ghost frames workshop i delivered at GLI.TC/H 2112

ghost frames is a term i use to describe video frames which do not actually exist in a video file but which are sometimes revealed when using damaged data and/or glitch processes. for the sake of discussion, i'll also use "ghost frames" to refer to the process i use to invoke these frames. my process involves a bug in certain versions of avidemux, though there are likely other ways you could do something similar. (nick extracrispy tells me he can do something similar in quicktime, for example.)

specifically, i use avidemux version 2.5.4. the process is likely to work in other 2.5 and earlier versions, though i haven't tested them. it definitely does not work in the current version, 2.6. also, during the workshop, some mac users reported that they couldn't get avidemux to install. for os x lion/mtn lion, check these instructions for installing avidemux.

i initially discovered ghost frames thanks to another glitchy behavior in avidemux. my desktop computer runs windows 7 and has a video capture card for recording tv signals off a coax connection. the default tv capture program in windows 7 is called windows media center, which records programs in a format called WTV. WTV is proprietary (closed) format, but it's close enough to MPEG that when i try to open a WTV file in avidemux 2.5.4, it asks me "This looks like an MPEG. Do you want to index it?"

answering yes to this question and no to any follow-up questions results in a video that is recognizable but thoroughly mangled. here are some typical examples:

arise_glitch8

occasionally you'll get weirder, unpredictable glitches like this:

arise_glitch11

i'd been playing around with this wtv transcoding glitch for months before i saw my first ghost frame, which i discovered by accident. to invoke ghost frames in avidemux, scroll through a damaged video backwards. just skip to the end (or any point in the middle, if you prefer), and use your left arrow key to reverse through the file. if your file is damaged in the "right" way (say, via WTV transcoding), then you'll start to see frames that aren't really there:

parade45

what seems to be happening here is that avidemux is getting confused and applying incremental changes in the wrong direction. to save space, MPEG files only occasionally send full frames (keyframes or i-frames) and fill the rest of the video with p- and b- frames that only describe which pixels have changed since the previous frame. reversing through the file seems to trick avidemux into applying these changes wrong, thus revealing glitched frames that aren't actually in the file. (and i love the fact that you do this by going in reverse; it really makes it seem like some magic ritual.)

you can even do something i call dancing between the keyframes, inching back and forth through a glitchy area of the file as the artifacts become increasingly pronounced:

ninja125

of course, it's one thing to see a ghost, and something else altogether to capture a ghost. ghost frames disappear just as suddenly as they appear. skip so much as one frame too far and those ghostly glitches you've been cultivating could vanish. you can't just hold down the left arrow key and storm through the place: you have to inch along. you have to creep.

so that's how you summon ghost frames from WTV files. it's all well and good if you're running windows 7 with a tv capture card, and have video content you can pipe through a coax connection, but that's a fairly narrow demographic. also, WTV transcoding introduces some artifacts i'd prefer to be able to turn off, like the banding at the top of the screen. so when preparing for the ghost frames workshop, i spent a few hours digging around for non-WTV files that would work with this process. most files i tried wouldn't. but by chance, i found one file that did. further investigation revealed that the teaser videos on NSFW adult website ellinude.com could be used to summon ghost frames—but only the videos from the past several months. (i don't have a subscription to the site so i don't know if the full-length subscriber videos glitch in the same way, though it seems likely.)

fun glitches! but because i found these files in the wild and they seem to be normal working video files, i don't know what about them makes them glitch like this. more about that below.

areas to explore

  1. crack the mystery of what makes the ellinude videos glitch. is it something in the encoding settings? something in the camera that shot them? something in the actual frame content? a combination, or other factors? the obvious goal is to reproduce these effects on files of our own choosing.
    my analysis suggests these files were encoded using ffmpeg, but i haven't quite been able to reverse-engineer the encoding settings. (and don't know if it would do much good even if i could.) some metadata from some of the video files is available here (thanks to antonio roberts for his help compiling this metadata).
  2. if we can summon ghost frames from transcoded WTV files and from ellinude's MP4s, then they could likely be found in other types of mangled/miscoded video. damaged/databent files, deliberately poor encoding, hacked codecs, etc are possible avenues for source video.
  3. this process has a lot of similarities with datamoshing. the two processes could likely be combined to interesting effect.
  4. version/system sensitivity testing—will this process work in other versions of avidemux, and if so which ones? will the same file produce the same ghost frames using different versions of avidemux or different operating systems?
  5. identify other tools/processes for summoning ghost frames.

18 comments:

Andrés said...

i've been working only with images. i'm just starting on this work. i use "ghost frames" but as images. i like to reduce the reproduction speed to watch the "ghost frames" appear. then i capture the image and the results are pretty awesome. video or gif files look great too.

Adam Garet said...

Forget about all the things bothering you, it ain’t indeed as significant as your real cheerfulness – it is what really matters!

Unknown said...
This comment has been removed by the author.
Unknown said...

Man i have something interesting, i was converting a pdf book downloaded from the internet archive to .bmp using a cracked version of debenu and i got some freakish glitch i've never seen. I tried to figure i out but... dont know nothing about codes and stuff.
http://s30.postimg.org/3zb8xlwox/mariamorevna00biliuoft.jpg

Cheap Flights said...

vé tết 2019 đi đà lạt
vé tết 2019 đi đà nẵng
vé tết 2019 đi đồng hới
vé tết 2019 đi buôn mê thuột
vé tết 2019 đi cần thơ

Teqhow said...

Commenting on a blog is an art. Good comments create relationships. You're doing a great job. Keep it up.

TeQHow
TeQHow.com
TeQHow Blogger
TeQHow Technology

safecredit said...

The Building CommYOUnity week, held in September, is essential for SAFE's extended support of the network to help address difficulties presented by the pandemic. SAFE gave $60,000 in crisis awards to four nearby associations to help them with monetary misfortunes and extra requirement for their administrations because of COVID-19. SAFE likewise dispersed huge number of defensive face covers to territory not-for-profits to assist them with helping neighborhood inhabitants.safecreditunion

Field engineer said...

hi,
computer science career paths
If you love all things tech and computer related, a Computer Science or Computer Engineering degree may be right for you. But what’s the difference between the two subjects? It may seem like a daunting decision to make so to make it easier for you to decide between the two, we’ve created a list of the core curriculum and potential career paths you could take!

lucifer1413 said...

All your banking hassle, your payments, and receipts are taken care of. It is, in fact, a single-stop solution to all your monetary transactions. One of the two best things about PayPal is its instantaneity. You don’t have to worry about the transaction getting failed, or taking way too much time. Another thing that makes PayPal worthy is its service availability across the world. It follows you wherever you go to make your life easier and quicker. Let’s discuss steps to making an account, and resolutions to PayPal login issues.
PayPal login |
www.amazon.com/mytv |
youtube.com activate |
Xfinity Login |
DirecTV login |
Zoom login

mokshamask said...

Dust mask and pollution mask are made to keep you safeguard from viruses, bacteria, dust, and other harmful particles. There are several types of face masks available in the market serving different purposes. Our n95 mask online is one of the most protective and laced with innovative technology available in the market at the lowest price. The replaceable n95 filter, washable cloth mask, silent fan, environment friendly, and cost-effective n95 mask online ensures soothing airflow without compromising safety.

jeewangarg said...

It is extremely nice to see the greatest details presented in an easy and understanding manner. Get Digital Marketing Services At Affordable Price At JeewanGarg.Com (SEO Company in Delhi)

houseofcandy said...

House of Candy is your one-stop-destination to satisfy all your candy needs. Discover from our extensive collection of sweet candies, sour candies perfect for gifting, birthday party and much more. Buy from our online candy shop the sweet candy, sour candies , gums and jellies, fizzy, chocolates, lollipops, marshmallow at excellent prices. Buy candy online India from the house of candy, the leading candy manufacturers, suppliers and wholesaler in India availing the online discount.

Vipul Singh said...

Splendid information!
ENT Specialist Doctor in Meerut
Top School in Ghaziabad
Quality Education in Ghaziabad
Sinusitis Surgery in Meerut
Website Designing Company in Meerut

onlinebattery said...

Onlinebattery.co.in is No.1 inverter battery shop in Faridabad, under the aegis of Garg Trading Company, offering 100% genuine batteries with manufacturer warranty. We have the complete range of automotive, two-wheeler, inverter, VRLA/SMF Batteries, generator, solar batteries and water purifier. We sell all types of battery brands like Exide, Amaron, Luminous, Okaya, MtekPower, Su-Kam, Tata Green, and Adwin. From solar panels to water purifiers, Garg Trading Company is the podium for a wide range of car batteries, inverter batteries at the best price online in Delhi, NCR.

s said...

Mukherjee is one of the Associate Designer Members of Fashion Design Council of India and the youngest board member of the National Museum of Indian Cinema. It is a dream of every girl to walk down the aisle in your vague bridal lehenga. Shop online the best of sabyasachi lehenga , Sabyasachi Sarees, Sabyasachi Anarkali & Sharara Set with Panache Haute Couture. Shop now with us online and avail the best of Sabyasachi Lehenga and give a dreamy start to your second inning of life.

mokshamask said...


n95 mask online and pollution mask are made to keep you safeguard from viruses, bacteria, dust, and other harmful particles. There are several types of face masks available in the market serving different purposes. Our n95 mask online is one of the most protective and laced with innovative technology available in the market at the lowest price. The replaceable n95 filter, washable cloth mask, silent fan, environment friendly, and cost-effective n95 mask online ensures soothing airflow without compromising safety.

s said...

cutting corrugated plastic not only have these wonderful qualities but also are super easy to cut. With the right kind of tools, you can quickly cut these sheets using the followingCorrugated Plastic sheets comprise of three layers- two flat plastic sheets with a wave-like ribbed center layer. The two flat plastic sheets are generally referred to as twinwall plastic. The Corrugated Plastic sheets may also be like sheets of plastic that have a wave-like structure that might have chopped glass fiber reinforced in it

prakash tradewell said...

Prakash Tradewell, with over decades of experience in manufacturing and supplying electrical insulating sleeves and cables, has boarded on new industry benchmarks. Our range includes the best fibre sleeves, heat-shrinkable sleeves, fiberglass sleeve, insulating wires, fibre glass manufacturer. We also offer polyester yarns and polyester threads. Prakash Tradewell’s procure high-quality raw material from our reliable vendors for the manufacturing of these products.