I like to study films in an NLE like Premiere. You can see the rhythm of the scenes a lot clearer when you look at clips in the timeline. Like this:
Unfortunately it used to be very tedious to go through a scene (let alone a whole film) and set all the cuts again by hand. Until now. Today I created a workflow to automate edit detection for use in an NLE. All you have to do is run two tools and you get an EDL that you can import into your NLE of choice, link the media and off you go. The whole process takes maybe 15 minutes for a feature length movie.
You need to touch the command line for two commands, but stay with me, it's really simple. The hard part, the actual scene detection in the movie file, is done by the wonderful ffmpeg program, more specifically the ffprobe program. It takes a video file and creates a spreadsheet file with the times of the edit points it detected.
The second part is creating an EDL from this spreadsheet file. I wrote a little tool for this today that you can download below. It is written in C# as a console application, is release as GPL code and hosted on bitbucket if you want to compile it yourself or modify it. It should work as is on Windows and on OSX and linux if you install Mono.
If you want to use it, here is a step by step guide.
- Download ffmpeg from their website. Either put the bin directory in your path or if you don't know how to do that then put ffprobe.exe into the directory where your movie is.
- Open a command line and go to the folder where the movie file is (start->cmd.exe on windows)
Run this command and replace MOVIEFILENAME with the name of your movie file. It shouldn't contain any spaces. Be sure to copy the command exactly as stated here:
ffprobe -show_frames -of compact=p=0 -f lavfi "movie=MOVIEFILENAME,select=gt(scene\,.4)" > MOVIEFILENAME.csv
This will take a while and output a bit of status information. The ".4" is the scene detection level between 0.0 and 1.0, lower numbers create more edits, higher numbers create less edits. 0.4 should be a good default.
Download EDLGenerator.exe and run it, again from the command line, like so:
EDLGenerator.exe MOVIENAME.csv FRAMERATE MOVIENAME MOVIENAME.edl
The first file is the csv file you generated earlier, FRAMERATE is the framerate of the movie (needed for dropframe timecode corrections when appropriate), the second MOVIENAME is the source filename that should be written into the EDL file (might help with some NLEs to make linking easier) and the last is the name of the edl file to generate
- Import the edl file into your NLE (In premiere in File->Import)
- Link the media (in Premiere CS6 you can select all clips in the bin and choose link to media and just have to select the source file once even though premiere creates one source item for each edit)
- Voilla, you are done!
I have only used it on a couple of movies so far so there may be some rough edges - if you run into a problem, drop me a line.
The (german only) slides to my two recent lectures at filmArche Berlin are online now. One online presentation about Colour in Movies, and two PDFs about Digital Video Workflow and Codecs and Backups.
In my last post I wrote in length about backups but I omitted one thing: how to make incremental backups that use so called hard links and that barely take more space than 1:1 backups (on both windows and osx). First though, let me explain what is so nice about this concept.
Backups with a history
If space were no concern, it would be nice never to throw backups away. We would simply have folders that contain the date and time the backup was taken as part of the backup target folder name and keep all those backups. Then if one day we discover that we now need a file that was deleted two weeks ago we would simply access the backup from 16 days ago and restore it. If, like me, you have several TB of important data and can barely afford 2 additional sets of hard drives (one to keep as a daily backup, one that is stored at another location and that is swapped regularly) then this seems to be impossible.
If you look at your whole hard drive(s) then you will notice that between two backups only a fraction of the data actually changes. This is what incremental backups use to their advantage. They only store the new and changed files and thus save a lot of space. However now you have a full backup at one point in time and every time you run the backup again you get a new folder structure (or, if you choose a bad backup software, a proprietary single file) containing only the new and changed files. This is a bit cumbersome. Wouldn't it be great to have a full snapshot each time?
Hardlinks to the rescue!
This is where a feature called Hardlinks comes in handy. Hardlinks are a way for file systems to reference the same file several times, but only storing it once. Both NTFS (the main windows file system) and HFS+ (the main OSX files system) support hardlinks, but both operating systems hide this feature from the user interface.
What we gain from this approach
So taken together, these features enable incremental backups that look like full snapshots but only store the new and changed data. This way you only need a backup drive that is a bit bigger than your source (since you will want to have some additional space for the newly created and modified files) and you can keep a full history on it.
rsync and two GUIs for it
rsync is an open source application that is used to copy data. Since version 3 or so it supports creating snapshot copies using hardlinks. On OSX the tool backuplist+ allows you to easily create incremental backups by checking the "Incremental backups" check box and entering how many past snapshots to keep. On windows QtdSync allows you to do the same thing if you change the backup type from "synchronisation" to "incremental".
I always thought that one of the great things about digital technology is the ability to have backups - physical items can break but with digital data there is no reason why you should ever loose it, because creating exact copies of it is easily possible. And yet few people I know have a convincing backup strategy. Since I will hold a lecture at the filmArche filmschool next week on Workflow with digital files and Backups I thought this would be a good time to write the most important points about it down.
Your hard drives will fail
The question is not whether your hard drive will fail, but when. Not to have a backup of your important data is negligent and easily avoided. So I think it's well worth thinking a bit about it. If reading the title of this blog post made you feel a little guilty because you do not have a backup of your important data, read on. I promise to explain things in simple terms and walk you through some common backup scenarios for individuals or small groups of people.
Oh and one little disclaimer: thinking about backups can be a lot more complex that what I present here. This is just the bare minimum any normal person should know about backups in this digital age :)
What is a backup?
Let's start with a simple thought - what is a backup? It's an complete, independent copy of your data that does not share a single point of failure with the original copy. What's a single point of failure? Anything that can go wrong, that will destroy both copies at once. For example, if you have a backup on the same hard drive as the original file then this hard drive is a single point of failure. If it dies, then both your backup and the original data are gone.
This single point of failure thing is the key. I worked on a video shoot once where they shot with a camera that records to flash cards. The content of the flash cards was copied onto a hard drive and then to another backup hard drive before the contents of a flash card could be deleted and reused. So far so good. But then at the end of the first day the guy who did the data wrangling packed both hard drives into the same backpack and put it with the rest of the equipment for someone else to transport it to the next location. It didn't occur to him that if that backpack would fall down or get lost, both copies would be affected - it was the single point of failure.
What to back up?
Ideally you would just backup everything and be done with it. But some things are trickier to backup than others. Let's start with the obvious thing: all your "User" data should be backed up, i.e. your photos, the texts you write, the ("project") files of the applications you work with (spreadsheet software, video editing software, databases etc). This is the stuff that you really should not loose. I work as a photographer and so all photos I take fall clearly into that category, as do all the business related files like my accounting softwares files etc..
The nice thing about this category of "user data" is that you usually work with it on a regulary basis and thus know where it is. Ideally you do what I do and put all that stuff on it's own hard drive(s), away from the main operating system hard drive. This makes it easy to identify what is the most important stuff to backup (this one hard drive).
The second class of data is stuff that the software you use saves but that you do not directly interact with. Preferences for example. I work with software like Adobe Lightroom or Microsoft Visual Studio, both of which are very complex pieces of software that have a ton of user settings and user generated presets. If you were to loose these, it would probably not be the end of the world, but it would suck. The nice thing about this group of data is that it usually is rather small.
The third group of data is stuff like your operating system or the installed applications. Now while it would be nice if you could just copy all this stuff onto it's own disk and if the main hard drive fails restore it to a new disk and be done with it, this usually doesn't work. Operating systems need a boot loader and need to have certain data at certain sectors on the disc, stuff like that. So you need disc imaging software to backup this class of data which is why I do not bother to back it up. If the main system drive fails, it will take me a day or so to install the OS and the rest of the software again from the orignal DVDs, but that is ok. Your milage may vary though - if you simply can't afford a day of downtime it may make sense to create a scheme were you can back this kind of data up as well or have a second computer ready as a standby machine so you can switch quickly.
What kind of backup to do?
Most of the data we produce is not static - text files change, photos are edited, new files created all the time. So a backup needs to be done regularly. And with this comes an important decision: Do you need just one copy of your data or is it important to be able to go back to the way things looked some time ago? If all you need is an up to date copy, then it may be enough to just run a program that can mirror all the changes that happend onto your one backup. This is called a 1:1 backup. But if you need to be able to have a history of your data things become more difficult.
When you need past versions of your data, one approach is to buy X discs and use them in turn. So if you have 7 backup drives and you create 1:1 backups to one of them every day than you can go back in daily increments up to one week into the past. But this is expensive. So there is some software that let's you do this in clever ways and only store the data from the last backup. Because, you know, usually only a few files change between two backups.
There are a lot of different ways to these so called incremental backups. The easiest on the mac is Time Machine - on PC it's a bit difficult (Genie Timeline does something similiar). For smaller amounts of data, online services like dropbox usually provide some sort of history (although you have to trust that service provider to take good care of your data). But even time machine and genie timeline don't work well if you need to manage several disks, which is a common case today.
I have recenlty discovered that the powerful command line tool rsync now has the ability to create incremental backups with hard links, much like Time Machine does. I will write up my findings in another blog post.
A good solution for the real world
If all the data you care about are a few hundred MB and you usually have internet access, then a service like dropbox does all a normal user needs. But nowadays, even my grandparents have a few dozen GB of photos etc, and I have around 5 TB of important data that I need to keep safe. So I will now describe the setup I use and that I think is pretty safe and not overly complicated.
Step 1: data organization
It's important to know where your important data is. So I have a policy not to use the usual "My documents" folder or the "Documents" folder on my OS harddrive but instead put all the important data on dedicated data disks. Let's call the two data disks A and B. All your important data should be on these two disks. If the main OS drive dies, your main data should survive, even without the backups.
Step 2: get two more hard drives for each data hard drive
Why two you may think? Two reasons: First, while your backup drive is attached to your computer, your computer is a single point of failure. If you have a really nasty virus that wipes all your attached hard drives (or worse, encrypts them and extorts you with the password :) ) and you only have one backup then that's it. You had no backup because your two copies had a single point of failure. The second reason is that you should keep one of your two backup drives at another location. At a friend, your office, your lover, doesn't matter. Just pick a secure place and store it there and exchange the two disks every now and then (maybe every week or so). This way, even if you get robbed or your house burns down, the data will still exist somewhere. It will be a couple of days, maybe (if you're lazy) a few weeks old, but at least most of it will still exist. (Hint: if you do not encrypt your backup drives then you will want to store it with people you fully trust :) )
Step 3: Create your first backup
Because time machine and similar tools usually do not work with more than one hard drive, I recommend using simple 1:1 backups instead (or use a clever rsync strategy which I will desribe in another blog post soon). With todays huge disks, even a 1:1 backup that overwrites everything takes quite long, so I recommend to use rsnyc on linux or mac or robocopy on windows to create efficient 1:1 copies that only copy what has changed. This way I can backup 3 TB of data every night within 1-3 hours.
Rsync and robocopy come preinstalled with mac os x and windows respectively and both have gui frontends available for those who do not enjoy working on the command line (e.g. arrsync and yarcgui). Rsync defaults to creating a 1:1 backup with the -a option, robocopy useses /MIR for the same result.
With the gui frontends it is pretty simple to set up one copy job per hard drive (just tell it from where to where it should copy the data).
Once you finished setting up rsync or robocopy to copy the data disks to the backup drives, let it run once and check if everything worked (this first run will take a few hours per TB, possibly longer if you use a slow USB2 or Firewire connection). Subsequent runs should be much faster.
Step 4: Setup dropbox or similar for your most important, smallish files
The most important files we have are often pretty small. Many may be text files. Many will change frequently. For this kind of data it is reasonable to use dropbox and so I advise you to use dropbox in addition to the setup described above for the kind of data that is rather small and that changes a lot and where earlier versions of the file may be useful in the future.
Step 5: Setup reminders to switch backup drives, test backups
It's important to switch your two sets of backup drives regularly and to check if the backup works, so set up reminders in your calendar to do so. The best way to check if a backup works is to go to a different computer and try to open your files. If your files work with all their dependencies (linked media files in a video editing software e.g.) then you are safe. If not it's time to improve your setup.
A note on the side: RAID is not a backup
One last thing: some people have RAID setups configured for their data and think that they don't need any further backup. This is wrong. RAID is a system to protect you against hard drive failure by using redundant drives but of course the raid system itself and the computer it is attached to and the apartment that computer is located at are all single points of failure. RAID systems are nice but they are not a replacement for backups.
Whew, quite a post. This stuff may seem a bit complicated but it is the simplest version I found that keeps me safe and makes reasonable compromises for my personal use case. I could go into a lot of detail on the various thoughts on why I prefer uncompressed backups etc but I think this will do for now. If you found this post useful, if you have questions or if you spotted an error, please let me know in the comments below!
I finally finished work on the website for the film I am currently working on, 920 Milligray. The design was done by Marius Wawer and I built the html page and set up a drupal cms for the future (it currently only serves the FAQ page).
920 Milligray is going to be a drama centred on Katja, a young girl who barely knew the world before the catastrophe, and her older brother Uwe. The film is set in a post apocalyptic Europe and is currently in development. BTW, we are actively looking for production companies to get on board so if you know someone who might be interested, please tell them about the project!
One short side note on the workflow I currently use to update my websites: a lot of people who work with websites still use ftp to push new html pages or themes for a CMS to their servers. I seriously recommend to do away with that concept and instead use a distributed version control system like Mercurial or Git for this purpose. This way you have full history locally as well as on the server(s), can maintain branches (Version 2.0 e.g.) and switch once everything is ready. Not to mention the advantage of easily being able to share work between people and merge changes from different people with ease.
Git is the system used for the Linux Kernel project among others and very powerful but a bit convoluted in daily use IMHO. I prefer Mercurial which is also powerful but quite a bit simpler to use.
The way I like to work is to have a local Mercurial Repository where I commit changes in small logical batches (command line: hg addremove and hg commit). Then I have a private repository at bitbucket.org where I have easy to use and secure https access (hg push locally to push to the remote repository). Finally, on my webserver (with shell access) I have the repository that contains the content that is visible to the public (hg fetch to get the up to date version from the bitbucket repository). This way whenever something goes wrong I can just go back in the history and fix things locally. I use a similar system for all software development I do as well and more and more with other stuff as well.
Our webserver is pretty heavily firewalled but at some point I want to implement https access to the webserver repository directly so I can push without the detour via bitbucket. With a simple post-push script it will then be possible to have an up-to-date version on the webserver with a simple local hg push. Nice.
One last note: if you work on windows, tortoiseHG is a very nice GUI for daily mercurial use.
I'm doing a little lecture tomorrow at filmArche on colour grading and thought that I could share a few links on the topic here.
To give a feeling for the use of colour in movies, I really like the moviebarcodes. The artists who does them wants to stay anonymous, so I can only link to his tumbl site. The image below is that site where you can also order prints (I personally find them beautiful). Wired uk described the process of making these images very well as: [the software the artist wrote takes] every single frame of the film (and its constituent colours), stretches them vertically and lines them up in chronological order to create an image that gives a visual overview of how the film would look if you saw it all in one go.
A Scanner Darkly as a Moviebarcode
Also very useful is prolost, the blog of Stu Maschwitz. He is a filmmaker with a vfx background and helped develop a number colour grading plugins that can be used with several NLEs. Stu has a collection of colour grading tutorials for one of these plugins, Colorista II, on his site. I will show Magic Bullet Looks at my little lecture as it incorporates most of the important tools for a more complex grade and is compatible with NLEs but also usable as a standalone tool to grade stills.
Grading stills in preproduction and finding reference stills from movies or other sources is a very important preproduction step in my opinion. I really like screenshotworld for browsing stills from a number of different movies.
As you may know, digital colour grading is a rather recent practice that took off only in the early 2000s with O Brother Where Are Though being credited as the first feature film to be fully digitally graded. Digital colour grading offered a whole array of more sophisticated and more targeted manipulation of colour, which is sometimes used to a very positive effect. However, as this blog post on the Teal and Orange plague nicely illustrates, it has also led to an overuse of this very powerful colour contrast.
Human colour perception is very sophisticated (although some animals have even weirder colour perception) and colour grading therefore a very important and complicated subject. I'll write more on it some time soon, but for now I hope the links here are a nice read.
I started writing a series of blog post with the title "Artefacts of Light" at the Iconoclash Photography blog. The series will cover different phenomena in photography and the first post is about Bokeh (the rendering of out of focus light sources, as seen above). Future posts will cover things like Film Grain, Back Light and Empty Space. I always appreciate feedback, so let me know what you think about it - thanks!
Together with DoP Jan-Paul Gauly and editor Marius Wawer I have worked on the concept for a post apocalyptic action short film for quite some time now. Over time the story and setting became better and better, but our day jobs kept us all very busy this summer so we never got to actually shoot anything.
We now found a couple of days next week when all of us will have time to shoot a trailer for our film. If all goes well this will help us sell the idea and acquire some funding for a short or maybe even feature length film.
The film is set in the near future, 12 years after a big catastrophe led to the radioactive contamination of most of Europe. The few survivors live a bleak live, poisoned by radioactivity and threatened by wild animals and other survivors in the constant fight for food and shelter. Our two heroes, KATJA and her brother UWE, roam the lands, searching for a hospital that is said to be functional. On the way they are joined by MINCHI, a woman who claims to know the way. They fight off gangs and try to survive the contamination, but the longer they go on the more their personal conflicts escalate...
We created a (German) folder, have a look at it.
As some of the people who read this blog may know, I study directing at the filmschool filmArche in Berlin. Over the last year I have been asked to colour correct a number of movies since this is one of my favourite steps in filmmaking and I think I have become quite decent at it by now. I love to see my fellow filmmakers faces when I show them how much a shot can be enhanced / altered with subtle changes to the colour of the film.
But the workflow for high quality colour correction is in many cases pretty poor, especially at our school were almost everyone edits on a different system & editing software - some use avid, some final cut pro, and then there is the odd vegas or premiere user in between. I recently described our various setups and the solutions that are available today for solving this in a post to the rebel cafe: my post (the rebel cafe is a forum website for readers of Stu Maschwitz' DV Rebels Guide, a book about very low budget action filmmaking).
I usually had two choices: do the colour correction in the editing software that was used to edit the film, or import the final movie into after effects and do it there.
Working in After Effects is the better option quality wise. It can work in 32bit per channel (not 32bit per pixel!) floating point which means that no matter how many colour operations you stack ontop of each other, you will never loose any detail in your image. You can also make complex selections, use a number of very nice plugins etc. It's not realtime, but if you want the highest possible quality, it's the way to go.
Working in the editing software has some other advantages: one of the biggest is that each cut is still there and you can start grading right away. If you work in after effects, usually the first thing you have to do is split your layer at all edit points so that brightening up at one point in the film because that shot is too dark does not brighten up the whole film, only that shot. In the editing software you have all those edit points and so there is no additional work there. What's also nice is that it is (mostly) realtime. But, as I said, the quality is not the best you can get.
Usually this meant that if there was little time and/or only slight corrections required, I would do the grading in the editing software, and only if I had the time to re-set all the edit points would I go for the whole thing and do it in After Effects.
But, *drumroll* with this new script I just wrote, you can export your movie from your edit software of choice, then export an EDL (Edit Decision List) as well, import the movie into AE and then run my new script to split your layer at all edit points given in the EDL. Nice. If you want to give it a try, download the script here and let me know how it worked for you.
As the 5D mark II can still only record in 30 fps because there is no firmware update to record in more useful 25 fps (for pal DVD/TV) or 24 fps (for cinema projection), the need to retime footage from 30 fps to 25 fps definitely still exists. There are many methods to do this, but many are not as high quality as one would like. In this blog post I will explain what methods exists for retiming exist, what they look like and in the next follow up post I will show how to do high quality retiming in After Effects.
First, an overview of the problem: Your camera records in 30 frames per second (fps). You get the stuff into your editing software of choice, set the timeline to 30 fps and edit you film. You export your final film and up to here, everything is fine. But now you need a Pal DVD (to be played in Europe) or a film copy to screen in cinemas (lucky bastard). So now you have the problem of how to go from 30 fps to 25 or 24 fps.
If you had shot a silent film, you could simply play it back a little slower. A hypothetical 5 second film at 30 fps could be played at 25 fps and now be 6 seconds long. Everything would move a little slower, but maybe that could be cool. Unfortunately, if you try this with a film that uses audio, the sound would have to be slowed down as well and thus be significantly lower in pitch.
Since most films nowadays want to use sound, this leaves us with 3 options that slow down the images while maintaining the same duration and thus keep the audio unchanged.
- Frame dropping
- Frame blending
- Frame generation (aka Optical Flow Analysis, Pixel Motion, Motion Vectors)
All following samples will use PALs 25 fps, the same methods can of course be used to go to 24 fps, just the numbers are slightly different. The sample videos linked to below are all the same clip of some location scouting footage that I edited specifically so that one could study the effects of different retiming methods. Here is the original clip at 30 fps (29.97 to be exact because I converted it to cineform via NeoScene)
The first option is frame dropping. It simply removes every 6th frame. Our 5 second clip from before would still be 5 seconds long, but of the original 150 frames, 25 would have been dropped. Here is a little graphic I made to illustrate what happens.In this example we see 6 frames at 30 fps that need to become 5 frames at 25 fps (so that at both frame rates, the clip would play in exactly 1/5th of a second or 0.2 seconds).
Frame dropping leads to a strange jerky sensation that can be seen here. Some people are more sensitive to this than others:
Advantages of this method
- It works consistently
- It renders very quickly
- No problems with hard cuts
Disadvantages of this method
- Looks very strange
Frame blending always blends two frames together. So the first frame A of our new 25 fps sequence is a blending of frame A and B in the original sequence; the next frame is constructed from B and C and so forth (the exact details how and which frames are blended can vary). Here is an illustration of what happens:
In this graphics this looks pretty bizarre, but since most individual frames of a film look very much alike, this usually works pretty well. It gets more noticeable when there are quick movements, especially when the scene was filmed with a short shutter time and the objects have little motion blur. Here is a frame where the blending can be seen very well:
The good thing about frame blending is that it looks relatively pleasant most of the time and it works in any case. One of the problems with it is that it will soften your hard cuts. Here is an example where what used to be a hard cut from this frame
To this one
Gets turned into this blended frame
Here is the sample clip with frame blending.
In after effects one can protect hard cuts (see this post for more information on how to do this). The resulting sample can be seen here:
Advantages of frame blending
- It is more pleasant to watch than frame dropping
- It renders quickly
- Most of the time it looks ok
- It works consistently
- Extra care needs to be taken to protect hard cuts
- Noticeable "double exposure" effect
Frame generation is a process that goes by a myriad of names (aka Optical Flow Analysis, Pixel Motion, Motion Vectors, ...). The principle is always the same though: a clever algorithm tries to understand where similar groups of pixels move in the image and calculates in between frames by moving the pixel groups further than they used to move, an extra 1/6th of the original distance to be precise (in this particular case). The upside is that if it works, the image looks perfect and no matter how much movement is in a frame, there will never be the "double exposure" effect seen with frame blending. The downside is that even though these algorithms have gotten very good, sometimes they get badly messed up and produce something like this:
This happens basically at every hard cut, so you have to protect your hard cuts against this problem (see this post for more information on how to do this). However, it can even happen when there are no cuts, e.g. when the lighting changes dramatically (see this blog post for an image where that happened). If that happens, tweaking the parameters can often help, but sometimes you will have to resort to frame blending for a couple of frames where frame generation simply does not work.
Here is an example with pixel motion where hard cuts are unprotected (this sample uses twixtor instead of After Effects, but the results in this case are very much alike):
And here with hard cuts are protected (native After Effects again):
See the bottom of this post for some more clips retimed with the Twixtor plugin.
- If it works, the quality is very good
- Extra care needs to be taken to protect hard cuts
- Can sometimes produce messed up frames, output has to be carefully watched and problems eliminated
- Longer rendertimes
My advice would be to go with frame blending if are in a hurry, and use frame generation if you have time to check and maybe fix the output and want very high quality. In both cases you should remember to protect your hard cuts – something that to my knowledge can only be done in compositing programs like After Effects. If you have the option to use Twixtor, I can highly recommend it as it seems both more stable and significantly faster than TimeWarp. Details on how to use After Effects for retiming can be found in my next blog post.
For the sake of completeness, here are 3 samples of the result when using Twixtor with frame blending and frame generation:
Twixtor frame blending, hard cuts not protected
Twixtor frame blending, hard cuts protected
Twixtor frame generation, hard cuts protected