Whenever I peruse the forums at Creative Cow I see questions about upconversion techniques, so I thought I would write a post about it. There are some very expensive ways to get an upconversion, but there are also some very cheap ways.
So what is an upconversion? An upconversion is when you convert video from one resolution to a higher resolution. Some people also include frame rate conversions in their upconversion. Frame rate conversions should be avoided at all costs, and I will only cover them briefly here. There are also often some aspect ratio issues to consider. The most common upconversion is from SD to HD. Many people shoot their projects in SD and then want to move it to HD for projection, broadcast, or to shoot out to film. Or the project was shot on film, but only got an SD film transfer and there isn’t any money to get it retransferred to HD.
Let me stress that an upconversion is not a magical transition to HD with no quality costs. An upconversion will never look as good as real HD. At best it will be a bit soft, and at worst it will be extremely soft and aliased with a lot of ghosting.
The most important thing for a good upconversion is good quality SD to start with. If it is nice and sharp with low noise, few compression artifacts, and is in its native aspect ratio then you will be happy with the results. If your source is already a little soft, noisy, or letterboxed you will be disappointed. As I covered in a previous post, letterboxing will kill you if you want to upconvert. You will really miss that third of resolution.
If you go to your favorite post house and tell them you want an upconversion they will happily make one for you for around $400 an hour at the very least. That usually does not include the actual HD recording, which may run you another $600 an hour. You can probably do it for under $1000 an hour, but for a lot us that is just out of the question. There are cheaper ways to do it if you are willing to put some work in. If you don’t need an HD tape, than you can do it for $0 assuming you already have a recent copy of After Effects or comparable compositing app.
What on earth are you going to do with a file only upconversion? You can play it back from a computer connected to a HD TV or projector, or you can make an HD-DVD or Blu-ray disc.
Lets start with what you need to make an HD file from your SD video. First thing you need is to get it on your computer. I’m assuming that has already happened for editing purposes. Final Cut Pro and Adobe Premiere can make an upconversion for you, but not as well as After Effects, so you need to export your project from your editing app to one uncompressed file. Do not export a DV file, export a 10 or 8 bit uncompressed mov or avi. If you have the disk space choose 10-bit, it will look slightly better.
I should mention before going on that if you have created any titles or effects it would be best if you turned all of those off before exporting your video. On upconversion the text will become very soft and aliased, and any effects will become soft and less accurate. After the upconversion you can add the titles and effects to the upconverted HD version with much higher quality results.
After you’ve made your huge video file fire up After Effects and import it into your new project. You are only going to be working with this one file, so just drag in onto the create new composition button at the bottom of the project window, it looks like a tiny pixelized film frame. This will create a composition matching your file’s length and format. This next step requires that you have After Effects Pro. Under the File menu choose Project Settings, and under Color Settings choose 16 bits per channel, or if you are really anal choose 32. This setting change saves your color information from being wrecked during the upconversion. If left in 8 bit your colors and contrast will be really degraded. It is important to make sure not to use any effects in this project that do not support 16 or 32 bit processing, as it will make your colors and contrast really wonky. You can tell that an effect does not support a higher bit-depth when it shows a small little triangle next to its name in the effects window.
Ok now we are ready to make the conversion. So we need to decide what to upconvert it to. This depends on what you want to do with it. Lets look at some possible targets.
- HD playback from a computer. In this scenario you do not have to worry about sticking to actual video standards. You can make it whatever crazy resolution or frame rate you want. I would suggest converting it close to the native resolution of the displays you are aiming for. Are these 720p or 1080p? I know that there are some 1080i displays out there, but you are going to have a hard time making a 1080i encode, and only the CRT HD displays can even display true 1080i anyway.
- HD-DVD or Blu-ray. These formats are supposed to support 720p, but I don’t really trust them, they support 1080 better. So 1080p is best.
So you’ve decided whether to go to 720p or 1080p. Now which frame rate? You should always stick with the native frame rate if possible. Hopefully your footage is already progressive, but if it isn’t you’ll have to deinterlace it, which I will cover in my next post. 29.97 should stay 29.97, 25 should stay 25, and 23.98 should stay 23.98. If your 23.98 is stuck in 59.94i footage you need to inverse telecine it out, which I will cover in a future post.
The next step is to change the Composition Settings in the Composition menu to the desired output format. The Presets drop down box covers most of your settings except for 1080 23.98p. There is a 24p, but unless you know that you want 24p, you want 23.976 (to be exact). And watch out for the HDV settings unless your final output is to HDV, and I would recommend against it if at all possible, because they have strange resolutions and pixel aspect ratios. You want to make sure that your 720p is 1280 x 720, that your 1080p is 1920 x 1080, and that the pixel aspect ratio for either is set to square. Computers and HD displays expect square pixels, so if you give them something else the picture could look squished or stretched.
We are almost there. Now you should have a 720p or 1080p sized composition with your SD video sitting sadly in the middle with black all around it. The last step is to resize the SD video. There are a few options depending on your aspect ratio. If your video is 4:3 then you should click on the video image, hold down the shift key to maintain the square aspect ratio and drag out until the top and bottom of the image fill out the 720p 0r 1080p frame. There should be black bars on the left and right. If your video is 16:9 or some wider aspect ratio in a 16:9 matted frame click on the video image do not hold shift and drag the image until it completely fills the 720p or 1080p frame without cropping anything.
You could be done right now, and render out your upconversion. There is an optional step however. Your upconverted material is going to look soft, you can never recover the resolution difference, but you can fake it. This is called sharpening. All it really does is find the edges of things and increases the contrast there. It sounds a little dumb, but it does actually look like it is sharper. I recommend using the Unsharp Mask in the Effects menu. Unsharp sounds like its the opposite of what you want, but it is the one you want. To learn more about the history of the Unsharp Mask I recommend this Wikipedia article. I usually set Amount somewhere around 35, and leave Radius and Threshold at their defaults. You want to try not to sharpen it too much or it will look really fake. Using this effect will increase your render time.
After you have have decided whether to sharpen or not you are ready to render out your file. Ideally you will want to render out an uncompressed HD file, add your titles and effects again and then encode to whatever playback file you will need, be it WM9, h.264, Blu-ray, etc. You will need a good amount of disk space though. An 8-bit uncompressed 1080 24p file will take around 500GB for a 90 minute feature, 10-bit will be around 660GB, but none of the above encoding formats support 10-bit so there really isn’t much point in creating a 10-bit file unless you are going to output to a HDCAM SR or D5.
To render out your nice upconversion choose Add to Render Queue in the Composition menu, and the render queue tab will be selected with your composition in it. In the Render Settings you want to make sure that Quality is at Best, Resolution is at Full, and that your frame rate is correct. In the Output Module you want to select your uncompressed HD mov, avi, or image sequence. And finally under Output To you want to select the directory where you want to save your huge file. The last thing you have to do is hit the Render button and then wait.
On my dual dual-core Opteron PC it takes around 8 hours per 90 minutes to render with Gridiron Nucleo installed. Nucleo is a great plugin if you have multiple cores and/or processors. I can’t recommend it enough. Check it out at http://www.gridironsoftware.com/. I have heard that the new After Effects CS3 has better multi-processor support, so Nucleo may not continue to be amazing, but for the time being it is. Regardless of how fast your machine is, just make sure you have enough disk space and enough time to render and go relax, have a sandwich, go to bed, and let your computer do all of the heavy lifting.