There are 3 HD tape formats that are generally used for mastering, HDCAM, D5, and HDCAM SR. The two HDCAM formats are also used as capture formats for a variety of cameras, most often for Sony’s CineAlta line of HD cameras. Each format is significantly different and good for different things. Each of these formats use differing degrees of image compression to fit the uncompressed HD video signal onto the tape, and because of this each successive copy loses a generation of quality. There are unfortunately no uncompressed tape formats. I have a post that lists all of the relevant specs for the most common tape formats, including these three here.
HDCAM (7.1:1 compression ratio): HDCAM is the cheapest and lowest quality of the three mastering HD tape formats. It has the lowest data rate of the three at 144 Mbps, and because of that has to reduce the image quality in 2 important ways, by reducing the resolution to 1440×1080 and reducing the bit-depth to 8-bit. HDCAM also supports only 4 audio channels, making it difficult to include 5.1 audio.
Despite being the lowest quality you can still get a great image from HDCAM. If you don’t need to ever process the video again, color correction, effects, film out, etc, than HDCAM is really good enough for a low budget. HDCAM is the film festival tape standard, and PBS’s tape standard. There is also a cheap playback deck available from Sony (JH3). Despite being the cheapest of the 3 tape formats HDCAM is the most reliable. I have never had any playback issues with them. However if you have a 5.1 audio mix I would recommend against mastering to HDCAM. There is a way to cram 5.1 into 2 of the audio channels with Dolby E, but it is a pain to get converted, and many places do not have the necessary decoder hardware.
D5 (4:1 compression ratio): D5 technically is lower quality than HDCAM SR, as it has a lower bit rate 270 Mbps, versus HDCAM SR’s 440 Mbps, but in practice there is very little difference in quality other than HDCAM’s greater possible color sampling. It also has 8 channels of audio, enough for a full 5.1 mix and stereo mix.
D5 has great image quality but has very serious reliability issues. I have processed many D5 tapes, and see brief video hits in 60-70 percent of the tapes. Rewinding and trying again can sometimes recover the missed frames, but any playback problem is really unacceptable. The tapes are fragile, and should be handled carefully and shipped with a lot of packing material. D5 is a good master, but be very careful with it.
HDCAM SR (2.7:1 compression ratio): HDCAM SR is the newest and most expensive tape format, but it also will give you the best quality, and the most flexibility. It has 2 advantages over D5, the aforementioned higher bit rate and greater possible color sampling. HDCAM SR gives you the option of sampling color at 4:2:2 or 4:4:4. Both HDCAM and D5 sample colors at 4:2:2. What that means is that the color resolution is half that of the luminance. Our eyes are less sensitive to color than luminance so in general 4:2:2 is fine, but if you want to process the video at some point, whether color correcting, or shooting out to film, or any color sensitive process the greater color sampling will give you more flexibility and better results. If you want to learn more about color subsampling you can here. And It has 12 channels of audio, more than most people could ever use.
Even though HDCAM SR is new and fancy, it too has reliability issues, though it is better than D5. I see problems in maybe 30 percent of the tapes I receive, but it is more able to recover from problems than D5. I have yet to have a playback error that I was unable to playback correctly after rewinding and trying again a few times.
My recommendation is to use HDCAM SR if you can afford it and record in 4:4:4 if the budget will allow.