These formats are known as the 18 "ATSC Table 3" formats (see Glossary). Each frame rate has a 1000/1001 frequency change to accommodate NTSC color (24=23.98, 30=29.97, 60=59.94), for a total of 36 formats.
The Future
Whether you believe HDTV or DTV is coming, going or staying where it is, the bottom line is that the future is Electronic Widescreen High Resolution TV/Computer/Movie Theater. Period. That is evolutionary, not revolutionary. As a species, we have evolved from primitive finger painted drawings in our caves to primitive low resolution moving images (television) in our homes and we will not stop there. Humankind has always wanted bigger, brighter, sharper. (How else can you explain the clothing styles of the 1970's?) This change will affect every one of us, but initially it will have the biggest impact on the broadcasters. And you don't have to take my word for it. The government has told the broadcasters they have to switch from analog to digital television. The clock is ticking.
HDTV is not television as we know it, it is not an NTSC camera shooting in 4:3, low resolution information, nor is it a 2:1 film camera. High definition is another animal, another tool. To say one is better or worse than the other is like saying a Picasso is better than a Henry Moore. It is just different. It has the advantages of film in that it has the wideness in shape and the ability to show incredible contrast and detail. It also has an advantage over NTSC, not only because of the improved resolution, but because you can do something more than the continual 4:3 TV close up when framing shots. Finally, the creatives can frame a much more interesting shot, off centered and allow a shot to be loose enough to give a sense of place.
HDTV/DTV conversion, or at least a statistically-significant, financially-viable deployment of the technology, is the largest marketing challenge in the history of consumer electronics--and whatever happens, it will rock the very economic foundation of the broadcasting industry as we know it today. Unfortunately, HDTV, ATV and now DTV have been regarded as a science project by industry executives. Only now are we beginning to see these broadcast executives react to the business-model issues of HDTV/DTV.
Regarding that business model, one of the key issues surrounding it is the idea of producing in a higher-resolution widescreen (16:9) format and downconverting the center (4:3) section of that image to NTSC in order to provide the feed for the existing (revenue-generating) NTSC channel. I have been producing HDTV since 1986, and for various creative reasons, it is likely that this process will limit the quality of programming in both formats. Again, this issue has been regarded as part of a science project: scalability, interlace-verses-progressive scan, quality of vertical filtering, downconversion, pan-and-scan, etc. But the real issue is: Can you produce compelling programs that are tailored to the strengths and weaknesses of each format through the use of this process? I believe the answer to this question is no.
That being the case, the slow adoption rate of DTV predicted by the consumer electronics industry means two things: First, that the integrity of your NTSC programming must be retained to protect your position in the NTSC marketplace, your primary source of revenue for the next 10 to 20 years, and; second, you must provide high-quality widescreen HDTV programming tailored to the format so that consumers actually perceive a quality difference when evaluating their purchases. In essence, this means that in most cases, you must produce the two streams separately.
News Programs
Take the case of local news. The most obvious problem here is creation of cross-standard graphics elements. While each station generally invents its own style for presenting these graphics, it is unlikely that a design that works in 4:3 will be optimum when presented in 16:9, and if 16:9-optimized graphics are converted to NTSC-cropped-4:3, any information in the side panels will be lost. And this doesn't even touch on the graphics composition issues created by the wider viewing angle, bigger screen size and closer viewing distance, which are the main differences between true HDTV and widescreen NTSC/SDTV. Much has been said about upconverting the existing news programs in the HDTV world, but once the viewer watches one station's news in HDTV, will they watch the competing station in upconverted NTSC?
Framing for Sports or How I learned to Pan and Scan Live
Over the past 10 years, I have noticed that the aspect ratio of HDTV fits perfectly with many sports. The shape of the playing fields of football, basketball, hockey, and soccer to name a few, is rectangular in design. When soccer is shot in HDTV, you can see more of the field and watch how the team positions itself to attack the opposition. In football you can see both the defensive back and the offensive back in the same shot without having to be completely wide. Not only is the shape of HDTV almost the same as a skating rink, but with HDTV the viewer may finally find the puck in hockey. These sports are a natural for the 16:9 aspect ratio, but there are problems that come with the crossover that will happen during the transition period.
Imagine a basketball game shot in HDTV and downconverted live, edge cropped, because no viewer, let alone no network, would allow letterbox sporting events. There is also a limited amount of space for cameras, so many events will not have the luxury of both HD and NTSC cameras and crews. Therefore, simulcasting using the HDTV equipment as source equipment will be the order of the day. In that scenario, where would the director put the net in the frame? Safe action for NTSC leaves the net almost in the middle of the 16:9 frame. Put the net in the edge of the 16:9 frame and it doesn't even appear in the NTSC feed. Live pan and scan seems to be the immediate compromise. This means that there is a new operator in the mobile. His sole job would be to pan with the action, making sure that the framing of the shot remains in the safe action area at all times, giving the best possible framing in both worlds. Difficult but do-able.
However, with every advantage there are also limitations. Think tennis. The primary camera position is behind the tennis player, where the 4:3 frame works perfectly. The 16:9 frame fights that angle, there is too much air on each side of the court from that camera position. Pan and scan from the end camera would be next to impossible. With a few sports, new camera positions will have to be experimented with and we will have to find new and compelling ways of covering certain sporting events.
Editing for Two Worlds
There are many subtle editing issues as we evolve into this new and complex world. Take the case of a comedy program (sitcom) where the editing sets the pace of the show and in large measure determines the comedic timing of a performance. Say you want to edit just after someone enters the shot. Does the edit occur just after the actor enters the 16:9 frame or the 4:3 frame? Which do you compromise? If you edit separately, will the lengths of the scenes be different? Framing and composition for the same show possess new and interesting problems. Picture, if you will, a head to toe shot in 16:9. There is an immediate feel and look to that framing. Now picture the same shot in 4:3. The look completely changes. No editing can change that problem.
Picture a close-up two-shot in 16:9. In NTSC that shot is either a shot that has each face half off the picture or you need to pan and scan the shot in post much like you do when you do film transfers for television release. In a sitcom with bang-bang timing, this would bring new meaning to the words whip pan.
Simulcasted edge--cropped camera moves present an interesting scenario. Pan an actor into the 16:9 frame. As he enters the picture, he starts to talk, while in the 4:3 frame we hear a mysterious voice with no one in the shot. If he waits until he enters the 4:3 frame to speak, the 16:9 audience will think he has forgotten his lines. Zooming in and out would further complicate this already dizzying scenario. Depending on what aspect ratio your television is, the actor may or may not be in the shot at different times during the zoom. Again, what works for one ratio may not work at all in the other.
One possible solution to these problems would be to convert from HDTV (or a lower-resolution 16:9 format) to NTSC using the "letterbox" method. This puts black bars above and below the 16:9 image, or places a single large bar above or below the image. Although this unused area could contain program-related graphics material, experience has shown that the U.S. market is reluctant to accept the smaller images that letterbox provides on a given size display.
For many programs, the network broadcasters and their TV-station partners must maintain two completely separate program streams during the conversion period when legacy-hardware consumers must be protected, or they will have to compromise the creative integrity of the 16:9 program, hardly creating a compelling case for the purchase of new television receivers.
It is not only an issue of who will pay for the transmitters, antennas, towers and distribution infrastructure, but who will pay for the creation of the separate programs, particularly in the early days (the first 10 to 20 years) when the penetration of DTV is projected to be quite limited.
Can There Be Compromises?
A Real World Example: Woodstock '94
There seems to be a common fear in the pay-per-view world that viewers will demand their money back on the grounds of "technical problems" if they see black bands because of letterboxing. This usually leads to a decision to air the live event in "edge crop" where the sides of the wide screen image are discarded in the conversion process. Of course, the high definition users are then concerned that the director will focus his attention on the 4:3 area and not use the widescreen (16:9) advantage. That usually leads to concerns over how the person switching the selectable ISO will cover the event for post production.
A settlement was negotiated that not only quieted the concerns, but allowed the pay-per-view broadcasters to test the market in non-prime time and gave the viewers a way of identifying live and delayed broadcasts. The live broadcasts were converted in a compromise 14:9 aspect ratio which allowed the under scan on most receivers to cover the black border on the top and bottom. The backup converters ran in letterbox mode and the NTSC D2 recordings for Woodstock Overnight were made from these signals. The Overnight replays were made over a stylized background which made the top and bottom area seem to be part of the program, much the same as the sports ticker on CNN Headline News. So the widescreen format was converted from a drawback to an advantage. The promoters seemed happy with this approach, and even agreed not to complain if we had to switch to the letterbox mode in the live broadcast because of equipment failure.
The conventional NTSC mobile unit provided all the available bells and whistles for packaging the program, including digital effects, still store and live graphics. All transmission was standard NTSC and was no way affected by the high definition production. The high definition production was broadcast at a later date in Japan1.
Conclusion
Over the next 10 to 15 years, the creative and technical community will be forced to deal with having to simultaneously transmit the same program using two TV channels, one for programs in the existing analog format and the other in any or all of the new digital formats. There are many difficult issues to be worked out during this period. Some programs will make this transition seamlessly, others will be compromised. But is it impossible to do? No. Simulcasting has a future and upconverting has a limited future. What the real challenge is and what will be truly exciting are the new possibilities--having to re-think existing programs and develop new types of programs. Programs that are best suited to this new format. HDTV and widescreen are enhancing technologies. We are in its infancy, only just beginning to look at the possibilities. Having recently watched a few hours of old black and white television programs, it is easy to see that over the past 30 years we've come a long way. With this exciting new development in Electronic Widescreen High Resolution TV/Computer (television), it is apparent that we still have much to learn--and learn it we must. There is no going back.
Shooting HD: My First time
By Jeff Alred
I was approached to demo a high definition camcorder at the ESPN Summer
X Games in San Diego, California in 1998. Gladly I accepted the offer; finally I could use the "format of the future" I only read about in college some years ago. My experience as an ENG photographer leaves me with limited digital exposure, so as you can imagine the HD format presented many new challenges. Two of the biggest challenges I ran into while using this format were the aspect ratio and clearer resolution.
Aspect Ratio
The HDCAM acted much like a Sony 600 or 700, only the viewfinder showed a 16:9 aspect ratio. I picked up the camera and started shooting without too much guidance. I actually passed the camera off to show many different handheld camera operators, and they used the HD camcorder almost instantly. White balance, audio settings and other manual functions were in the same location, so most of my attention could be focused on shooting with a letterbox look instead of the 4:3 aspect ratio I was so used to.
Once I began shooting with the camera, I found myself trying to maximize the whole frame, especially with the extra thirds the format presented to me. Since my proportions were different with this format, I was looking at the "fringes" of the frame rather than the subject I was interviewing. As I overcame the initial newness of the format, I began to utilize the frame to my advantage. I started to use right or left justification of my subject so the editor could use the other two-thirds of the frame for effect or B-roll. I framed shots with specific items in one-third of the frame and the subject sharing the remainder of the frame. This aspect ratio allows the photographer to marry together one or two products with the subject, something that 4:3 just doesn't allow you to do.
When shooting action, it took some getting used to the skinnier, longer frame. At first, I began tilting up and down too slowly, realizing the action was going through my frame twice as fast as I was used to. I really enjoyed having a longer frame, especially for the speed shots where I could let the athletes enter and exit through the whole frame. It was a good method to relate the speed to the fan sitting at home.
Clearer Resolution
The second major challenge was the clearer resolution presented through the improved scan rate. I was really impressed with the crispness of the picture--even subtle details were clear. We set up an interview where we had to make sure not even a gum wrapper laid on the ground in fear of it being seen by the camera!
When we returned to the edit room to screen the footage, I was amazed every time with how the format translated an image compared to how it looked in person. Most of our interviews were done on location, but this format may cause some headaches for set designers or lighting directors. It really exposes any flaws or patches in set work.
Since this is such a clear and concise image, some shots could tell stories without words. Shots of faces which show the wrinkles of experience, the ripples of muscles during competition--I used them all because the format's resolution is almost truer than life. I also utilized the shutter with this format, and it gave me an image that was almost unexplainable. I thought the format was already impressive, but this function could capture every frame even more crisp than before. I began shooting the majority of my video with shutter, and the reaction from the edit area was equally enthusiastic.
This HD camcorder is very easy to operate. The aspect ratio definitely kept me on my toes, but I found many positive ways to use this format to my advantage once I became comfortable with it. With the clearer resolution and 16:9 aspect ratio, this format will be a favorite of sports producers, because it gives the viewer a fan's perspective, the peripheral vision that has never been previously offered.
Real--World Digital And Production Problems
By Michael Silbergleid
The one good thing that analog and NTSC did for broadcasters and videographers was to hide a multitude of sins. Here's a simple example: If you shoot a black and white glossy photograph, a monitor will show you sharp, highlighted edging. This edging has high frequencies that the camera and monitor can see. But a tape format like VHS doesn't see them at all. So instead of the picture looking as bad as it did on the live monitor, it looks "better" from tape. All the high frequency noise was removed by simply recording on a narrow bandwidth analog tape format. Record that same image in digital (and especially high definition digital), and you'll get all that high frequency "noise" and a worse-looking image.
Another NTSC savior: 30 frames per second interlace. Look at a still frame (not a field) of video with motion not shot using a fast electronic shutter, and you'll see that the object moving is blurry. Even when that object is at rest, the image is not perfect. What this means is, for the most part, that there is not a great difference between the sharpness of the two images. (See figure 1.)
Now do the same test with high definition. What you'll see is a very sharp picture of the object at rest, but blur when it moves. That blur is called motion blur. You can eliminate motion blur by using a high speed electronic shutter, but the video then takes on a strobe-like effect.
Here's a real-world example: During a demonstration of HDTV, blur was noticed in the image. At first it was thought that the focus was slightly off on a scene of a woman walking down a street towards the camera. Although tight focus on the extra-sharp high definition image will be critical, it was not the problem in this case--motion blur was. As the woman moved up and down in step, her image blurred. This blur would not normally be noticeable in NTSC, but in high definition it was.
It was noted that the scene was shot in interlace and we were also watching an interlace image. When the image was shown in progressive, about one-half of the motion blur was removed. But because of the interlace production and shutter speed of 1/60th of a second, the blur was still noticeable.
By now, we all know that we can no longer use fake props and set pieces or an old- looking studio set with HDTV as the clarity gives away all our tricks and illusions.
Older NTSC sets shot in high definition do not hold up well under the sharp scrutiny of the high definition camera. The sets look cheap. Graffiti on the sets "reads" for the first time and backgrounds begin to look awful. Immediately it was realized that new sets would be needed for high definition production, and traditional NTSC set "tricks" would no longer work. Those "tricks" include fake bookcases with painted-in books, slightly off paint jobs on walls, and other things that producers and designers knew would not be reproduced when going through NTSC.
 |
 |
Simulation of motion blur in a high definition image. On the left an image shot without helicopter movement in the frame (could also be with movement in the frame but with a high speed electronic shutter). On the right, a simulation of the same image and quality with movement in the frame and no electronic shutter. Motion blur can make high definition look soft. |
|
You may be shooting for today, but you want to protect for tomorrow's high definition widescreen world.
Today, many productions are shot in 16:9, posted in 16:9 and then center sliced with an aspect ratio converter for the final 4:3 air master. On screen composition is for the 4:3 image, while protecting the 16:9 image.
Keep in mind that shots composed for a 16:9 aspect ratio may look awkward in 4:3 and that if you adjust camera detail for monitoring on an HD monitor, your NTSC picture may look soft. There are always going to be compromises to be made in the world of widescreen and high definition.
16:9 Letterbox
A widescreen 16:9 television does not guarantee the premanant exile of letterbox. 16:9 has an aspect ratio of 1.77:1, but a number of motion pictures are produced in an aspect ratio of 2.35:1. To accomodate this wider-than-16:9 image, the motion picture is presented in a letterbox format.
To avoid what many consider to be the annoying black bars of letterbox, Home Box Office has decided to pan-and-scan all newer 2.35 film to high definition transfers to fit within the 1.77 aspect ratio of their HBO HD service. This means approximately 26 percent of the image of a 2.35 motion picture will be lost at any one time. While purists demand that films be shown in their intended aspect ratio, others (like HBO) believe in filling up the screen.
An interesting exercise is to take a DVD that has both a 4:3 pan-and-scan on one side and a 16:9 letterbox on the other (several films, including Austin Powers: International Man of Mystery, have been released with this feature). Watch a scene in one format then the other and see if you find what is missing in 4:3 or visible in 16:9 to be of major importance.
14:9: The Great Widescreen Compromise
As you've seen before, 14:9 might answer many of the problems of 4:3 versus 16:9. At the International Broadcasting Convention held in Amsterdam, Holland, The Netherlands in 1996, a paper called "The Simultaneous Transmission Of Widescreen And 4:3 Programmes" by M. L. Bell and H. M. Price was presented. Within this paper was the first major look at the 14:9 compromise, as made by the BBC.
Let's start with production and end with distribution: Our old programs (and our current ones) are 4:3. Our new programs will increasingly be 16:9. We can protect for 4:3, or pan-and-scan for 4:3, or do true 16:9 and letterbox for 4:3.
The BBC had that problem: Simultaneous transmission of widescreen and 4:3.
The BBC and all the U.S. networks know that most viewers dislike letterbox. So how do you show 16:9 on a 4:3?
Their answer--14:9 "half-letterbox." There is minor letterboxing and some minor loss of the edges of 16:9, but the BBC found that most viewers were okay with this format.
Now, how do you present 4:3 material on a widescreen 16:9 television set?
As bad as it sounds, and the BBC admits that there are no easy solutions, it is with side curtains. The picture could be expanded slightly to reduce the curtains, but at the expense of the top and bottom of the frame being cropped.
In the years to come, experimentation will help to further answer these 16:9 and 4:3 questions.
Compression
Compression is used to make digital video affordable. And the type of compression we use literally throws bits of data away based on how humans visually perceive information. Look at how a format (and its compression scheme) performs under different conditions (lighting, camera movement, subject movement, color saturation, etc.) and judge for yourself.
There are only two issues that can cause you grief with regard to production and compression: overloading your encoder and concatenation.
All of the encoders that you'll be using are realtime encoders. They must compress your video data at a steady rate regardless of the images you shoot. Problems arise when the visual information is too complex for the encoder to handle. All encoders react differently to the data they are given based on the type of encoding (DV, MPEG, M-JPEG) and the compression ratio they must work at.
A scene too complex for an encoder might be one with a quick pan, zoom or dolly. Or one that has lots of motion in it where the image changes dramatically from pixel to pixel and from frame to frame. When an encoder "chokes," two things can happen. It can either freeze up and re-output the last frame of properly encoded video again or it can produce macroblocks--a compression artifact--making a large block of pixels look all the same (color and brightness). (See figure 2.)
Decoders can also "choke" since they need to work in realtime as well, and suffer from the same symptoms.
Practice with your encoder (your camcorder or deck) will let you see what it is really capable of handling...and most importantly, what it is not capable of handling and what you must avoid.
Concatenation is the other problem--when you use different types of compression on the same piece of video (DV and then MPEG, for example). Each compression scheme handles data differently, so compression artifacts (known as macroblocks because they look like blocks in the picture) and loss of video resolution are more apparent. Even staying within the same type of compression can cause problems, as the same brand and model of encoder may encode the same signal in a different manner.
While concatenation may not show up until generations later (if at all), you might want to know what compression will be used in post production as well as in the duplication master and distribution copies. Limiting the times the video gets re-compressed and the number of different types of compression that the video will be subjected to will only benefit you in the end.
 |
figure 2. Macroblock encoder errors. |
How Different Production Formats Can Be Used
While people, stations, facilities and networks begin to position themselves with their production format or formats of choice, there is some agreement as to how these formats will be used.
Film will continue to be used as the ultimate high definition imaging format. Currently, more than 80 percent of primetime programming (not including news programs) is shot in film. Telecines can output the film image in any number of formats or even as a data file.
While looking at the information below, keep in mind what each format offers as well as the complexities of each type of production. You'll soon realize that all of these are natural fits. Also, the 704x480 format stretches pixels either horizontally for 16:9 or vertically for 4:3, so most people feel that it won't see that much use. It should be noted that equipment is just being introduced for many of these formats, such as the 1080p/24 video format introduced in 1999.
Movies (produced at 24 frames per second)