Video Format Basics

This post is for me to summarize what I learn about shooting, color grading, and editing, and to share it along the way.

I’m starting from zero, but I care a lot about recording the world through a lens. I want to approach shooting and post-production the same way I approached becoming a music producer: take the first step, keep learning and practicing, and keep getting better.

My hardware and software:

• Camera: iPhone 17 Pro Max
  • Apps: Blackmagic Camera / 飓风相机 (MediaStorm’s camera app)
• Stabilizer: DJI Osmo Mobile 8
• Editing machine: Mac Studio
  • Software: DaVinci Resolve / Final Cut Pro

For now I’m focusing on shooting with a phone. Once I can make some decent footage, I’ll consider buying a camera. My main source is MediaStorm’s course “只看8节课，用iPhone拍出电影感” (roughly, “Cinematic iPhone footage in 8 lessons”, in Chinese). For some topics I dig further with generative AI such as Gemini, and I check what it tells me by hand to keep things as accurate as I can. If you spot a mistake, please let me know, and read critically.

If I use any new sources in the future, I’ll list them at the top of this post.

The Basics

Data Types and File Formats

Let’s start with a made-up conversation.

In the editing room, the director says to the colorist:

“The A-cam footage is ProRes RAW and the B-cam footage is 10-bit Log. Apply that LUT to all the Log footage first to convert it from Log to Rec.709, so we can see normal color.”

After the colorist does it, the director frowns: “Too clean. Swap the B-cam footage for the ProRes 422 proxies and let’s start cutting. When the edit is done, put that film-emulation creative LUT on it and let’s see.”

After the edit, the colorist says: “I’ll export the final master as ProRes 4444. For preview, I’ll compress a 50 Mbps H.265 file, plus a separate H.264 version for better compatibility.”

1. What is a LUT?

LUT stands for Look-Up Table. In shooting and post-production, a LUT is essentially a set of instructions, or a preset, for converting colors. It plays two key roles in a video workflow: translating the “raw”, gray Log footage from the camera into standard color that a display understands, and applying a specific color recipe on top of that standard image to give it a cinematic or artistic look.

Think of it as an extremely precise, advanced filter. It defines which output color a particular input color should become. It doesn’t just layer an effect on top; it maps each pixel’s color values mathematically.

There are two main types: 1D LUTs and 3D LUTs. A 1D LUT is simple. It adjusts the red, green, and blue channels independently, so it can’t precisely handle combinations of saturation and hue. A 3D LUT is the most common and most powerful type in video. It maps colors through a three-dimensional color cube and can control hue, saturation, and luminance together, which allows very complex and precise conversions. For example, it can specify “turn every low-saturation green in the frame into a high-saturation cyan while keeping its brightness the same”, which a 1D LUT can’t do.

2. What is Log?

Log is a gamma curve (also called a transfer function) for encoding light. Its purpose is to compress dynamic range. More specifically, it compresses the sensor’s linear distribution of light into a limited data space, so highlights don’t blow out and shadows don’t crush to black, recording as wide a dynamic range as possible. “Log” mainly refers to the gamma curve, but it usually comes together with the brand’s color matrix as a complete picture profile. The logarithmic curve squeezes the highlight range into a limited set of code values, which is what extends the dynamic range. Log footage looks washed out and gray straight out of the camera and needs grading to restore color and contrast and to add a look. Most major camera brands have their own Log format, tuned to their sensors. There’s no shared standard between companies, and the formats aren’t interchangeable.

The one I use is Apple Log. On iPhone it’s only available when recording ProRes (the ProRes 422 family); HEVC doesn’t support Apple Log.

3. What are RAW, ProRes, and H.264/H.265?

RAW is a type of data, but different manufacturers use different RAW formats. Every RAW format is a “digital negative”, but each brand’s negative is made and developed differently. RAW stores the charge readings the sensor samples at each pixel, usually as single-channel Bayer-pattern data. A RAW file has to go through a process called “developing”, or debayering, before it becomes a color image we can see. RAW gives you a lot of room in post: you can adjust exposure gain and white balance freely. But because it’s raw data, the files are huge, and your computer needs a lot of power to “develop” them in real time.

ProRes is a codec for compressing images that have already been formed. The H.264/H.265 files a camera records by default are small but compressed very aggressively (inter-frame compression), so the computer has to decompress them while editing, which makes editing slow and choppy. ProRes uses intra-frame compression: a lower compression ratio and bigger files, but very smooth editing. It exists for editing performance. A ProRes file (4444 or 422) stores an image that has already been developed. The camera’s white balance, ISO, exposure, and Log curve (such as Apple Log or S-Log3) are baked in. You can grade a ProRes (Log) file, but you can’t re-develop it the way you can with RAW, so you can’t change white balance losslessly in post. Note that ProRes RAW is different from regular ProRes: it contains RAW data, not a developed image.

H.264 is short for Advanced Video Coding (AVC); H.265 is High Efficiency Video Coding (HEVC). They’re the most popular and widely used codecs in the world. Their goal is to make video files as small as possible while keeping quality as high as possible, so they’re easy to store and to stream over the internet. H.265’s compression is more advanced: at the same quality (measured by PSNR/VMAF), it saves about 30–50% bitrate compared with H.264.

4. What is Rec.709, and what’s related to it?

Rec.709’s formal name is ITU-R Recommendation BT.709, a unified standard for HD television. It mainly defines a color space (color gamut) and a gamma curve. Rec.709 is one of the common color spaces for SDR. SDR as a concept covers the color space (such as Rec.709), the EOTF (BT.1886), brightness standards, and so on.

HDR, by contrast, is an umbrella term for a set of newer technologies that aim to go beyond SDR and produce images closer to what the eye sees. For example, traditional BT.709 SDR has a reference peak brightness of about 100 nits, while HDR displays can reach 1,000 nits or more. HDR also uses a wider gamut, mainly Rec.2020. All HDR content must be 10-bit or higher (for example, 12-bit), which means HDR can show more, and more saturated, colors, like Ferrari red, Coca-Cola red, or neon green, that Rec.709 can’t reproduce accurately. In practice, many displays can’t fully cover Rec.2020, so the gamut of a typical HDR screen is closer to DCI-P3.

HDR has several main standards, which use different EOTFs. Here are three:

• HDR10: the basic open standard. It lets you set a “maximum brightness” for the whole video.
• Dolby Vision: Dolby’s proprietary standard. It lets you adjust HDR metadata scene by scene, or even frame by frame.
• HLG (Hybrid Log-Gamma): designed for TV broadcasting, such as live sports.

5. How the concepts connect

All of the above show up somewhere between shooting and delivery:

• Capture: shoot with a device that supports Log.
• Edit: transcode the Log footage to ProRes for smooth editing.
  • With Apple Log, the footage is already recorded in ProRes, so there’s no need to transcode.
• Deliver: SDR or HDR.
  • SDR: use a LUT to “squeeze” the 10-bit Log footage, discarding the extra highlight and color information in Log and keeping only what fits in the Rec.709 “box”. Export as an H.264 (8-bit) file that plays on any device.
  • HDR: grade the 10-bit Log footage to HDR10 or Dolby Vision, keeping Log’s highlights and wide color. Export as an H.265 (10-bit) file for viewing on HDR screens.

Brightness Basics

The human eye adapts quickly between very bright and very dark, but a camera can’t. A camera can only hold a limited range of brightness, called its dynamic range.

A larger dynamic range means a wider distance between the brightest detail it can record (without blowing out) and the darkest detail it can record (without going black).

Anything beyond that range shows up as:

• Overexposure / highlight clipping
• Crushed shadows

The brightness a camera records comes from the combination of aperture, shutter speed, and ISO.

• Aperture
  • The size of the opening. Larger lets in more light.
• Shutter speed
  • How long the exposure lasts. Slower lets in more light.
• ISO
  • The sensor’s “gain” on light. Higher is brighter, but noisier.
  • If brightness goes beyond what the camera can hold, it “blows out”.
    • Each pixel on the sensor can only record up to a certain maximum charge.
    • Beyond that → the charge overflows → the pixel becomes pure white → clipping.

Some cameras can warn you before overexposure happens:

• Zebra stripes
  • When an area of the frame goes above a brightness threshold you set, say 95%, stripes appear over it. It’s essentially an overexposure warning.
  • Zebras judge brightness in IRE.
    • IRE is the unit on the video brightness scale (0 = black, 100 = white).

One way to reduce the light entering the lens without changing its color, and so avoid overexposure, is a neutral density (ND) filter.

(To be continued)