音频格式

我们通常说的音频格式准确地讲应该是音频文件格式，它是计算机系统上用于存放数字音频数据的文件格式，也可以看作一个容器。

音频数据的比特分布我们称为音频编码格式，它可以非压缩编码或压缩编码。压缩编码又分为无损压缩和有损压缩。

编码器(codec)就是来编解码原始音频数据的。

声音源 --ADC--> raw audio data --codec--> audio data(uncompressed/compressed) --packed--> audio file format(container format)

An audio file format is a file format for storing digital audio data on a computer system. The bit layout of the audio data (excluding metadata) is called the audio coding format and can be uncompressed, or compressed to reduce the file size, often using lossy compression. The data can be a raw bitstream in an audio coding format, but it is usually embedded in a container format or an audio data format with defined storage layer.

It is important to distinguish between the audio coding format, the container containing the raw audio data, and an audio codec. A codec performs the encoding and decoding of the raw audio data while this encoded data is (usually) stored in a container file. Although most audio file formats support only one type of audio coding data (created with an audio coder), a multimedia container format (as Matroska or AVI) may support multiple types of audio and video data.

There are three major groups of audio file formats:

• Uncompressed audio formats, such as WAV, AIFF, AU or raw header-less PCM;

• Formats with lossless compression, such as FLAC, Monkey’s Audio (filename extension .ape), WavPack (filename extension .wv), TTA, ATRAC Advanced Lossless, ALAC (filename extension .m4a), MPEG-4 SLS, MPEG-4 ALS, MPEG-4 DST, Windows Media Audio Lossless (WMA Lossless), and Shorten (SHN).

• Formats with lossy compression, such as Opus, MP3, Vorbis, Musepack, AAC, ATRAC and Windows Media Audio Lossy (WMA lossy).

视频格式

视频文件格式是计算机系统上一种用来存放数字视频数据的文件格式。视频几乎都是以压缩格式的形式存储的以便减小文件大小。

视频文件格式也是一个容器，里面包含编码完的视频和音频数据，同样是使用编码器来完成编解码工作。

A video file format is a type of file format for storing digital video data on a computer system. Video is almost always stored in compressed form to reduce the file size.

A video file normally consists of a container format (e.g. Matroska) containing video data in a video coding format (e.g. VP9) alongside audio data in an audio coding format (e.g. Opus). The container format can also contain synchronization information, subtitles, and metadata such as title. A standardized (or in some cases de facto standard) video file type such as .webm is a profilespecified by a restriction on which container format and which video and audio compression formats are allowed.

The coded video and audio inside a video file container (i.e. not headers, footers and metadata) is called the essence. A program (or hardware) which can decode video or audio is called a codec; playing or encoding a video file will sometimes require the user to install a codec library corresponding to the type of video and audio coding used in the file.

iOS and Android supported audio & video codec formats

iOS

在 iOS Technology Overview 中 Apple 介绍说，

iOS supports many industry-standard and Apple-specific audio formats, including the following:

AAC

Apple Lossless (ALAC)

A-law

IMA/ADPCM (IMA4)

Linear PCM

µ-law

DVI/Intel IMA ADPCM

Microsoft GSM 6.10

AES3-2003

Preferred Audio Formats in iOS

For uncompressed (highest quality) audio, use 16-bit, little endian, linear PCM audio data packaged in a CAF file.

For compressed audio when playing one sound at a time, and when you don’t need to play audio simultaneously with the iPod application, use the AAC format packaged in a CAF or m4a file.

For less memory usage when you need to play multiple sounds simultaneously, use IMA4 (IMA/ADPCM) compression. This reduces file size but entails minimal CPU impact during decompression. As with linear PCM data, package IMA4 data in a CAF file.

iOS supports many industry-standard video formats and compression standards, including the following:

H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second, Low-Complexity version of the H.264 Baseline Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats

H.264 video, up to 768 Kbps, 320 by 240 pixels, 30 frames per second, Baseline Profile up to Level 1.3 with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats

MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats

Numerous audio formats, including the ones listed in Audio Technologies

在 Multimedia Programming Guide 的 Using Audio 中对音频格式相应介绍如下，

Audio playback formats and codecs

Audio decoder/playback format

Hardware-assisted decoding

Software-based decoding

AAC (MPEG-4 Advanced Audio Coding)

Yes

Yes, starting in iOS 3.0

ALAC (Apple Lossless)

Yes

Yes, starting in iOS 3.0

HE-AAC (MPEG-4 High Efficiency AAC)

Yes

-

iLBC (internet Low Bitrate Codec, another format for speech)

-

Yes

IMA4 (IMA/ADPCM)

-

Yes

Linear PCM (uncompressed, linear pulse-code modulation)

-

Yes

MP3 (MPEG-1 audio layer 3)

Yes

Yes, starting in iOS 3.0

µ-law and a-law

-

Yes

Audio recording formats and codecs

Audio encoder/recording format

Hardware-assisted encoding

Software-based encoding

AAC (MPEG-4 Advanced Audio Coding)

Yes, starting in iOS 3.1 for iPhone 3GS and iPod touch (2nd generation) Yes, starting in iOS 3.2 for iPad

Yes, starting in iOS 4.0 for iPhone 3GS and iPod touch (2nd generation)

ALAC (Apple Lossless)

-

Yes

iLBC (internet Low Bitrate Codec, for speech)

-

Yes

IMA4 (IMA/ADPCM)

-

Yes

Linear PCM (uncompressed, linear pulse-code modulation)

-

Yes

µ-law and a-law

-

Yes

我们将上面的信息综合，可以得到下表：

Format / Codec

Encoder

Decoder

File Extension

AAC (MPEG-4 Advanced Audio Coding)

Yes

Yes

MPEG/3GPP container

.m4a, .mp4, .3gp

Apple container

.m4a, .m4b, .m4p, .m4r, .m4v

ADTS stream

.aac

ALAC (Apple Lossless)

Yes

Yes

.m4a .caf

HE-AAC (MPEG-4 High Efficiency AAC)

No

Yes

Same as AAC

iLBC (internet Low Bitrate Codec, another format for speech)

Yes

Yes

.lbc

IMA4 (IMA/ADPCM)

Yes

Yes

Linear PCM (uncompressed, linear pulse-code modulation)

Yes

Yes

.L16, .WAV, .AIFF, .AU, .PCM

MP3 (MPEG-1 audio layer 3)

No

Yes

.mp3

µ-law and a-law

Yes

Yes

Same as Linear PCM

Android

Audio

Format / Codec

Encoder

Decoder

File Extension

AAC LC

Yes

Yes

MPEG/3GPP container

.m4a, .mp4, .3gp

Apple container

.m4a, .m4b, .m4p, .m4r, .m4v

ADTS stream

.aac

HE-AACv1 (AAC+)

Yes(Android 4.1+)

Yes

HE-AACv2 (enhanced AAC+)

No

Yes

AAC ELD (enhanced low delay AAC)

Yes(Android 4.1+)

Yes(Android 4.1+)

AMR-NB

Yes

Yes

.3gp

AMR-WB

Yes

Yes

.3gp

FLAC

Yes(Android 4.1+)

Yes(Android 3.1+)

.flac

GSM

No

Yes

.gsm

MIDI

No

Yes

Type 0 and 1 (.mid, .xmf, .mxmf)

RTTTL/RTX (.rtttl, .rtx)

OTA (.ota)

iMelody (.imy)

MP3

No

Yes

.mp3

Opus

No

Yes(Android 5.0+)

.mkv

PCM/WAVE

Yes(Android 4.1+)

Yes

.wav

Vorbis

No

Yes

Ogg (.ogg)

Matroska (.mkv, Android 4.0+)

Video

Format / Codec

Encoder

Decoder

File Extension

H.263

Yes

Yes

3GPP (.3gp)

MPEG-4 (.mp4)

H.264 AVC Baseline Profile (BP)

Yes(Android 3.0+)

Yes

3GPP (.3gp)

MPEG-4 (.mp4)

MPEG-TS (.ts, AAC audio only, not seekable, Android 3.0+)

H.264 AVC Main Profile (MP)

Yes(Android 6.0+)

Yes

H.265 HEVC

No

Yes(Android 5.0+)

MPEG-4 (.mp4)

MPEG-4 SP

No

Yes

3GPP (.3gp)

VP8

Yes(Android 4.3+)

Yes(Android 2.3.3+)

WebM (.webm)

Matroska (.mkv, Android 4.0+)

VP9

No

Yes(Android 4.4+)

WebM (.webm)

Matroska (.mkv, Android 4.0+)

iOS 和 Android 都支持的音频、视频格式

Audio

AAC LC (Low-Complexity profile)

Linear PCM

MP3(Decode only)

Video

H.264 AVC

MPEG-4 SP (Simple Profile)

Profile

To address various applications ranging from low-quality, low-resolution surveillance cameras to high definition TV broadcasting and DVDs, many video standards group features into profiles and levels. MPEG-4 Part 2 has approximately 21 profiles, including profiles called Simple, Advanced Simple, Main, Core, Advanced Coding Efficiency, Advanced Real Time Simple, etc. The most commonly deployed profiles are Advanced Simple and Simple, which is a subset of Advanced Simple.

Reference: