本书全面讨论了现代多媒体系统的原理、方法和实践，包括内容创建、压缩、分发和数字版权管理。两位作者有多年的工业经验，在讲述基本理论和方法的基础上，还给出了许多实际的工业实例（包括数字电视、IPTV、移动开发和数字电影院等）。书中还包括最新的多媒体技术和发展。

本书将现代多媒体流水线的不同方面（如内容的创建、压缩）集成到了一起。作者根据自己多年的教学实践和在行业中的工作经验，讨论了设计端到端多媒体流水线时涉及的问题，并给出了丰富的实例，如数字电视、IPTV、移动部署以及数字电影流水线。此外，本书还包括当前多媒体问题的最新内容，包括对MPEG-4的讨论和MPEG-21中的最新进展，从而创建一种能够进行无缝数据交换的框架。
本书基于英文原书进行改编，保留了其中的基本内容，压缩或删除了一些高级内容，更加适合作为国内高校本科生或研究生的多媒体课程教材，同时也适合作为多媒体技术研究人员与开发人员阅读和学习的参考用书。
作者简介
保劳格·哈沃达尔（Parag Havaldar） 于1996年从美国南加州大学获得计算机视觉与图形学博士学位，此后一直在多媒体行业为多媒体流水线（从创作、压缩、发布到数字版权管理）设计/架构软件解决方案。目前，Havaldar博士任索尼图片图像工作室（Sony Pictures Imageworks）的软件总监，他的团队所开发的技术用于在各种电影中创建程式化和逼真的角色动画，如《爱丽丝漫游仙境》、《怪兽屋》、《全民超人汉考克》以及《蜘蛛侠》等。Havaldar博士目前还兼任南加州大学计算机科学系教员，教授研究生的多媒体课程。
拉德·梅迪奥尼（Gérard Medioni） 于1977年在法国国立巴黎高等电信学院获得工程师证书，分别在1980年和1983年从美国南加州大学获得硕士和博士学位，此后一直在南加州大学工作，目前是计算机科学与电子工程系教授、机器人和智能系统研究所（IRIS）副主任。他的研究兴趣广泛，包括计算机视觉和人工智能、模式识别与机器学习、多媒体系统等。Medioni教授已经出版过3本著作，发表过50余篇期刊论文和150余篇会议论文。

计算机\多媒体

本书将现代多媒体流水线的不同方面（如内容的创建、压缩）集成到了一起。作者根据自己多年的教学实践和在行业中的工作经验，讨论了设计端到端多媒体流水线时涉及的问题，并给出了丰富的实例，如数字电视、IPTV、移动部署以及数字电影流水线。此外，本书还包括当前多媒体问题的最新内容，包括对MPEG-4的讨论和MPEG-21中的最新进展，从而创建一种能够进行无缝数据交换的框架。
本书基于英文原书进行改编，保留了其中的基本内容，压缩或删除了一些高级内容，更加适合作为国内高校本科生或研究生的多媒体课程教材，同时也适合作为多媒体技术研究者与开发人员阅读和学习的参考用书。

[美] 保劳格·哈沃达尔（Parag Havaldar）,杰拉德·梅迪奥尼（G·rard Medioni）著：保劳格•哈沃达尔（Parag Havaldar） 于1996年从美国南加州大学获得计算机视觉与图形学博士学位，此后一直在多媒体行业为多媒体流水线（从创作、压缩、发布到数字版权管理）设计/架构软件解决方案。目前，Havaldar博士任索尼图片图像工作室（Sony Pictures Imageworks）的软件总监，他的团队所开发的技术用于在各种电影中创建程式化和逼真的角色动画，如《爱丽丝漫游仙境》、《怪兽屋》、《全民超人汉考克》以及《蜘蛛侠》等。Havaldar博士目前还兼任南加州大学计算机科学系教员，教授研究生的多媒体课程。
拉德• 梅迪奥尼（Gérard Medioni） 于1977年在法国国立巴黎高等电信学院获得工程师证书，分别在1980年和1983年从美国南加州大学获得硕士和博士学位，此后一直在南加州大学工作，目前是计算机科学与电子工程系教授、机器人和智能系统研究所（IRIS）副主任。他的研究兴趣广泛，包括计算机视觉和人工智能、模式识别与机器学习、多媒体系统等。Medioni教授已经出版过3本著作，发表过50余篇期刊论文和150余篇会议论文。

Preface v
C H A P T E R 1
Introduction to Multimedia—Past, Present, and Future 1
1 Multimedia: Historical Perspective 2
2 Multimedia Data and Multimedia Systems 4
2.1　Inherent Qualities of Multimedia Data 4
2.2　Different Media Types Used Today 6
2.3　Classification of Multimedia Systems 8
3 A Multimedia System Today 9
4 The Multimedia Revolution 11
5 A Possible Future 13
6 Map of This Book 14
7 How to Approach the Exercises 15
P A R T 1 Multimedia　Content Creation
C H A P T E R 2
Digital Data Acquisition 17
1 Analog and Digital Signals 18
2 Analog-to-Digital Conversion 19
2.1　Sampling 19
2.2　Quantization 20
2.3　Bit Rate 23
3 Signals and Systems 24
3.1　Linear Time Invariant Systems 25
3.2　Fundamental Results in Linear Time Invariant Systems 25
3.3　Useful Signals 26
3.4　The Fourier Transform 26
4 Sampling Theorem and Aliasing 28
4.1　Aliasing in Spatial Domains 30
4.2　Aliasing in the Temporal Domain 30
4.3　Moiré Patterns and Aliasing 30
5 Filtering 33
5.1　Digital Filters 33
5.2　Filtering in 1D 35
5.3　Filtering in 2D 35
5.4　Subsampling 38
6 Fourier Theory 39
7 Exercises 44
Programming Assignments 47
C H A P T E R 3
Media Representation and Media Formats 51
1 Digital Images 51
1.1　Digital Representation of Images 52
1.2　Aspect Ratios 55
1.3　Digital Image Formats 55
2 Digital Video 60
2.1　Representation of Digital Video 60
2.2　Analog Video and Television 61
2.3　Types of Video Signals 64
2.4　YUV Subsampling Schemes 65
2.5　Digital Video Formats 67
3 Digital Audio 69
3.1　Digital Representation of Audio 69
3.2　Surround Sound 70
3.3　Spatial Audio 71
3.4　Commonly Used Audio Formats 72
4 Graphics 73
5 Exercises 77
Programming Assignments 80
C H A P T E R 4
Color Theory 81
1 The Color Problem 81
1.1　History of Color and Light 82
1.2　Human Color Sensing 84
1.3　Human Color Perception 85
2 Trichromacity Theory 86
2.1　Cone Response 87
2.2　The Tristimulus Vector 88
3 Color Calibration 90
3.1　Color Cameras 90
3.2　Rendering Devices 92
3.3　The Calibration Process 93
3.4　CIE Standard and Color-Matching Functions 94
4 Color Spaces 95
4.1　The CIE XYZ Color Space 96
4.2　RGB Color Space 97
4.3　CMY or CMYK Color Space 98
4.4　YUV Color Space 99
4.5　HSV Color Space 101
4.6　Uniform Color Spaces 102
4.7　Device Dependence of Color Spaces 103
5 Gamma Correction and Monitor Calibration 104
6 Exercises 105
Programming Assignments 108
C H A P T E R 5
Multimedia Authoring 111
1 Examples of Multimedia 112
2 Requirements for Multimedia Authoring Tools 117
3 Intramedia Processing 118
3.1　Intramedia Issues Related to Images 119
3.2　Intramedia Issues Related to Video 119
3.3　Intramedia Issues Related to Audio 122
3.4　Intramedia Issues Related to 2D/3D Graphics 122
4 Intermedia Processing 124
4.1　Spatial Placement Control 125
4.2　Temporal Control 126
4.3　Interactivity Setup 127
5 Multimedia Authoring Paradigms and User Interfaces 127
5.1　Timeline 128
5.2　Scripting 129
5.3　Flow Control 131
5.4　Cards 131
6 Role of User Interfaces 132
6.1　User Interfaces on Mobile Devices 132
6.2　Multiple Devices as User Interfaces 133
7 Device-Independent Content Authoring 134
8 Distributed Authoring and Versioning 136
9 Multimedia Services and Content Management 137
10 Asset Management 138
11 Exercises 139
Programming Assignments 141
C O L O R I N S E R T
P A R T 2 Multimedia Compression
C H A P T E R 6
Overview of Compression 145
1 The Need for Compression 146
2 Basics of Information Theory 147
2.1　Information Theory Definitions 148
2.2　Information Representation 151
2.3　Entropy 151
2.4　Efficiency 153
3 A Taxonomy of Compression 154
3.1　Compression Metrics 155
3.2　Rate Distortion 155
4 Lossless Compression 156
4.1　Run Length Encoding 157
4.2　Repetition Suppression 157
4.3　Pattern Substitution 158
4.4　Huffman Coding 160
4.5　Arithmetic Coding 161
5 Lossy Compression 164
5.1　Differential PCM 165
5.2　Vector Quantization 166
5.3　Transform Coding 169
5.4　Subband Coding 172
5.5　Hybrid Compression Techniques 173
6 Practical Issues Related to Compression Systems 175
6.1　Encoder Speed and Complexity 175
6.2　Rate Control 176
6.3　Symmetric and Asymmetric Compression 176
6.4　Adaptive and Nonadaptive Compression 177
7 Exercises 177
Programming Assignments 184
C H A P T E R 7
Media Compression: Images 187
1 Redundancy and Relevancy of Image Data 189
2 Classes of Image Compression Techniques 190
3 Lossless Image Coding 191
3.1　Image Coding Based on Run Length 192
3.2　Dictionary-Based Image Coding (GIF, PNG) 192
3.3　Prediction-Based Coding 192
4 Transform Image Coding 193
4.1　DCT Image Coding and the JPEG Standard 194
4.2　JPEG Bit Stream 198
4.3　Drawbacks of JPEG 200
5 Wavelet Based Coding (JPEG 2000) 201
5.1　The Preprocessing Step 202
5.2　The Discrete Wavelet Transform 203
5.3　JPEG 2000 Versus JPEG 205
6 Fractal Image Coding 207
6.1　Fractals 208
6.2　Fractal Block Coding 209
6.3　The Future of Fractal Image Compression 210
7 Transmission Issues in Compressed Images 210
7.1　Progressive Transmission Using DCTs in JPEG 211
7.2　Progressive Transmission Using Wavelets in JPEG 2000 213
8 The Discrete Cosine Transform 213
9 Exercises 216
Programming Assignments 221
C H A P T E R 8
Media Compression: Video 223
1 General Theory of Video Compression 224
1.1　Temporal Redundancy 227
1.2　Block-Based Frame Prediction 228
1.3　Computing Motion Vectors 231
1.4　Size of Macroblocks 233
1.5　Open Loop versus Closed Loop Motion Compensation 235
2 Types of Predictions 236
2.1　I Frames 237
2.2　P Frames 238
2.3　B Frames 238
2.4　Multiframe Prediction 240
2.5　Video Structure—Group of Pictures 242
3 Complexity of Motion Compensation 243
3.1　Sequential or Brute Force Search 244
3.2　Logarithmic Search 245
3.3　Hierarchical Search 246
4 Video-Coding Standards 247
4.1　H.261 248
4.2　H.263 248
4.3　MPEG-1 248
4.4　MPEG-2 249
4.5　MPEG-4—VOP and Object Base Coding, SP and ASP 251
4.6　H.264 or MPEG-4—AVC 252
5 VBR Encoding, CBR Encoding, and Rate Control 254
6 A Commercial Encoder 256
7 Exercises 258
Programming Assignments 265
C H A P T E R 9
Media Compression: Audio 269
1 The Need for Audio Compression 270
2 Audio-Compression Theory 271
3 Audio as a Waveform 273
3.1　DPCM and Entropy Coding 273
3.2　Delta Modulation 274
3.3　ADPCM 275
3.4　Logarithmic Quantization Scales—A-law and law 275
4 Audio Compression Using Psychoacoustics 276
4.1　Anatomy of the Ear 277
4.2　Frequency Domain Limits 277
4.3　Time Domain Limits 278
4.4　Masking or Hiding 278
4.5　Perceptual Encoder 281
5 Model-Based Audio Compression 283
6 Audio Compression Using Event Lists 285
6.1　Structured Representations and Synthesis Methodologies 286
6.2　Advantage of Structured Audio 287
7 Audio Coding Standards 287
7.1　MPEG-1 288
7.2　MPEG-2 291
7.3　Dolby AC-2 and AC-3 292
7.4　MPEG-4 294
7.5　ITU G.711 294
7.6　ITU G.722 295
7.7　ITU G.721, ITU G.726, and ITU G.727 295
7.8　ITU G.723 and ITU G.729 295
7.9　ITU G.728 295
7.10　MIDI 296
8 Exercises 297
Programming Assignments 300
C H A P T E R 10
Media Compression: Graphics 301
1 The Need for Graphics Compression 303
2 2D Graphics Objects 305
2.1　Points 305
2.2　Regions 305
2.3　Curves 305
3 3D Graphics Objects 306
3.1　Polygonal Descriptions 307
3.2　Patch-Based Descriptions 308
3.3　Constructive Solid Geometry 308
4 Graphics Compression in Relation to Other Media Compression　309
5 Mesh Compression Using Connectivity Encoding 311
5.1　Triangle Runs 312
5.2　Topological Surgery (TS) Compression Algorithm 313
5.3　Analysis of Topological Surgery 314
6 Mesh Compression Using Polyhedral Simplification 316
6.1　Progressive Meshes 317
6.2　Analysis of Progressive Meshes 319
7 Multiresolution Techniques—Wavelet-Based Encoding 320
8 Progressive Encoding and Level of Detail 320
9 3D Graphics Compression Standards 322
9.1　VRML 322
9.2　X3D 323
9.3　MPEG-4 325
9.4　Java 3D 326
10 Exercises 328
Programming Assignments 331
P A R T 3 Recent Trends in Multimedia
C H A P T E R 11
MPEG-4 333
1 General Features and Scope of MPEG-4 334
1.1　MPEG-4 in Relation to MPEG-2 and MPEG-1 337
1.2　MPEG-4 Sample Scenarios 338
1.3　Representational Features 340
1.4　Compositional Features of MPEG-4 340
1.5　Multiplexing and Synchronization Features of MPEG-4 341
2 Systems Layer 342
3 Audiovisual Objects 343
3.1　Representation of Scenes and Interactivity Setup Using AVOs 343
3.2　Encoding of AVOs 346
3.3　Synchronization and Delivery of AVO Streams 348
4 Audio Objects 350
4.1　Natural Sound 350
4.2　Synthetic Sound 352
5 Visual Objects 354
5.1　Natural 2D Video 355
5.2　Synthetic Video Objects 361
6 Synchronization and Transport in MPEG-4 363
6.1　MPEG-4 Transport over MPEG-2 TS 364
6.2　MPEG-4 Transport over the Internet 365
7 Applications Currently Using MPEG-4 365
7.1　Television Broadcasting 366
7.2　IP-Based Television Distribution 366
7.3　Mobile Communication and Entertainment 366
8 Exercises 366
C H A P T E R 12
Multimedia Databases and Querying 371
1 Multimedia Data Versus Multimedia Content 373
1.1　Semantic Extraction 373
1.2　Query Processing 374
1.3　Nature of Multimedia 375
2 Multimedia Metadata 375
2.1　Creation/Extraction of Metadata 376
2.2　Storage of Metadata 377
2.3　Metadata Management 377
3 Multimedia Systems and Databases 377
4 Standards for Metadata 380
4.1　MXF and Descriptive Metadata Scheme-1 (DMS-1) 380
4.2　TV-Anytime 382
4.3　MPEG-7 383
4.4　Dublin Core 386
4.5　IPTC Standards 386
5 User Interface and Browsing Paradigms 388
5.1 Presentation of Semantics 388
5.2　Organization of Results 388
6 Examples of Media Database Projects 389
6.1　The Early 1990s 389
6.2　Turn of the 2000 Century 391
6.3　Current Status 391
7 Exercises 392
C H A P T E R 13
Multimedia Frameworks 395
1 The Need for a Unified Framework 395
2 MPEG-21 Objectives 398
3 Digital Items 400
3.1　Digital Item Declaration (DID) 402
4 Digital Item Identification (DII) 405
5 Digital Item Adaptation 406
5.1　Adapting to the Terminal’s Capabilities 406
5.2　Adapting to Network Characteristics 408
6 Digital Item Processing 408
7 Digital Rights Management in Digital Items 410
7.1　Rights Expression Language (REL) 411
7.2　Rights Data Dictionary (RDD) 413
8 Exercises 414
C H A P T E R 14
Concluding Comments and Future Perspectives 417
1 What Has Been Discussed in This Book 418
2 What Has Not Been Covered 419
3 Current Advances and Impacts 420
3.1　Impact on Information Organization 420
3.2　Impact on Content Delivery 422
3.3　Impact on Service Providers 423
3.4　Impact on Source of News 423
4 Future Trends 424
4.1　Need for Content Adaptation 424
4.2　Intelligent Interfaces and Semantic Processing 424
4.3　Generating Personalized Content 425
4.4　Information Overload and Filtering 425
4.5　User Connectivity, Digital Communities, and Beyond 426
Answers to Selected Numerical Problems 429