Taking another lesson from nature, the latest advances in image processing technology seek to combine image data from several diverse types of sensors in order to obtain a more accurate view of the scene: very much the same as we rely on our five senses. Multi-Sensor Image Fusion and Its Applications is the first text dedicated to the theory and practice of the registration and fusion of image data, covering such approaches as statistical methods, color-related techniques, model-based methods, and visual information display strategies. After a review of state-of-the-art image fusion techniques, the book provides an overview of fusion algorithms and fusion performance evaluation. The following chapters explore recent progress and practical applications of the proposed techniques to solving problems in such areas as medical diagnosis, surveillance and biometric systems, remote sensing, nondestructive evaluation, blurred image restoration, and image quality assessment. Recognized leaders from industry and academia contribute the chapters, reflecting the latest research trends and providing useful algorithms to aid implementation. Supplying a 28-page full-color insert, Multi-Sensor Image Fusion and Its Applications clearly demonstrates the benefits and possibilities of this revolutionary development. It provides a solid knowledge base for applying these cutting-edge techniques to new challenges and creating future advances.
Author(s): Rick S. Blum, Zheng Liu
Series: Signal processing and communications 25
Edition: 1
Publisher: Taylor & Francis
Year: 2006
Language: English
Pages: 535
City: Boca Raton, FL
dk493xfm.pdf......Page 1
Multi-Sensor Image Fusion and Its Applications......Page 4
Preface......Page 6
Editors......Page 10
Contributors......Page 11
Table of contents......Page 14
Dedication......Page 16
CONTENTS......Page 17
Table of Contents......Page 0
I. INTRODUCTION TO IMAGE FUSION......Page 18
B. APPLICATIONS OF IMAGE FUSION......Page 20
A. MULTISCALE-DECOMPOSITION-BASED FUSION METHODS......Page 23
1. Multiscale Decomposition......Page 25
3. Coefficient Grouping Method......Page 32
B. NONMULTISCALE-DECOMPOSITION-BASED METHODS......Page 33
3. Estimation Theory Based Methods......Page 34
5. Artificial Neural Networks......Page 35
A. PERFORMANCE EVALUATION......Page 36
1. Objective Evaluation Measures Requiring a Reference Image......Page 37
2. Objective Evaluation Measures Not Requiring a Reference Image......Page 40
B. IMAGE REGISTRATION......Page 42
REFERENCES......Page 45
I. INTRODUCTION......Page 52
II. MUTUAL INFORMATION AS A GENERIC SIMILARITY MEASURE......Page 55
III. INTERPOLATION INDUCED ARTIFACTS......Page 58
IV. GENERALIZED PARTIAL VOLUME ESTIMATION OF JOINT HISTOGRAM......Page 60
V. OPTIMIZATION......Page 62
A. SIMPLEX SEARCH ALGORITHM......Page 64
B. MULTIRESOLUTION OPTIMIZATION......Page 65
VI. APPLICATION TO 3D BRAIN IMAGE REGISTRATION......Page 66
VII. SUMMARY......Page 67
REFERENCES......Page 70
CONTENTS......Page 72
II. IMAGING MODALITIES......Page 73
A. TRANSFORMATION TYPES......Page 77
a. Correlation Methods......Page 82
b. Sequential Methods......Page 83
2. Fourier Methods......Page 85
3. Feature-Based Methods......Page 87
a. Vessel Segment Ends......Page 90
c. Blood Vessel Bifurcations......Page 91
6. Mutual Information Methods......Page 96
2. Quantitative......Page 98
IV. FUSION......Page 99
A. COMBINATION BY GRAPHICAL SUPERPOSITION......Page 100
c. Multiresolution Methods......Page 102
c. MIT False-Color Method......Page 106
1. Qualitative......Page 110
c. Cross-Entropy......Page 112
d. Image Noise Index......Page 113
e. Spatial Frequency......Page 114
V. CONCLUSION......Page 115
REFERENCES......Page 117
FURTHER READING......Page 120
CONTENTS......Page 122
A. CONTEXT......Page 123
B. MR/US REGISTRATION......Page 124
D. INTENSITY BASED NONRIGID REGISTRATION ALGORITHMS......Page 125
E. OVERVIEW OF THE ARTICLE’S ORGANIZATION......Page 126
2. Correlation Coefficient (CC)......Page 127
B. BIVARIATE CORRELATION RATIO......Page 128
C. PARAMETRIC INTENSITY FIT......Page 129
D. ROBUST INTENSITY DISTANCE......Page 130
A. PARAMETERIZATION OF THE TRANSFORMATION......Page 131
C. MINIMIZING THE SSD FOR A FREE-FORM DEFORMATION......Page 132
E. REGULARIZATION ENERGY......Page 134
F. FROM REGISTRATION TO TRACKING......Page 135
IV. EXPERIMENTS......Page 136
2. Patient Images During Tumor Resection......Page 137
3. A Phantom Study......Page 140
B. MR/US RIGID REGISTRATION CONSISTENCY EVALUATION......Page 145
1. Registration Loops......Page 148
2. Bronze Standard Registration......Page 149
3. Consistency Results......Page 150
C. 3D US TRACKING PERFORMANCES......Page 151
V. DISCUSSION......Page 153
ACKNOWLEDGMENTS......Page 154
REFERENCES......Page 155
CONTENTS......Page 159
I. INTRODUCTION......Page 160
II. BACKGROUND AND PROBLEM STATEMENT......Page 161
A. PROBLEM STATEMENT......Page 163
1. Reconstruction from Projection Data......Page 164
2. Deblurring of Locally Sensed Data......Page 166
B. RELATED WORK IN IMAGE-BASED FUSION......Page 167
1. The Mumford–Shah Variational Approach to Image Processing......Page 168
2. Single Parameter Image Fusion......Page 169
3. Multiparameter Image Fusion......Page 170
III. SHARED BOUNDARY FUSION FORMULATION......Page 172
A. SENSOR OBSERVATION MODEL TERM......Page 174
B. NOISE SUPPRESSION TERM......Page 175
C. ALIGNMENT TERM......Page 176
D. BOUNDARY TERM......Page 177
IV. OPTIMIZATION APPROACH......Page 178
A. SHARED BOUNDARY ESTIMATION......Page 180
B. BOUNDARY AWARE IMAGE FORMATION......Page 181
C. MULTIMODAL ALIGNMENT......Page 183
1. Observation and Inversion Model......Page 187
2. Fusion Results......Page 190
1. Data Acquisition......Page 192
2. Fusion Results......Page 193
REFERENCES......Page 195
CONTENTS......Page 199
I. INTRODUCTION......Page 200
A. RÉNYI ENTROPY AND DIVERGENCE......Page 205
B. MUTUAL INFORMATION AND alpha-MUTUAL INFORMATION......Page 206
D. alpha-GEOMETRIC-ARITHMETIC MEAN DIVERGENCE......Page 209
E. HENZE–PENROSE AFFINITY......Page 210
III. CONTINUOUS QUASIADDITIVE EUCLIDEAN FUNCTIONALS......Page 211
A. A MINIMAL SPANNING TREE FOR ENTROPY ESTIMATION......Page 212
B. NEAREST NEIGHBOR GRAPH ENTROPY ESTIMATOR......Page 217
IV. ENTROPIC GRAPH ESTIMATE OF HENZE–PENROSE AFFINITY......Page 220
V. ENTROPIC GRAPH ESTIMATORS OF alpha-GA AND alpha-MI......Page 221
A. ICA BASIS PROJECTION FEATURES......Page 225
B. MULTIRESOLUTION WAVELET BASIS FEATURES......Page 226
A. REDUCING TIME-MEMORY COMPLEXITY OF THE MST......Page 227
B. REDUCING TIME-MEMORY COMPLEXITY OF THE KNNG......Page 230
VIII. APPLICATIONS: MULTISENSOR SATELLITE IMAGE FUSION......Page 233
A. DEFORMATION AND FEATURE DEFINITION......Page 234
A. DEFORMATION LOCALIZATION......Page 236
B. LOCAL FEATURE MATCHING RESULTS......Page 240
REFERENCES......Page 242
A1. APPENDIX......Page 247
I. INTRODUCTION......Page 278
A. IMAGERY......Page 280
B. FUSION METHODS......Page 281
C. TEST METHODS......Page 283
D. RESULTS......Page 286
E. DISCUSSION......Page 287
A. IMAGERY......Page 289
C. TEST METHODS......Page 293
D. RESULTS......Page 296
1. Perception of Global Structure......Page 297
2. Perception of Detail......Page 298
3. Summary......Page 299
E. DISCUSSION......Page 300
IV. CONCLUSIONS......Page 301
REFERENCES......Page 302
CONTENTS......Page 306
I. INTRODUCTION......Page 307
A. TREE STRUCTURE OF THE WAVELET COEFFICIENTS......Page 308
C. IMAGE FORMATION MODEL......Page 309
III. FUSION WITH THE EM ALGORITHM......Page 311
B. UPDATING PARAMETERS USING THE EM ALGORITHM......Page 313
C. INITIALIZATION OF THE FUSION ALGORITHM......Page 314
A. CWD WITH VISUAL AND MMW IMAGES......Page 316
V. CONCLUSIONS......Page 318
REFERENCES......Page 321
A.1. DERIVATION OF CONDITIONAL PROBABILITIES......Page 323
A.2.1. Upward Step......Page 324
APPENDIX B OUTLINE OF THE DERIVATION OF THE UPDATE EQUATIONS......Page 325
I. INTRODUCTION......Page 329
II. LEVELS OF FUSION......Page 333
III. FUSION SCENARIOS......Page 335
V. INTEGRATION STRATEGIES......Page 336
VI. DESIGN ISSUES......Page 337
VII. SUMMARY AND CONCLUSIONS......Page 338
REFERENCES......Page 339
CONTENTS......Page 342
II. A PRIORI INFORMATION......Page 343
1. Methodology......Page 344
2. Choosing the Clique Topology and the Optimization Algorithm......Page 347
B. RESULTS ON RADARSAT-1 IMAGERY......Page 349
A. HIERARCHICAL CLASSIFIER......Page 353
B. SELECTION OF GLCM FEATURES USING GENETIC ALGORITHMS......Page 354
3. Results......Page 355
IV. RESULTS......Page 356
REFERENCES......Page 359
CONTENTS......Page 361
A. DEFINITION AND IMPORTANCE OF REMOTELY SENSED IMAGE REGISTRATION......Page 362
1. Data Acquisition Issues......Page 364
b. Temporal Changes......Page 365
c. Terrain Relief......Page 366
d. Multisensor Issues......Page 367
b. Lack of Ground Truth......Page 369
A. CHARACTERISTICS OF IMAGE REGISTRATION METHODS FOR REMOTE SENSING......Page 370
1. Intensity, Area-Based Algorithms......Page 372
4. Mutual Information Algorithms......Page 373
A. CORRELATION-BASED EXPERIMENTS......Page 374
C. SIMILARITY MEASURES EXPERIMENTS......Page 375
D. COMBINATION ALGORITHMS EXPERIMENTS......Page 376
2. Multitemporal Dataset......Page 377
3. Multisensor Dataset......Page 378
IV. CONCLUSION AND FUTURE WORK......Page 379
ACKNOWLEDGMENTS......Page 380
REFERENCES......Page 381
I. INTRODUCTION......Page 384
A. LINEAR MINIMUM MEAN SQUARE ERROR FILTER......Page 386
1. LMMSE Filter — System with N Inputs without Degradation......Page 387
2. LMMSE Filter — System with N Inputs Degraded by Additive Noise......Page 390
B. MORPHOLOGICAL PROCESSING APPROACH TO FUSION......Page 396
III. MODEL-BASED DATA FUSION......Page 400
A. Q-TRANSFORM......Page 401
B. DEFINITION AND MAPPING PROPERTY......Page 404
C. NUMERICAL COMPUTATION OF THE Q-TRANSFORM......Page 405
1. Signal Level Data Fusion......Page 407
2. Feature Level Data Fusion......Page 409
REFERENCES......Page 410
I. INTRODUCTION......Page 412
B. NDI TECHNIQUES FOR CORROSION DETECTION......Page 416
C. TEST COMPONENT......Page 417
D. QUANTIFICATION OF NDI RESULTS......Page 420
A. DATA ALIGNMENT AND REGISTRATION......Page 421
B. VERIFICATION AND EVALUATION......Page 422
1. Pixel-Level Fusion......Page 423
2. Classification-Based Approach......Page 425
3. Estimation with a General Additive Model......Page 430
IV. DISCUSSION......Page 435
ACKNOWLEDGMENTS......Page 438
REFERENCES......Page 439
I. INTRODUCTION......Page 442
II. MULTICHANNEL IMAGE ACQUISITION MODELS......Page 444
III. PIECEWISE IDEAL IMAGING......Page 445
A. APPLICATION IN CONFOCAL MICROSCOPY......Page 446
IV. UNIFORMLY BLURRED CHANNELS......Page 448
A. ALTERNATING MINIMIZATION ALGORITHM......Page 450
2. Regularization of the Blurs R(h)......Page 451
3. Iterative Minimization Algorithm......Page 452
B. EXPERIMENT WITH ARTIFICIAL DATA......Page 453
C. EXPERIMENT WITH REAL DATA......Page 454
V. SLIGHTLY MISREGISTERED BLURRED CHANNELS......Page 457
A. MAXIMUM A POSTERIORI PROBABILITY ALGORITHM......Page 458
VI. HEAVILY MISREGISTERED BLURRED CHANNELS......Page 459
VII. CHANNELS WITH SPACE-VARIANT BLURRING......Page 462
VIII. CONCLUSION......Page 464
REFERENCES......Page 465
CONTENTS......Page 468
I. INTRODUCTION......Page 469
II. MULTIMODALITY DISPLAYS......Page 470
III. FOCUS + CONTEXT DISPLAYS......Page 471
V. GAZE-CONTINGENT DISPLAYS......Page 476
VII. GAZE-CONTINGENT MULTIMODALITY DISPLAYS......Page 477
A. TWO-DIMENSIONAL GCMMD FOR IMAGE FUSION......Page 479
2. Surveillance Images......Page 480
3. Remote Sensing Images......Page 481
1. Challenges of Three-Dimensional Gaze-Tracking......Page 484
2. Three-Dimensional Spatially Variant Rendering......Page 489
3. Three-Dimensional GCMMDs of Medical Images......Page 490
1. Implementation......Page 492
2. Performance......Page 494
1. Implementation......Page 495
2. Performance......Page 498
C. INTEGRATION WITH EYE-TRACKERS......Page 500
3. Three-Dimensional Gaze-Tracker......Page 501
ACKNOWLEDGMENTS......Page 504
REFERENCES......Page 505
I. INTRODUCTION......Page 509
II. THE STRUCTURAL SIMILARITY PARADIGM......Page 513
A. THE STRUCTURAL SIMILARITY INDEX......Page 514
B. SSIM INDEX IN IMAGE QUALITY ASSESSMENT......Page 518
III. THE INFORMATION THEORETIC PARADIGM......Page 519
A. NATURAL SCENE MODEL......Page 522
C. HVS MODEL......Page 523
D. THE VISUAL INFORMATION FIDELITY MEASURE......Page 524
IV. PERFORMANCE OF SSIM AND VIF......Page 526
REFERENCES......Page 533
