Virtual and Remote Control Tower: Research, Design, Development, Validation, and Implementation

Foreword: On the Origins of the Virtual Tower
Preface to the Second Edition
Preface to the First Edition
Contents
About the Editor and Co-Editors
Abbreviations
Preconditions
Introduction: Basics, History, and Overview
1 Some Basics
2 History of Virtual and Remote Tower Research and Development (2001–2014)
2.1 Previous Work, Vision and Initial Steps
2.2 Basic Research (2002–2005)
2.3 Proof-Of-Concept Project (2005–2008)
2.4 From Prototype Development to Technology Transfer (2008–2014)
3 Towards MRTO Standardization and Implementation (2010–2021)
4 Chapter Overview
References
Visual Features Used by Airport Tower Controllers: Some Implications for the Design of Remote or Virtual Towers
1 Introduction
2 Visual Information Used in the Airport Tower
3 Visual Features at SFO
4 Deceleration During Landing at SFO
5 Summary
References
Detection and Recognition for Remote Tower Operations
1 Introduction
2 Tower Control
2.1 Basic Duties
2.2 Airport Radar and Surveillance Systems
3 Analysis of Visual Features
3.1 Analysis of Tower Tasks and Visual Needs
3.2 List of Visual Features
4 Method and Results
5 Discussion and Effect on Image Resolution
References
Remote Tower Research in the United States
1 Background
2 Present Use of the Out-the-Window (OTW) View
2.1 SNT Walkthrough (FAA/George Mason University, Fairfax, VA, 2009)
2.2 Verbal Protocol Analysis (Boehm-Davis, 2010)
3 The Operational Concept for SNT (FAA, 2008b)
3.1 Assumptions Related to the SNT Concept
3.2 Substituting for the Window View
3.3 2-D Surveillance Display for the Controllers
3.4 Decision Support Tools
3.5 Use of Aircraft Derived Data (ADD)
3.6 SNT Configurations
4 Summary of the SNT Concept
5 Assessment of the SNT Concept
5.1 Off-Nominal Events in SNT Operations (Nene, 2009)
5.2 SNT Safety Impact Assessment (Cheng, 2010; Colavito & Nene, 2010)
5.3 Use of Digital Camera for Surface Surveillance (FAA, 2011a; Grappel, 2009)
6 Some Observations Related to the SNT Concept
7 Change in Focus of the FAA’s Remote Tower Research
7.1 Current Operations at Non-towered Airports (NTAs)
7.2 Present Shortfalls in NTA Operations (Colavito, 2013)
7.3 Concept for Remotely Providing Selected NTA Services
8 Present Effort on the Colorado Initiative
9 Remote Tower Demonstration Project at the Leesburg Executive Airport (KJYO), Leesburg, Virginia
10 Future of Remote Towers in the U.S.
References
Remotely-Operated AFIS in Japan
1 The AFIS Situation in Japan
2 Remote AFIS in Japan
3 System Displays and Their Functions
3.1 APDU (Aircraft Positioning Display Unit)
3.2 ITV
3.3 FACE (Flight object Administration CEntre system)
3.4 TDU (Terminal Display Unit)
3.5 MPID (Multi-Purpose Information Display)
4 Remote AFIS Operational Challenges
4.1 Effect of ITV System
4.2 Multiple Operation and Issues in Remote AFIS (Double Contact)
5 Summary
References
Development and Field Testing of Remote Tower Prototype
Remote Tower Experimental System with Augmented Vision Videopanorama
1 Introduction
2 Work Analysis
3 Experimental Remote Tower System
3.1 Optical Design and Expected Performance
3.2 Digital Reconstruction of the Out-of-Windows View
3.3 Videopanorama Interaction and Control Display
3.4 Augmented Tower Vision and Movement Detection
3.5 Triangulation
4 Field Testing for Verification of System Performance
4.1 Latency
4.2 Optical Resolution: Static Measurements
4.3 Performance Verification: Flight Tests
5 Simulation Environment
6 Conclusion and Outlook
References
Remote Tower Prototype System and Automation Perspectives
1 Introduction
2 Work and Task Analysis for Requirements Specifications and Prototype Design
3 The RTO-System Setup and Human-System Interface
3.1 Video-Panorama Camera System
3.2 RTO-Controller Working Position
3.3 High-Bandwidth Wide-Area Network
3.4 RTO Software and Human–Machine Interaction
4 Perspectives of Automatic Movement and Object Detection
4.1 Movement Detection via Optical Flow Analysis
4.2 Region Tracking Algorithm Based on Background Estimation
4.3 Object Classification
4.4 Thermal Imaging
5 Functional Tests and Verification
5.1 Measuring Camera-Display Latency
5.2 Electromagnetic Compatibility
5.3 Image Optimization
6 Conclusion
References
Integration of (Surveillance) Multilateration Sensor Data into a Remote Tower System
1 ENRI’s Remote Tower Research Project
2 System Functions and Components
2.1 System Overview
2.2 The Out of the Window (OTW) View System
2.3 The OCTPASS Surveillance Sensor
3 Object Following Techniques
3.1 Video-Based Object Detection and Following
3.2 Automatic Object Recognition
4 MLAT Integration
4.1 Target Tracking on OTW (Video + MLAT)
4.2 Example of a Phenomenon Caused by MLAT Issues
4.3 PTZ Camera Automatic Tracking (Video + Monitoring Sensor)
4.4 Integrating Panoramic Image, PTZ and Surveillance Information
5 Summary
References
Which Metrics Provide the Insight Needed? A Selection of Remote Tower Evaluation Metrics to Support a Remote Tower Operation Concept Validation
1 Introduction
2 Extended Field Trial Infrastructure
3 Remote Tower Metrics
4 Method
4.1 Participants
4.2 Apparatus
4.3 Design
4.4 Procedure
5 Results
5.1 Basic Analysis of Safety Related Metrics
5.2 Evaluation for the RTMs
6 Discussion and Conclusion
6.1 Basic Analysis of Safety Related Metrics
6.2 Evaluation for the RTMs
7 Outlook
References
Model Based Analysis of Two-Alternative Decision Errors in a Videopanorama-Based Remote Tower Work Position
1 Introduction
2 Methods
2.1 Participants
2.2 Experimental Environment and Conditions
2.3 Experimental Design and Task
3 Results
4 Data Analysis and Discussion
4.1 Technical Limitations
4.2 Bayes Inference: Risk of Unexpected World State
4.3 Discriminability d′ of Aircraft Maneuvers
4.4 Nonparametric Discriminability A
4.5 Error Prediction Using the Information Processing/Time Pressure Hypothesis
5 Conclusion
References
Human-in-the-Loop Simulation for RTO Workload and Design
Multiple Remote Tower Simulation Environment
1 Motivation
2 Method: Design of a Comprehensive RTO Validation Platform
2.1 E-OCVM Guideline
2.2 FTS-HITL-Coupling
2.3 HITL-Adoption
2.4 HITL-Field-Transfer
2.5 Specification
3 Implementation of the RTO Validation Platform
4 Results of the MRT Simulation Platform Application
4.1 Case Study: Application of FTS-HITL-Coupling
4.2 Case Study: Remote Tower Human Factors Study
4.3 Case Study: Remote Tower Center (RTC) Study
4.4 Case Study: Multi Remote Tower Study
4.5 Case Study: SESAR2020 Shadow Mode Trials
5 Conclusion and Outlook
References
Assessing Operational Validity of Remote Tower Control in High-Fidelity Simulation
1 Introduction
1.1 Motivation
1.2 Related Work
1.3 Aerodrome Control Work Environment
1.4 Characteristics of Regional Airports and Consequences for Air Traffic Control
1.5 Controller Assistance via Information Super-Imposition and Automatic Zoom Camera Tracking
1.6 Research Questions
2 Method
2.1 Subjects
2.2 Experimental Design
2.3 Simulation Setting
2.4 Experimental Task
2.5 Experimental Conditions
2.6 Controllers Working Positions
3 Dependent Variables and Data Analysis
3.1 Feasibility, Acceptance and Usability of the Workplace
3.2 Assessing the Relevance of the Far View
3.3 Benefit of the Assistance Tools and Analysis of the Eye-Tracking Data
4 Results
4.1 Feasibility, Acceptance and Usability of the Concept
4.2 Relevance of Far View
4.3 Benefit of Assistance Tools
5 Discussion
5.1 Feasibility, Acceptance and Usability of the Concept
5.2 Relevance of the Far View and the Visual Information
5.3 Benefit of Assistance Tools
6 Conclusion
References
Model Based Analysis of Subjective Mental Workload During Multiple Remote Tower Human-In-The-Loop Simulations
1 Introduction
2 Mental Workload and Workload Measures
2.1 Definition of Mental Workload
2.2 Operational Approaches and Models to Predict Mental Workload in ATC Based on Task Load
3 Experiment and Data Collection
3.1 Sample and Procedure
3.2 Simulation Setup
3.3 Experimental Design and Workload Assessment
4 Nonlinear Work- and Task Load Model
4.1 Logistic Mental Workload Model ISA(n)
4.2 Logistic Model for Rate of Radio Calls RC(N)
4.3 Power Law Model ISA(RC)
5 Experimental Results
5.1 ANOVA Data Analysis: General Linear Model and Multivariate Regression
5.2 Logistic Model Based Regression Analysis of ISA(n) Workload and RC(n) Taskload Means
5.3 Psychophysics Power Law Model for ISA(RC) Regression Analysis
6 Discussion
6.1 H1: Dependence of Subjective WL on Traffic Load and Impact of MRTO
6.2 H2: Dependence of Objective Communication TL on Traffic Load and Impact of MRTO
6.3 H3: Positive Correlation Between WL and Communication TL Variables RD, RC
6.4 H4: Mediator Effect and Sufficiency of Communication Load for Explaining Workload
7 Conclusion
Appendix
References
Changing of the Guards: The Impact of Handover Procedures on Human Performance in Multiple Remote Tower Operations
1 Introduction
2 Method
2.1 Participants
2.2 Design and Material
3 Results
3.1 Mental Workload
3.2 Situation Awareness
3.3 Safety
3.4 Impact of Handover Procedures
4 Discussion
References
Which Minimum Visual Tracking Performance is Needed in a Remote Tower Optical System?
1 Introduction and Background
2 Experimental Design
2.1 Participants
2.2 Experimental Platform
2.3 Test Variables and Test Procedures
3 Results
4 Discussion
5 Conclusions and Outlook
References
Videopanorama Frame Rate Requirements Derived from Visual Discrimination of Deceleration During Simulated Aircraft Landing
1 Introduction
2 Methods
2.1 Subjects
2.2 Apparatus
2.3 Experimental Design and Task
3 Results
3.1 Response Times
3.2 Decision Statistics: Response Matrix
4 Data Analysis and Discussion
4.1 Simulation of Movement After Touchdown
4.2 Bayes Inference: Risk of Unexpected World State
4.3 Response Bias
4.4 SDT Discriminability d′ and Decision Bias c
4.5 Nonparametric Discriminability A and Decision Bias b
5 Conclusion
References
Which Minimum Video Frame Rate is Needed in a Remote Tower Optical System?
1 Introduction
2 Theorethical Background
3 Methods
4 Results
5 Discussion
6 Conclusion
References
Advanced and Multiple RTO: Development, Validation, and Implementation
The Advanced Remote Tower System and Its Validation
1 Introduction
2 ART Functions
2.1 360° Circular Panorama Display
2.2 Visibility Enhancement Technology
2.3 Presentation of Airport and Geographic Information
2.4 Presentation of Weather Information
2.5 Sensor Data Fusion
2.6 Presentation of Aircraft and Vehicles
2.7 Pan-Tilt-Zoom (PTZ) Camera
3 Test and Validation Program
4 Results
4.1 Results for the Panorama Display
4.2 Geographic Information Display
4.3 Weather Presentation
4.4 Results of Traffic Presentation (Labelling and Tracking)
4.5 Results of Validation of the Visibility Enhancement Technology (VET)
4.6 Results for the PTZ Camera and Object Tracking
4.7 Results of Validation of the Combination of All ART Functions
4.8 Results from the Expert Judgement Workshop
5 Analysis and Recommendations
5.1 Observations
5.2 System Maturity
5.3 Operational Aspects and Recommendations
References
Designing and Evaluating a Fusion of Visible and Infrared Spectrum Video Streams for Remote Tower Operations
1 Introduction
2 Theoretical Background
2.1 Characteristics of VIS and IR Data
2.2 Influence of the Atmosphere and Aspect
2.3 Image Fusion
2.4 Detection and Recognition of Objects
3 Designing Image Fusion
3.1 Key Requirements
3.2 Hot-Spot Only Fusion
4 Evaluating Hot-Spot Only Fusion
4.1 Research Questions
4.2 Sample
4.3 Remote Tower Camera Set-Up
4.4 Preparation of Video Streams
4.5 Independent Variables
4.6 Dependent Variables
4.7 Apparatus and Testbed
4.8 Procedure
4.9 Processing of Timestamps
5 Evaluation Results
5.1 Departures and Arrivals
5.2 Specific Traffic Scenarios
5.3 Research Question 4: How Do the Operators Subjectively Evaluate the Fusion of VIS and IR Images?
6 Discussion
6.1 Research Question 1: Objective Detection Performance
6.2 Research Question 2: Visibility of Relevant Information to the Operators
6.3 Research Question 3: Benefit of Hot-Spot Only Fusion
6.4 Research Question 4: Subjective Evaluation
6.5 Comparison with Theoretical Distances According to the Johnson Criteria
7 Summary and Outlook
References
Planning Remote Multi-airport Control—Design and Evaluation of a Controller-Friendly Assistance System
1 Introduction
2 Multi-airport Control
2.1 Concept Behind VICTOR
2.2 Necessity and Elements of a Planning Tool
2.3 Requirements for MasterMAN Planning Tool
3 Usability/User Experience
3.1 Usability
3.2 User Experience
4 Evaluation of MasterMAN
4.1 Test Design and Procedure
4.2 Test Preparation
4.3 Conduction
4.4 Results and Analysis
5 Summary and Outlook
References
The Certification Processes of Multiple Remote Tower Operations for Single European Sky
1 Introduction
2 Background of Policy and Practice
2.1 The Evolution of Remote Tower Operation
2.2 The Cost Efficiency of Multiple Remote Tower Operation
2.3 Safety Assessment Methodology of Multiple Remote Tower Operations
2.4 The Processes of Regulatory Approval for Practical Implementation
3 Demonstration of the Live Exercises
3.1 The Development of Safety Case for 50 Live Exercises
3.2 Preparations of Live Exercises
3.3 Exercise Execution
4 Findings of Live Exercises
4.1 Findings on Batch-1 Demonstration
4.2 Findings on Batch-2 Demonstration
4.3 Findings on Batch-3 Demonstration
5 Discussion of Impacts
5.1 New Technology Induced Unexpected Behaviours Related to HCI
5.2 Impact on Safety
5.3 Impact on Capacity
5.4 Impact on Human Performance
6 Conclusion and Recommendation
References
Designing a Low-Cost Remote Tower Solution
1 Introduction
2 The Need for a Low-Cost Solution
2.1 Constraints for Uncontrolled Aerodromes
2.2 Operational Use-Cases
3 Design for Low-Cost Remote Surveillance
3.1 Camera Set-Up
3.2 Visualization Concepts
3.3 The Problem of Cybersickness
4 Validation
4.1 Method
4.2 Results
4.3 Discussion
5 Comparison with Standard Remote Tower Solutions
6 Summary and Outlook
References
Appendix A Basic Optics for RTO Videopanorama Design
A.1 Geometrical Optics for Panorama Design and Pixel Resolution
A.2 Diffraction Limit and Resolution
A.3 Contrast and MTF
A.4 Determining Effective Resolution by Detectability Experiments
A.5 Basics of Physiological Optics for Detectability
A.6 Luminance Sensitivity and Gamma Correction
Appendix B Signal Detection Theory and Bayes Inference for Data Analysis
B.1 Bayes Inference
B.2 Signal Detection Theory
B.2.1 Parametric Discriminability d′ and Subjective Criterion c
B.2.2 Nonparametric Discriminability A and Subjective Bias b
Appendix C Mental Workload and Measures for (Quasi) Real-Time Applications
C.1 Mental Workload
C.2 Importance of Workload for Design of Operational Concepts and Work Environment
C.3 Subjective Quasi Real-Time WL Measures
C.4 Psycho-Physiological Measures Heart Rate and HR-Variation
C.5 Neurophysiological (EEG-Based) Measures
C.6 Formal Models
C.6.1 Time Pressure and Information Processing TP/IP for Objective Workload
C.6.1.1 Sector Capacity for Networks (CAPAN)
C.6.1.2 Information Processing/Time Pressure (IP/TP) Theory
C.6.2 Sigmoid Model of Subjective WL During En-route Sector Control
C.6.3 Psychophysics of Subjective Mental Workload
Appendix D Psychophysics of Mental Workload: Derivation of Model Equations
D.1 Resource Limitation Model
D.2 Logistic ISA(n)-WL Model
D.3 Linearized Logistic ISA Model
D.4 Logistic Radio Calls Frequency Model RC(n)
D.5 Psychophysics Power Law Model ISA(RC)