|
|
| |
Research: System Performance Studies and Metrics |
|
Project Summaries:
The following are the NEXTOR II research projects in the area of System Performance Studies and Metrics:
Passenger Delay Analysis
Sponsor: Air Traffic Organization, System Operations Services
Participants: Massachusetts Institute of Technology, George Mason University
Many of the existing methods for evaluating an airline’s on-time performance are based on flight-centric measures of delay. However, recent research has demonstrated that passenger delays depend on many factors in addition to flight delays. For instance, significant passenger delays result from flight cancellations and missed connections, which themselves depend on a significant number of factors. Unfortunately, lack of publicly available passenger travel data has made it difficult for researchers to explore the nature of these relationships.
Under this project, we developed methodologies to model historical travel and delays for U.S. domestic passengers. We developed a multinomial logit model for estimating historical passenger travel and extended a previously-developed greedy re-accommodation heuristic for estimating the resulting passenger delays. We reported and analyzed the estimated passenger delays for calendar year 2007, developing insights into factors that affect the performance of the National Air Transportation System in the United States. We also analyzed the causes and costs of U.S. passenger travel disruptions by applying data analysis and statistical modeling to historical flight and passenger data.
We demonstrated that as much as 50% of passenger delays are caused by passenger travel disruptions, either flight cancelations or missed connections. The propensity for disruptions varies significantly across airports and carriers, based on key factors such as scheduling practices, network structures, and passenger connections. We found that cancelations, which are the largest cause of disruption-related passenger delays, vary substantially across carriers, even when accounting for baseline variability across airports. Passenger and operational considerations also play a significant role in cancelation decisions. Regarding missed connections, much of the variability can be explained just by flight delays for the airport and carrier, though flight schedule construction is also a critical factor. Highly peaked (or banked) flight schedules tend to reduce connection times and therefore increase the risk of missed connections. Last, we demonstrated the importance of a variety of factors on the ease of re-accommodating disrupted passengers.
In addition to the analyses already performed in this research project, access to estimated passenger travel and delay data provided through this project has also enabled many other passenger-centric research opportunities that would not exist otherwise.
The Impact of Lightning Strikes on National Airspace System (NAS) Performance:
A Business Model for FAA Infrastructure Investments
Sponsor: Air Traffic Organization, ATC Facilities
Participants: University of California, Berkeley
Duration: 9/2010 - present
This research addresses the Federal Aviation Administration’s (FAA) need to further develop a Business Case to support LPGBS changes and therefore protect NAS infrastructure from lightning strikes and convective weather. In the previous phases of this work, analyses were based on the National Lightning Detection Network (NLDN) data and outage reports of the FAA gathered between 1999 and 2005. The current work uses the updated NLDN and Logistics databases (i.e., data gathered beyond 2006) and further explores: (1) the impact of lightning strikes on NAS equipment outages, (2) correlation of strokes and outages with any other type of Systems upsets, (3) impact of lightning-induced outages and other types of convective-weather-induced outages on NAS performance in terms of throughput, and (4) if the cost and number of parts for equipment that need repairs due to lightning strikes are proportionally balanced with the number of reported outages. The research also explores if there are any statistically significant differences between the baseline case results (1999-2005), and the updated results (2006-2013).
A refined NAS Impact Risk Model for FAA Infrastructure Investments is also proposed with the objective to provide sound investment decisions for the NAS equipment (regarding the lightning strikes). The investment decisions are based on (i) the extent the outages have on the NAS performance, (ii) the type, importance and role of airport/equipment with the local region, TRACON or ARTCC, and (iii) the type of traffic served.
In order to perform such analyses additional information are used on FAA investments into facilities and FAA databases such as Aviation System Performance Metrics (ASPM), and Operations Network (OPSNET).
North Atlantic Alternate Benefits Study
Sponsor: Air Traffic Organization
Participants: Virginia Tech
In Phase 1 of this project we revised and validated many features of the NATSIM model developed by the FAA Tech Center. Virginia Tech delivered model changes in 23 weekly reports and design reviews accomplished in Phase 1. A final report on Phase 1 was delivered to the FAA in March 2012. In Phase 2 of this project we developed a new alternative benefits model called NATSAM benefits model to quantify cost/benefits of North Atlantic operations in the context of various ICAO/FAA equipage mandates. The NATSAM model development is complete and final modeling results will be presented at a future EFG meeting in Paris. Preliminary results of this model were presented at the October 4th INFORMS meeting in Myrtle Beach, SC. A final report for Phase 2 is due on December 2012.
This model will be used by FAA to support air navigation service provider decisions on future aircraft equipage mandates.
Flight Predictability: Concepts, Metrics and Impacts
Sponsor: Air Traffic Organization
Participants: University of Maryland, University of California, Berkeley
In this study, we analyze two benefit mechanisms by which improved flight time reliability could reduce airline operating cost. In one mechanism, reduced variability results in short scheduled block times, and airlines realize cost savings because scheduled block time is an important driver of several categories of airline costs. In the other, reduced variability results in less contingency fuel being loaded, and airlines save on the costs to carry the extra fuel. In examining both mechanisms, we employ econometric models to determine the relationship between flight time variability and the relevant behavior (scheduled block time setting or fuel loading) and additional models to estimate the cost impacts of changes in the behavior that could result from improving flight time reliability.
We first investigate how a change in actual block time distribution will affect scheduled block time (SBT) and system performance. This relationship is studied with empirical data and multiple regression models. The distribution of the historical block time for a flight is depicted by the difference between every 10th percentiles. We found that gate delay plays a minor role in setting SBT and that SBTs have decreasing sensitivity to historical flight times toward the right tail of the distribution. Using this behavior model, we generate different scenarios with hypothetical changes in actual block time distribution and evaluate the impacts of SBT, delay, and on-time performance. Focusing on the inner the right tail of the distribution would yield the most improvement in the system. It is also found that even an increase in average block time can still bring benefits such as reduced SBT and better on-time performance. Taking the heterogeneity across airlines into consideration, we also decompose the dataset to study specific airlines. Low cost carriers and United Airlines are found to be the more aggressive in setting SBT.
To measure the hypothetical economic benefits that such changes could have, Chapter 4 herewe use coefficients from the open literature to relate changes in scheduled block time to reductions in scheduled flights and crew staffing. Using fleet mix distributions and cost-of ownership data, plus pilot and cabin crew salary data, we monetize those benefits. Major carriers could save on the order of $20 million with fleet usage efficiencies; similar results could not be produced for the low-cost carriers because of missing data. For crew efficiencies, however, all categories of carriers could save millions of dollars, with about three quarters of that ascribed to cockpit crew and the remainder to cabin crew.
To study fuel loading, we analyze a large and recent dataset with flight-level fuel loading and consumption information from a major US airline. With thisthese data, firstly the relationship between the amount of loaded fuel and flight predictability performance is estimated using a statistical model. Then the impact on loaded fuel is translated into fuel consumption and ultimately, fuel cost for US domestic operations. We find that one minute of standard deviation in airborne time, capturing flight unpredictability, within a month for the same OD pair and shift of day would lead to 0.9 minute increase in loaded contingency fuel and 1.7 loaded contingency and alternate fuel.
We include a short chapter on preliminary investigations into predictability impacts of called rates for Ground Delay Programs. This effort was only gestational, so we are able to report only on some preliminary ideas for data analysis and presentation. This could be the starting point for a more exhaustive study at a later date.
Overall, we estimate annual cost savings to the US domestic airline industry on the order of $400 million from reductions in scheduled block time that could result from plausible changes in the block time distribution. We estimate fuel burn savings of $180-$240 million from eliminating flight time variability. We also find that different metrics are appropriate for these two mechanisms, with the scheduled block time based upon the inner right tail of the block time distribution (out to roughly the 70th percentile). Fuel loading considers the entire distribution, with particular focus on the outer right tail, making the standard deviation an appropriate metric.
Flight Cancellation Behavior and Aviation System Performance
Sponsor: Air Traffic Organization
Participants: University of California, Berkeley
Duration: 9/2011 - 3/2014
To better understand and mitigate flight delays, the FAA’s office of Performance Analysis and Strategy (PAS) forecasts delays at the nation’s 30 major airports on a monthly basis. The purpose of this effort is to identify airports with significant delays, so that appropriate steps may be taken to prevent or mitigate such delays. The delay forecasts are performed six months in advance using demand projections based on published (look-ahead) airline schedules. The demand sets used for delay forecasts do not include flight cancellations made in response to operational contingencies. Therefore, in order to improve the fidelity of the FAA’s delay model, it will be necessary to formulate a suitable model for cancellations.
This study focuses on two pieces of analysis to fulfill this goal. First, the factors that lead to flight cancellations need to be understood. We use econometric choice models applied to a large historical dataset to infer the preferences and behaviors of airlines with respect to flight cancellations. We have found that flight characteristics are an important factor for cancellations, even when accounting for adverse weather effects. Airline heterogeneity is large and reveals different behavior across the industry.
Second, we will use queueing simulation techniques similar to that used by the current FAA model to determine the effect flight cancellations have on delay estimates. The cancellation model estimates are used to predict flight cancellations and evaluate their impact on queueing delay forecasts used by the FAA.
We examine the effect of cancellations on delays using a sample of 160 airport-days. Our results indicate that delays are over-predicted by the FAA queueing models by 30% when not considering flight cancellations. Even a very naïve model of flight cancellations, where only time-of-day and distance effects are considered, will reduce the delay estimates by 9%.
Finally, a case study was performed based on the current operating environment at SFO, where runway closures in the summer of 2014 will lower the airport’s capacity. Moreover, airlines are proposing schedules with 5% more demand. The increased schedule leads to an increase in queueing delay. We used a cancellation model with queueing delay as an explanatory variable to predict the increase in cancellations caused by the increase in demand. The results from our model indicate that departure cancellations will increase at an almost one-to-one ratio with the proposed demand increase, thus negating any benefit to airlines from a denser schedule.
Temporal Analysis of Operational Errors at Air Traffic Control Facilities
Sponsor: Air Traffic Organization
Participants: University of California, Berkeley
Duration: 7/2010-12/2011
This project focuses on the occurrence of aircraft separation minimum violations as documented in the form of operational errors (OEs) at two types of air traffic controller facilities, terminal radar approach control (TRACON) facilities and air route traffic control centers (ARTCCs). Poisson regression was used to analyze the daily count of OEs at various facilities of both types. The occurrence of OEs was found to increase approximately with the square of daily traffic at TRACON facilities and slightly higher than the square of traffic at ARTCC facilities. At TRACON facilities, where separation violations are not automatically reported, an increase in reporting was seen after a new severity metric was introduced in 2007. It was also found that large, consolidated TRACON facilities tend to behave like a sum of several smaller facilities rather than a single larger facility with respect to the occurrence of OEs vs daily traffic. Weather effects such as visibility and wind were found to influence the occurrence of OEs as well. The model prediction for TRACON facilities is very good for the most severe OE types and very poor for the least severe OE types, indicating many unobserved factors contributing to the reporting of the least severe OE types in the terminal environment. Model prediction for the ARTCC facilities is very good for about half the facilities
Air Connectivity Index
Sponsor: Air Traffic Organization, System Operations Services
Participants: Purdue University
In this project, we investigate the measurement of air transport connectivity in the US air transportation network (ATN). In particular, we study the merit and applicability of the Air Connectivity Index (ACI) developed by researchers at the World Bank to measure the connectivity of the global air transport network. The ACI is derived using a generalized gravity model and accounts for both direct and indirect network effects. Our work has demonstrated that the ACI is also applicable to the US ATN, capturing the relevant aspects of connectivity and correlating with some econometric variables, but also presenting new challenges for correct implementation, especially in the proper estimation of regression parameters.
We determined how to adapt the original ACI calculation methodology to suit the US ATN. We applied the ACI at different levels of granularity (state, metropolitan, and airport) and conducted time series of the ACI at the metropolitan and airport level. We developed statistical models to predict ACI at the airport and metropolitan levels using facility and econometric variables. We investigated implications of results for the US ATN and how the index can be used by policy makers. We continue to substantiate the method for correct calculation in the US ATN and are collaborating with the original World Bank researchers to further improve and expand the global application of the ACI.
Optimized Profile Descent Modeling and Evaluation
Sponsor: Air Traffic Organization, System Operations Services
Participants: Purdue University (PI: Dengfeng Sun; Co-PI: Steven Landry)
The primary goal of this research was to evaluate the fuel saving/consumption of conflict-free Optimized Profile Descents (OPDs) under normal and heavy traffic conditions at different airport Metroplexes. In the research, we developed sequencing algorithms to strategically resolve conflicts for inbound traffic, and obtained fuel burn data using the FACET fast-time simulation model. The research results show that the fidelity of the fuel burn estimate strongly depends on how closely the simulation approximates the real operations. In a related previous research project, our research imposed several assumptions and constraints in the course of exercising the simulation, while in this NEXTOR II project, we employed more operationally realistic constraints to yield a more robust analytical result.
Related Publications:
1. Evaluation of Continuous Descent Approach as a Standard Terminal Airspace Operation, Yi Cao, Tatsuya Kotegawa, Dengfeng Sun, Daniel DeLaurentis and Joseph Post, the 9th USA/Europe Air Traffic Management (ATM) Seminar, Berlin, Germany, June 14-17 2011.
2. Evaluation of fuel benefits due to continuous descent arrival procedures,Yi Cao, Li Jin, Nguyen V. P. Nguyen, Steven Landry, Dengfeng Sun and Joseph Post, submitted to Air Traffic Control Quarterly.
CNS Availability and NAS Performance: Controller-Pilot Data Link Communications (CPDLC)
Sponsor: Air Traffic Organization, NAS Systems Engineering Services OfficeServices
Participants: University of California, Berkeley
Duration: 2011-2013
In this study we investigate how unscheduled outages of Data Comm impact the National Airspace System (NAS) performance in the Next Generation Air Transportation System (NextGen) environment. A methodology is developed using analytical models and simulated data for translating impact of Data Comm outages on NAS performance. Fault tree analysis method and reliability block diagrams are used as engineering modeling techniques to describe Data Com system components and calculate system’s availability. Because Data Comm is not in operation yet, the developed methodology is based on simulated operational data. The overall methodology is based on several analytical models for estimating changes in capacity, safety and controller’s workload. The goal is to find the optimal balance between human psychological characteristics and strict engineering disciplines. Using the proposed methodology, calculated are delays, safety risks and excess fuel consumption that would occur due to a Data Comm outage. The conclusion is that the disadvantages caused by the Data Comm failure are negligible compared to the benefits of increased capacity and savings in delays that the implementation of Data Comm would bring. We especially explore any negative relationships between (1) the excess distance (difference between the filed route and the route actually flown) and (2) controller cognitive utilization. A more generalized concept of DataComm is also developed, providing an example to establish a standard methodology for other NAS systems in the NextGen environment. This is very important as it will provide a clear path for future analysis of other NAS systems.
Survey and Analysis of Air Traffic Control Command Entries
and the Effect on Decision Support Tool Performances
Sponsor: Air Traffic Systems Concept Development Group,
Simulation & Analysis Team AJP-661; FAA William J. Hughes Technical Center
Participants: University of California, Berkeley
Duration: 2010
The Federal Aviation Administration (FAA) has developed a new Air Traffic Control (ATC) system to replace the existing Host Computer System (HCS) and User Request Evaluation Tool (URET) in the en route domain. This system is called En Route Automation Modernization (ERAM). Two key components of ERAM are the Trajectory Predictor (TP) and Conflict Probe (CP). While the TP predicts more accurate aircraft position, the CP predicts conflicts between two aircraft, or between an aircraft and a Special Use Airspace (SUA) using predicted flight paths (trajectories) generated by the TP. When the CP detects a conflict, the system alerts the air traffic controller of the problem in order to aid the controller to manage separation safely. The accuracy of these predictions is crucial to the performance of ERAM and the overall performance of en route airspace.
In order to support the implementation of improvements to the CP in ERAM, this study investigates and quantifies the impact and mitigation of CP prediction errors associated with missing aircraft intent data, attributable to the lack of automation input of verbal-only amendments by Air Traffic Control.
Hence, for the first time, a comprehensive methodology is developed to quantify the impact of missing aircraft intent information on the performance of the CP by analyzing the accuracy of the en route trajectory modeling capability and the usability of the CP. The proposed method focuses on the analysis of missing ATC clearances, while the application of the proposed method analyzes the impact of the 4th line ATC entries and then estimates the improvement of a complete, closed clearance of a 2-leg lateral maneuver. The dataset was obtained from the Washington, DC Air Route Traffic Control Center (ZDC) over 48 hours for January 26-27, 2010.
In the first step, a large database was created, comprised of the CMS, a record of air traffic controllers’ issued amendments, flight adherence measurement data, and weather information. Then, the amendments were interpreted and translated into new flight plans based on the amendments issued and the aircraft’s actual position. As a result, new CMS messages were created. In the second step, the new CMS messages were inserted into JEDI, a portable version of URET, to obtain error measurement metrics for trajectory accuracy.
There were four metrics measuring trajectory accuracy: horizontal, cross track, along track, and vertical errors. All were measured in nautical miles except for vertical error, which is measured in feet. The magnitudes of each treatment scenario’s error metrics were subtracted from the magnitudes of each baseline run’s error metrics to calculate error reduction (or improvement). Afterwards, the treatment scenarios were compared against the baseline run with a two sample paired t-test to determine if any statistically significant error reductions occurred, and study findings were discussed.
The study findings quantitatively demonstrate that (i) heading amendments increase trajectory accuracy, and (ii) speed amendments, noted in 4th line of the flight data block, decrease the overall number of false alerts generated by the Conflict Probe.
|
| |
|
|
|