The Art Of Calibration
Unleashing The Power Of Network Planning
Calibration can help airlines plan better through more accurate and thorough scenario-evaluation processes. To assist airlines in achieving optimal calibration results, Sabre Airline Solutions® offers an experienced team of operations-research practitioners dedicated to sharing their knowledge and expertise with airlines around the world.
In all successful businesses, planning is critical. Because entire departmental functions often revolve around it, business planning must be timely and accurate.
Particularly in the airline industry, a compelling argument can be made that business planning based on accurate and robust analyses can have a significant impact on a carrier’s bottom line.
Thorough airline business planning directly affects the performance of every organizational function — and it involves many diverse elements of airline activity.
The network-planning process, for example, helps airlines define and execute their long-term planning initiatives around key decisions on salient factors including fleet composition, product offerings, identification of growing markets, hub design, fleeting frequency and departure-time optimization.
To systematically support these initiatives, airlines need technological capabilities encompassing activities such as schedule creation and management, demand forecasting, profitability evaluation, and fleet and timing optimization under constraints.
Fortunately, highly sophisticated tools are available to help airlines deal with all these factors. These tools require calibration from time to time to maintain accurate and robust forecasting.
What Is Calibration?
Calibration is the process by which a robust set of parameters is created — using known information — so a model can generate what can be confidently characterized as an acceptable forecast.
By definition, parameter creation requires historical data collection and validation, statistical-parameter estimation methods, and forecast-quality measurement.
Among the tools in the marketplace, Sabre® AirVision™ Planning and Scheduling supports these decision-guiding requirements using Sabre® AirVision™ Profit Manager as the core forecasting and evaluation engine. In addition, Sabre® AirVision™ Global Demand Data serves as a primary historical market-data source for calibration, providing key inputs such as overall market sizes, historical market shares and passenger flows.
Industry-Leading Forecasting Capabilities
Profit Manager is a forecasting solution that supports complex scenario evaluations such as:
- New market viability;
- Frequency and timing changes;
- Codeshare, joint venture and alliance modeling;
- Impact of competitor schedule changes;
- New hub structures.
The forecasting process consists of a passenger-connection builder, a customer-choice model that emulates passenger behavior when choosing connections, and a “spill-and-recapture” model to estimate traffic when demand cannot be served by available capacity.
Profit Manager enables airlines to control several different parameters to model observed behavior, as well as fine-tune the forecasting results.
To maximize productivity when using the system for decision support, airlines must generate accurate forecasts. For larger airlines, this may equate to tens of thousands of market forecasts. Therefore, parameter setup and tuning are of paramount importance.
Profit Manager requires calibration to ensure quality output throughout its product lifecycle. Calibration for Profit Manager is an iterative process by which system parameters are estimated and validated using various market and internal airline data sources.
Typical tasks that need to be performed when calibrating the system include:
- Defining the network — Analyzing a larger network allows new market scenarios to be modeled more effectively; however, it inflates the run time and may add unnecessary bulk to stable scenarios. In addition, an airline needs to obtain data for all the markets considered in the system.
- Estimating economic data such as market demands, segmentations and fares.
- Estimating parameters for itinerary generation to balance the construction of valid itineraries as opposed to itineraries that may never, in reality, be chosen and will dilute market share in the system.
- Estimating passenger-preference parameters for the customer-choice model.
- Establishing parameters for spill and recapture — Revenue management practices are simulated by defining the proper fares, load factors on closed flights, and the coefficient of variation of demand.
- Measuring system performance and performing macro- and micro-level adjustments to fine tune the results.
Global Demand And Fare Estimation
As recently as a decade ago, estimating market demand for historical periods was not a significant challenge since global distribution systems (GDSs) captured most bookings, and these data points — through subscription — have been readily available to airlines.
With the growth of low-cost carriers and online bookings, the direct channel overall is now comparable to the GDS channel in terms of bookings.
The relative booking proportions vary by market segment, such as leisure or business travel, length of journey (short-haul versus long-haul), airline and market type (domestic versus international).
Thus, GDS data — while providing a certain richness of detail — only generates what might be properly described as an “incomplete” picture of total demand in a market.
Global Demand Data solves this problem by:
- Combining and cross referencing more than 50 data sources, including bookings from major global distribution systems (GDSs) such as the Sabre® GDS, fare data from the Sabre GDS, and government and other agencies that report traffic at various levels.
- Applying algorithms to estimate demand and fill gaps in areas with low GDS coverage.
Algorithms used in Global Demand Data are continuously updated with enhancements and new data sources as they become available. Extensive validation ensures consistency of the data from one period to another, and accuracy benchmarks with airline-provided data are used to continuously assess and improve the approach and process.
Additionally, Global Demand Data can be enriched with post-departure traffic data from an airline, as well as additional airline-subscribed GDS data, to increase solution accuracy.
Today, several airlines, aircraft manufacturers and airports use Global Demand Data to power their business intelligence and network planning functions.
Importance Of Calibration
Calibration is the foundation of good decision making. A poorly calibrated system will inevitably lead to suboptimal decisions.
A recent study conducted by Sabre Airline Solutions, which was presented to the AGIFORS Scheduling & Strategic Planning study group, quantified the impact of a poorly calibrated system on decision making.
The study utilized a benchmark-calibrated parameter set and multiple “bad” parameter sets to create different forecast sets. These forecasts were used to perform fleet- and hub-timing optimization to generate a new schedule. Each schedule was then evaluated against the benchmark-parameter set to measure the reduction in forecasted revenue opportunity.
Results for a mid-sized hub-and-spoke carrier showed that a 3 percent increase in leg-traffic error due to incorrect market sizes led to a 0.5 percent loss in total revenue opportunity. The study also tracked a 0.4 percent drop in revenue opportunity when the quality of connections built by the system dropped by 8 percent.
Data Sources And Calibration Methods
The data-collection and data-validation processes represent important steps in ensuring the success of the calibration process.
Typical market-data sources include:
- Passenger name record (PNR) data, also known as Marketing Information Data Tapes (MIDT), from GDSs help estimate itinerary-level passenger preferences and correctly establish connection-builder rules.
- Population-adjusted overall-market-size data, such as Global Demand Data, that adjusts for direct-channel and low-cost-carrier bookings are critical as airlines push harder on their direct sales.
- Industry data including airline schedules, industry minimum connect time (IMCT) data, aircraft seat configuration and airport locations.
- Market intelligence that encompasses data from airports or government agencies, competitive-fare information, etc.
- Internal airline data such as ticket coupon data from revenue accounting and post departure traffic data from the departure control system are critical to compute origin-and-destination (O&D) fares and yields, as well as validate passenger loads and flows on the host airline’s network.
Typically, calibration of data inputs involves estimating parameter sets at a market level — or for a logical group of markets, referred to as an “entity” — using various statistical and model-estimation techniques such as linear and logistic regression, clustering, maximum-likelihood and demand based on sampling techniques.
Extensive Validation Process
Two important factors determine the usability of the calibrated parameters:
- Robustness of the parameters during schedule changes and network expansion,
- Accuracy of the forecasted results when compared to historical data.
However, be advised: Maintaining robustness may lead to calibration of parameters at high levels of aggregation — possibly at the entity or system levels — that can reduce accuracy for some individual markets.
On the other hand, calibrating all parameters at a market level will be time consuming and can lead to bias toward the current market conditions and may not allow for optimal performance under future schedule changes.
Accuracy of forecasted results is measured by comparing system performance to historical data for the following attributes:
- Connection-builder results validated against historically observed connections
- Market-share-model accuracy
- Traffic estimates against post-departure traffic data and passenger flows using industry data (such as MIDT and internal data such as revenue accounting).
Accuracy targets are specified for key performance indicators (KPIs) defined for each of the above attributes, and Profit Manager is expected to conform to these targets. During the calibration process, KPIs must be reviewed continuously, and the relevant parameters are adjusted interactively to refine the calibration.
Accuracy is further improved by evaluating and identifying markets requiring parameter overrides and adjustments.
Market-level problems may indicate a pattern that affects a larger group of markets and should be thoroughly analyzed.
Key considerations in applying market-level overrides include size and schedule stability of the market, as well as the quality of data used to validate the market. Robustness can be further established by testing Profit Manager with predetermined “what-if” scenarios.
These scenarios may include the addition of new markets, frequency changes, codeshare and alliance partnerships, hub redesign (using a solution such as Sabre® AirVision™ Network Manager) or refleeting (using a solution such as Sabre® AirVision™ Fleet Manager).
Achieving Optimal Results
The calibration service offered by Sabre Airline Solutions provides airlines with two sets of calibrated parameters each year. Specific deliverables include market-demand and fare data, as well as parameter sets for control of itinerary building, passenger-preferences modeling and spill-and-recapture modeling.
Assistance is also provided in extracting schedule and industry information from data feeds subscribed to by individual airlines, such as minimum connect times, aircraft seat configurations and other relevant data, for use in Profit Manager.
Forecast-quality measurement is an integral part of the process. Reports measuring several KPIs at multiple levels of detail are shared with individual airlines.
When Sabre Airline Solutions personnel perform calibration for an airline, results are presented to the airline using standard templates that can help identify problem areas that must be addressed. These templates are reviewed with the airline, and methodologies to address identified issues are discussed and applied.
The technology company’s calibration experts work closely with an airline’s planning department to better understand the airline’s network and data sources, which are key in the performance and delivery of calibration.
Calibrations performed by Sabre Airline Solutions benefit airlines by:
- Eliminating the necessity for a dedicated in-house calibration team. As a result, airline resources can be deployed in more strategic areas such as network planning and market development, and recruiting can focus on hiring candidates with planning experience rather than operations research, statistics and other technical areas specific to calibration processes.
- Ensuring timely delivery of calibrated parameters for each season. An in-house calibration team may have other, more urgent responsibilities that relegate calibration tasks to a lower priority level within an airline’s planning area.
- Providing consistent high-quality calibration processes and outputs, including validating data quality and consistency among various sources, ensuring regular industry data updates, support for setting up user-defined inputs and evaluating final output parameters.
- Continuously adopting evolving calibration methodologies and common best practices.
Airlines typically appoint one or two users as liaisons between their larger user teams and the Sabre Airline Solutions calibration experts to ensure effective, full-operational adoption and incorporation of customer feedback in the calibration process.
Primary responsibilities of airline users include:
- Internal and external data collection and transmission to Sabre Airline Solutions;
- Attendance at meetings to discuss status, progress and issues during the calibration process;
- Receive and validate calibrated parameters as well as assist user teams with parameter adoption.
Support For In-House Calibrations
Several airlines currently using Profit Manager have chosen to invest in their own calibration processes and operations research teams. Sabre Airline Solutions supports these airlines in a variety of ways such as:
- Specialized customer care that addresses system issues and questions related to data sources and processes,
- Standardized health checks to review the end-to-end calibration process,
- Training and refresher training for current and new users when enhancements to calibration modules are introduced,
- Parameter updates for one or more seasons incorporating the latest methodologies.
Unleashing The Potential Of Calibration
Consistent and accurate calibration is an integral part of the planning process, enabling airlines to produce robust, accurate schedule evaluations.
To help airlines effectively compete in the marketplace, Sabre Airline Solutions continually invests in operations research expertise and calibration-process improvements and capabilities, such as Global Demand Data.
These investments create the foundation for decision-support systems that perform advanced network optimizations focused on improving airline schedules and unleashing the power of network planning.