Subscribe Bookmark
louv

Staff

Joined:

Jun 23, 2011

How On-Time Are the Airlines? A Frequent Flier's Story

During my last trip home from a JMP customer visit, my flight was delayed getting out of Raleigh, North Carolina, due to mechanical problems. This caused me to miss my connection in Philadelphia, and I ended up with a six-hour layover awaiting the next flight home to Rochester, New York, on a Friday.


As I sat patiently in the US Air lounge, I couldn’t help but notice the headline in a newspaper that read, “Airline on-time performance improves in April.” In my usual data-driven fashion, I decided to see if I could get some historical data to check out this claim. Sure enough, after some searching I found data at the Bureau of Transportation Statistics website that allowed me to pursue an analysis. After downloading the on-time airline data from 1995 through 2010 and performing the usual data cleansing, I chose to examine a control chart of the performance:




Sure enough, the Control Chart of the data shows some interesting patterns. At first glance, one notices that the low outliers occur frequently in December, which is not surprising because of the holidays and the potential for bad winter weather. In addition, it also shows that after 9/11/01 there was a significant mean shift upward in performance that lasted through 2003.


The performance then steadily deteriorated downward from 2003 through 2008, and it does appear that in late 2008 through April 2010 there may have been a performance improvement. The numbers displayed in the Control Chart show the alarms that indicate an out-of-control situation. Of course, I believe all travelers would love to see out-of-control alarms above the 3 sigma UCL (Upper Control Limit) as there was in November of 2009.


Well, I didn’t stop there. I also decided that an analysis of the on-time performance by month might be an interesting exercise. So I added a categorical column for month and ran a Fit Y by X where Y was the on-time performance and X was month, and I observed the following analysis:




I am not surprised to see that the months of December and January were not stellar-performing months, and I do clearly see the outlier in September 2001. But I do find it curious that the month of June fell in the same category as December and January, and it made me think that perhaps the summer months may have lower staffing due to vacations.


An additional piece of data that might be useful is the number of scheduled flights; if there were fewer flights scheduled, perhaps the on-time performance would be improved. A Control Chart of that data is below:




Could it be that the on-time performance is somehow related to the scheduled number of flights and that the most recent improvement may be due to that factor? I decided to model the on-time performance versus three factors: year, month and number of flights. I obtained the following output in the Prediction Profiler. It appears that the number of scheduled flights also plays into the on-time performance metric:




One final note: All of this analysis assumes that the flight time from destination to destination, which would be the specification to which on-time performance would be measured, was constant throughout this period. Perhaps that is the next piece of data that would be required to help crystallize this claim. I’ve uploaded the data set to the JMP File Exchange for all you fliers out there in case you would like to play with it. Let me know if you make any interesting discoveries.

9 Comments
Community Member

Dean Abbott wrote:

There are of course many factors involved in delays, and the definition of "delay" (as Jim Metcalf indicates) depends on which end of the trip you mean. Also, the departure delay does not include taxi out time which can be used by airlines to pad on-time statistics.

Other factors include time of day, other air traffic issues, weather, etc. Addison Schonland and I made a go of it for a while to package this information at http://www.airportbutler.com (don't worry--this isn't an informercial...the site is still up be we aren't updating anything).

On that site are some reports we built that show similar kinds of information. For example, the JFK report showed that while JetBlue and Delta had similar on-time performance, JetBlue departed on average 11 minutes sooner than Delta.

Two things I found particularly interesting:

one was the role of gate location in total flight time as some airlines have gate locations close to the runway which minimizes their taxi out time. The other was the role of time of day. Jet Blue does very well at JFK for instance in part because of when they fly (they did better with ontime performance even at peak periods as well). The JFK sample report (http://www.airportbutler.com/jfkdsample.pdf) shows the time-of-day effect dramatically.

Community Member

Lou V wrote:

That is certainly part of it. There is indication of special cause variation in Jan 96, and Dec 00, 07, and 08 (indicated in the JMP output with "1" which tell us it violates Western Electric test 1) presumably due to winter weather. We know deicing of planes can impact this metric considerably. However the biggest assumption that I mentioned is the data presumes that the specification, namely standard flight duration for a given trip was a constant throughout the time period. I do not know if that was indeed the case. The control chart is certainly useful to discover trends but in this case I'm not sure I would use it as a stand alone analytic method to make conclusions without more data. That is the reason I gathered additional data around the number of flights since this would play out too on the metric used.

Community Member

Sonya wrote:

Looks like normal variation til 9/11 then no one flew so improvement, then we started flying again, things got worse, do you think we are heading back into state of normal variation?

Community Member

Bill MacKrell wrote:

Lou, nice work! I enjoyed this article. A couple comments that may help:

1. There is some slack involved in published flight durations. Assuming a flight leaves on time and encounters no en-route delays, the published flight durations allow more than enough time to complete the flight. I think the FAA also lets the airlines claim "on time" if the flight lands within 15 minutes of the published ETA. Not sure how you would factor this in.

2. Since airlines operate under Instrument Flight Rules, there are a limited number of routes they can use to fly between two points. They can't make it up as they go along. So your assumption about constant flight times is probably reasonable, except for variability introduced by weather or traffic.

3. The final analysis where you model on-time performance against number of scheduled flights makes perfect sense. The more scheduled flights, the more holds the FAA has to use to maintain spacing and separation. This is pronounced in high-volume places like NYC.

Community Member

raf wrote:

neat I like the addition of number of flights. I would assume delays in June are due to weather, server thunderstorms and tornados play a huge part.

One thing of note in the data.

"On-Time" is an interesting number:

1. For take off On-time means you left the gate on time

2. For landing On-time means you landed on time.

How many times have you sat on a tarmak? or sat in the plane waiting for the gate? One time I landed early but ended up arriving at the gate late.

And because of this metric airlines tend to over estimate the landing time this also improves performance...

but hey what are you going to do

either way great job...good use of analytics to show what is driving the numbers

Community Member

Jim Metcalf wrote:

Hi Lou,

This is -really- interesting! Fun to read and a great post. Regarding the June decrease in on-time arrivals: I'll speculate it's because of afternoon thunderstorms. I never schedule flights in the afternoon in the summer because of this. It would be interesting to split the flights into before noon and after noon to see if the flights before noon are more on time than those that depart in the afternoon.

Community Member

Rick wrote:

The on-time arrival/departure of airlines was the topic of the ASA 2009 Data Expo poster session. You can read about one effort that uses SAS software to analyze historial data at

http://support.sas.com/rnd/app/papers/airlinedelays.pdf

You can browse the electronic version of several of the ASA posters at

http://stat-computing.org/dataexpo/2009/posters/

Community Member

Lou Valente wrote:

Thanks for the feedback. I am glad that you enjoyed it!

Community Member

Dion Newgard wrote:

Flying becomes a concern for some people right now due to the current disasters happen to Malaysia Airlines. anyway, this is a great post, thanks for sharing experience, you tell it in a very informative way with the graphs and interpretation. really understandable and interesting.