Aerospace Medicine Technical Reports
FAA Office of Aerospace Medicine
Civil Aerospace Medical Institute
Report No: DOT/FAA/AM-05/15
Title and Subtitle: Pilot Willingness to Take Off Into Marginal Weather, Part II: Antecedent Overfitting with Forward Stepwise Logistic Regression
Report Date: August 2005
Authors: Knecht WR
Abstract: Adverse weather is the leading cause of fatalities in general aviation (GA). In prior research, influences of ground visibility, cloud ceiling height, financial incentive, and personality were tested on 60 GA pilots� willingness to take off into simulated adverse weather. Results suggested that pilots did not see �weather� as a monolithic cognitive construct but, rather, as an interaction between its separate factors. However, methodological issues arose during the use of logistic regression in modeling the effect of 60+ candidate predictors on the outcome variable of takeoff into adverse weather.
It was found quite possible to obtain false �significance� for models comprised merely of random numbers, even when the number of model predictors was limited to a conventional 1/10. Therefore, Monte Carlo simulations were used to derive unbiased estimates of model significance and R2 values. Research in correction for this case/candidate predictor ratio effect is relatively new and noteworthy, particularly in the social sciences. It was given the name �antecedent overfitting� to contrast with the more commonly known �postcedent� type, which is based on a small case/model predictor ratio.
Key Words: Statistics, Methodology, Overfitting, Case/Predictor Ratio, Regression
No. of Pages: 17
Civil Aerospace Medical Institute
Report No: DOT/FAA/AM-05/15
Title and Subtitle: Pilot Willingness to Take Off Into Marginal Weather, Part II: Antecedent Overfitting with Forward Stepwise Logistic Regression
Report Date: August 2005
Authors: Knecht WR
Abstract: Adverse weather is the leading cause of fatalities in general aviation (GA). In prior research, influences of ground visibility, cloud ceiling height, financial incentive, and personality were tested on 60 GA pilots� willingness to take off into simulated adverse weather. Results suggested that pilots did not see �weather� as a monolithic cognitive construct but, rather, as an interaction between its separate factors. However, methodological issues arose during the use of logistic regression in modeling the effect of 60+ candidate predictors on the outcome variable of takeoff into adverse weather.
It was found quite possible to obtain false �significance� for models comprised merely of random numbers, even when the number of model predictors was limited to a conventional 1/10. Therefore, Monte Carlo simulations were used to derive unbiased estimates of model significance and R2 values. Research in correction for this case/candidate predictor ratio effect is relatively new and noteworthy, particularly in the social sciences. It was given the name �antecedent overfitting� to contrast with the more commonly known �postcedent� type, which is based on a small case/model predictor ratio.
Key Words: Statistics, Methodology, Overfitting, Case/Predictor Ratio, Regression
No. of Pages: 17
Last updated: Friday, June 1, 2012