|
Abstract
|
The publisher regrets that there were some editorial discrepancies in the original article. Below you'll find the exchange between the FRA and the authors of the article. Only minor editorial changes were made to the article.
The publisher would like to apologize for any inconvenience caused.
From the United States Department of Transportation, Federal Railroad Administration
The United States Department of Transportation, Federal Railroad Administration (FRA) firmly denies the allegation by Jacob Mathew and Professor Rahim Benekohal that the "New Model for Highway-Rail Grade Crossing Accident Prediction and Severity" (New APS) adopted any of their work. FRA determined that the Mathew-Benekohal articles relied on an outdated 3-equation approach and that their small sample sizes were inadequate. For a complete list of the resources used to develop the New APS, see full report, Section 8. References, page 50, posted here: https://railroads.dot.gov/elibrary/new-model-highway-rail-grade-crossing-accident-prediction-and-severity.
Mathew and Benekohal's response to The United States Department of Transportation, Federal Railroad Administration (FRA)'s alleged errata for the article "Highway-rail grade crossings accident prediction using Zero Inflated Negative Binomial and Empirical Bayes method".
Why we say the new FRA model was adopted from the work we had done previously.
The reasons we say the new FRA model was adopted from the work we had done previously are threefold.
A.
We had advocated for the Zero Inflated Negative Binomial model with Empirical Bayes Adjustment for a while. Before publishing this article in JSR, we had worked on developing the ZINB model and published the following 4 study documents that are easily available:
1.
Mathew J, Benekohal. R. F., "A New Accident Prediction Model for Highway-Rail Grade Crossings Using the USDOT Formula Variables". J. Traffic and Transportation Eng [Internet]. 2020;8(1):1-13. Available from: http://www.davidpublisher.com/Public/uploads/Contribute/5f43533833a24.pdf
2.
Mathew J, Benekohal RF, Medina J. C., "Accident Prediction Models using Macro and Micro Scale Analysis: Dynamic Tree and Zero Inflated Negative Binomial Models with Empirical Bayes Accident History Adjustment". 2019; Available from: https://conservancy.umn.edu/bitstream/handle/11299/201747/CTS 19-02.pdf
3.
Medina J. C, Shen S, Benekohal R. F. "Micro and Macro Level Safety Analysis at Railroad Grade Crossings" [Internet]. 2016. Available from: https://www.nurailcenter.org/research/final_reports/UIUC/NURail2012-UIUC-R02_Final_Report_Benkohal.pdf
4.
Medina J. C, Benekohal R. F. "Macroscopic Models for Accident Prediction at Railroad Grade Crossings". Transp Res Rec J Transp Res Board [Internet]. 2015;2476:85-93. Available from: https://journals.sagepub.com/doi/10.3141/2476-12
B.
We had discussed the ZINB model with Empirical Bayes adjustment models with an FRA Engineer in change grade crossing safety research and had provided him the above 4 documents (initial version of document 1 and the final versions of the remaining three documents) prior to the conduct of the study that led to the new FRA technical report. The FRA Engineer is acknowledged in the FRA report - "The authors appreciate the insightful review comments by Federal Railroad Administration (FRA) staff Karen McClure and Francesco Bedini". After the FRA report was published, we complained to the FRA engineer that our publications are not even listed among the references, and he suggested that our publications should have been listed among the references for the FRA report. It was his suggestion that we send him the list of our papers to be added to the reference list of the FRA report. We did, but now FRA refuses to cite any of these publications.
C.
The validation approach used by the new FRA report is very similar to the approaches we had developed and used in our publications. Those three approaches are:
a.
Plotting the cumulative predicted value with respect to the field data.
b.
Plotting the predicted value with respect to the observed value
c.
Ranking of the crossings based on the predicted value and comparing to the field data.
The new FRA model has used the three approaches that we used with very slight modifications. The validation approach used by the FRA is given in Section 6 (page 39) of the article "New Model for Highway-Rail Grade Crossing Accident Prediction and Severity.".
"The first validation compares cumulative predicted accidents by the new model and the APS with the actual risk as measured by accident counts.
The second validation shows the predicted accidents for the new model and the APS for crossings grouped by accident count.
The third comparison examines the model results (the new model and APS) for different groupings of high-risk crossings and shows the results in a chart.".
Therefore, when the methodology and the validation approaches used are very similar to a previously published article, it is expected to cite the original articles and give proper credit to them. We do not claim that we developed statistical procedures like the Zero Inflated Negative Binomial method or the Empirical Bayes method. We, however, have been advocating using these two procedures together for the prediction of accidents at Highway-Rail Grade Crossings for a while.
In our correspondence with Ms. Karen McClure of FRA, she claimed "FRA used the normalizing constants approach developed by the U.S. DOT in 1987 to validate the New APS model." Using normalization constant is not a validation procedure but rather an attempt to adjust the predictions to changes in the conditions over time. It should not be mistaken for a validation procedure. This is explained in the report titled "FRA Collects Reliable Grade Crossing Incident Data but Needs To Update Its Accident Prediction Model and Improve Guidance for Using the Data To Focus Inspections". Page 9 of this report states that "These constants keep the accident predictions matched with current accident trends, numbers of open public grade crossings, and changes in warning devices. Each constant is a ratio of the actual number of accidents to the predicted number of accidents. The constant used in the formula depends on the type of warning device at the pertinent grade crossing--a passive warning device, a flashing light, or a gate".
In a later correspondence with Ms. McClure, she claimed "The New APS validation methods (cumulative distribution and comparison of predicted to actual accident rates) have been used by FRA in the GradeDec model since 2001 and in the US DOT Accident Prediction and Severity Model since 1987.".
Regarding the use of validation approaches in the GradeDec model.
"The Federal Railroad Administration developed GradeDec.NET, a highway-rail grade crossing investment analysis tool, to provide grade crossing investment decision support." GradeDec model is "a web-based application and decision support tool for the identification and evaluation of highway-rail grade crossing upgrades, separations and closures."
https://railroads.dot.gov/program-areas/highway-rail-grade-crossing/gradedecnet-crossing-evaluation-tool
https://gradedec.fra.dot.gov/
We checked the following documents associated with GradeDec model that are available in the GradeDec website to check for Ms. McClure's claims:
1.
GradeDec.Net 2017 User's Manual.pdf. Available at https://railroads.dot.gov/elibrary/gradedecnet-2017-users-manual
2.
GradeDec.Net 2019 Reference Manual. Available at https://railroads.dot.gov/elibrary/gradedecnet-2019-reference-manual
3.
Workbook 2003. Available at https://railroads.dot.gov/elibrary/gradedec-crossing-evaluation-tool-workbook-2003
The word "validation" appears once in the GradeDec.Net 2017 User's Manual in Section 5.1, which is given below.
"This page possesses a number of features that let you easily visualize data and quickly develop probability distributions that best reflect available information and judgments on operations, future developments and social costs. These features include:
•
Ease of navigation among variables
•
Instant viewing of statistics and charts
•
Instant validation and saving of ranges"
We couldn't find any other reference to the validation of the GradeDec model in the GradeDec documentation.
Regarding the use of validation approaches in USDOT Accident Prediction and Severity model since 1987.
We found the following five articles related to the USDOT Accident Prediction and Severity model that were published in 1979, 1982, 1986, 1987, and 1987 respectively.
1.
Rail-highway crossing hazard prediction: research results. Available at https://rosap.ntl.bts.gov/view/dot/8584
2.
Summary of the Department of Transportation Rail-Highway Crossing Accident Prediction Formulas and Resource Model. Available at https://rosap.ntl.bts.gov/view/dot/11643
3.
Rail-Highway Crossing Resource Allocation Procedure. User's Guide. 2nd edition. Available at https://rosap.ntl.bts.gov/view/dot/10417
4.
Rail-Highway Crossing Resource Allocation Procedure User's Guide Third Edition. Available at https://railroads.dot.gov/elibrary/rail-highway-crossing-resource-allocation-procedure-users-guide-third-edition
5.
Summary of DOT Rail-Highway Crossing Resource Allocation Procedure - Revised. Available at https://rosap.ntl.bts.gov/view/dot/11385
Only in the first of the above five articles (Rail-highway crossing hazard prediction: research results), the author used a validation approach in which power factors (i.e., fraction of accidents occurring at a given fraction of the most hazardous crossings) and empirical operating characteristics (i.e., a table giving power factors, cumulative accidents at various percentages of hazardous crossings, etc.) to compare the model developed in the article to New Hampshire and Coleman-Stewart models. In none of the remaining four manuscripts, do the authors of the respective manuscripts describe a validation procedure.
After all, it is not our responsibility to prove that our approaches were used in a later study; it is the responsibility of the researchers conducting the new study to properly cite the previous study.
Regarding Sample Size
We would like to emphasize that we do not believe our sample size is small.
We used data from 10,292 crossings of which 5097 are in Illinois and 5195 are in Texas. The dataset from Illinois consists of 2755 datapoints for gates, 960 datapoints for flashing lights, and 1382 datapoints for crossbucks. The dataset from Texas consists of 3573 datapoints for gates, 346 points for flashing lights, and 1276 data points for crossbucks. We believe this a good sample size and yield a reliable model. Previous traffic safety studies that utilized the Empirical Bayes approach have used much smaller sample sizes to conduct their studies (a few of them are listed below).
1.
The study by Perusad et al. used Empirical Bayes approach to study stop controlled intersections in CA. The data consisted of 1669 stop-controlled intersections
Persaud, Bhagwant, and Craig Lyon. "Empirical Bayes before-after safety studies: lessons learned from two decades of experience and future directions." Accident Analysis & Prevention 39.3 (2007): 546-555. Available at: https://web.engr.uky.edu/~rsouley/CE%20635-2021/docs/Persaud%20-%2010%20years%20of%20EB.pdf.
2.
The study by Harwood et al, used Empirical Bayes to study rural two-lane highways. The data consisted of 619 segments from Minnesota and 712 from Washington.
Harwood, Douglas W., et al. Prediction of the expected safety performance of rural two-lane highways. No. FHWA-RD-99-207, MRI 4584-09, Technical Report. United States. Federal Highway Administration, 2000. Available at: https://rosap.ntl.bts.gov/view/dot/14465.
3.
Perusad et al. used 197 signalized intersections to develop an EB procedure to rank intersections.
Persaud, Bhagwant, Craig Lyon, and Thu Nguyen. "Empirical Bayes procedure for ranking sites for safety investigation by potential for safety improvement." Transportation research record 1665.1 (1999): 7-12.
Additionally, the notion that data form two large states (IL and TX) is not enough for modeling railroad grade crossing accident is baseless. Different states use their own data for modeling as well as planning and management of their system and the issue of sample size is not a concern when you use your own state's data.
Furthermore, we checked the adequacy of our sample size by fitting a model using half of the data (test-model) to see if it would drastically alter the coefficients of the model based on full data (which is our ZINEBS model). In Table 1, Table 2, Table 3 the coefficients of the test-models and ZINEBS models are shown. Table 1 is for the model for Gates, Table 2 is for the model for flashing lights, and Table 3 is for the model for crossbucks. In all the three cases, the coefficients of the test-models are similar to the coefficients of ZINEBS models. This clearly indicates that our sample size in more than adequate and is not small as the Ms. McClure claimed...
Language: en |