Description:
Statistical inference procedures in real situations often assume not only the basic assumptions on which the justification of the inference is based, but also some additional assumptions which could affect the justification of the inference. In many cases, the inference procedure is too complicated to be evaluated analytically. Simulation-based evaluation for such an inference could be an alternative, but generally simulation results would be valid only under specific simulated circumstances. For simulation in the parametric model set-up, the simulation result may depend on the chosen parameter value. In this study, we suggest an evaluation methodology relying on an observation-based simulation for frequentist and Bayesian inferences on parametric models. We describe our methodology with the suggestions for three aspects: factors to be measured for the evaluation, measurement of the factors, and evaluation of the inference based on the factors measured. Additionally, we provide an adjustment method for inferences found to be invalid. The suggested methodology can be applied to any inference procedure, no matter how complicated as long as it can be codified for repetition on the computer. Unlike general simulation in the parametric model, the suggested methodology provides results which do not depend on specific parameter values.The argument about the new EPA standards for particulate matter (PM) has led to some statistical analyses for the effect of PM on mortality. Typically, regression model of mortality of the elderly is constructed. The covariates of the regression model includes a suite of highly correlated particulate matter related variables in addition to trend and meteorology variables as the confounding nuisance variables. The classical strategy for the regression model having a suite of highly correlated covariates is to select a model based on a subset of the covariates and then make inference assuming the subset model. However, one might be concerned about the validity of the inferences under his classical strategy since it ignores the uncertainty in the choice of a subset model. The suggested evaluation methodology was applied to evaluate the inference procedures for the PM-mortality regression model with some data from Phoenix, Arizona taken from 1995 to early 1998 . The suggested methodology provided valid evaluation results for various inference procedures and also provided adjusted inferences superior to Bonferroni adjustment. Based on our results, we are able to make conclusions about the short-term effect of PM which does not suffer from validity concerns.