### Abstract

We study the performance of a screening procedure R of Chen, Melvin, and Wicks (1999) when it is adopted prior to the signal detection procedure R_{kkr} proposed independently by Kelly (1986) and Khatri and Rao (1987). Through simulation results, we show that the probability of detection for R_{kkr} is significantly improved when procedure R is first used to screen out nonhomogeneous data. Procedure R is a selection procedure which compares k (≥1) experimental populations with a control population and eliminates the dissimilar experimental populations. An experimental population with covariance matrix Σ is said to be similar to the control population with covariance matrix Σ_{0} if Σ_{0}Σ^{-1} is close to the identity matrix in certain meaning of closeness to be defined later in the paper. As mentioned in Chen et al. (1999), procedure R can be used as a screening process prior to any traditional signal processing detection algorithm which requires the assumption of the same covariance matrices for the experimental populations as for the control population. A commonly used such detection algorithm is the one proposed independently by Kelly (1986) and Khatri and Rao (1987). In this paper, we first simulate data from k + 1 complex multivariate normal populations which all have zero mean vectors and have different covariance matrices. One of the populations represents the control population and the remaining populations are the experimental populations. Then we apply procedure R to the simulated experimental populations to screen out the dissimilar populations. Finally, R_{kkr} is used to detect a target using respectively the unscreened and the screened data. We present simulation results on the powers of the test R_{kkr} when it is applied to the unscreened data and the screened data. The results illustrate that, under the nonhomogeneous environment where covariance matrices of the experimental populations are different from the covariance matrix of the control population, we can always improve the power of the test R_{kkr} by employing the procedure R. We also present an example illustrating the potential application of our study in medical imaging.

Original language | English (US) |
---|---|

Pages (from-to) | 119-128 |

Number of pages | 10 |

Journal | Biometrical Journal |

Volume | 42 |

Issue number | 1 |

DOIs | |

State | Published - Jan 2000 |

### Keywords

- Complex normal distribution
- Control population
- Covariance matrix
- Homogeneous
- Primary data
- Secondary data

### ASJC Scopus subject areas

- Statistics and Probability
- Statistics, Probability and Uncertainty