Sains Malaysiana 49(4)(2020): 885-898

http://dx.doi.org/10.17576/jsm-2020-4904-18

Pengujian Hipotesis Berbilang: Perbandingan Simulasi Monte Carlo Berdasarkan Ralat Jenis I

(Multiple Hypothesis Testing: Comparison of Monte Carlo Simulation Based on Type-1 Error)

NORA MUDA* & NOR SYAFAWATI JANI

Jabatan Sains Matematik, Fakulti Sains dan Teknologi, Universiti Kebangsaan Malaysia, 43600 UKM Bangi, Selangor Darul Ehsan, Malaysia

Received: 16 July 2019/Accepted: 27 December 2019

ABSTRAK

Pengujian hipotesis berbilang merupakan pengujian yang melibatkan ujian serentak lebih daripada satu hipotesis dan digunakan untuk mengawal kadar ralat berkumpulan (FWER) dan kadar penemuan palsu (FDR) dengan meminimumkan Ralat Jenis I. Kajian ini bertujuan untuk membuat perbandingan ujian pengujian hipotesis berbilang bagi ujian-t iaitu pengujian antara dua kumpulan sampel melalui perbandingan antara ujian Bonferroni, ujian Holm, ujian Hochberg, ujian Hommel, ujian Benjamini-Hochberg dan ujian Benjamini-Yekutieli dengan mengikut keadaan yang tertentu iaitu nilai α, bilangan ujian, m dan jenis taburan yang berbeza. Perbandingan pengujian hipotesis berbilang berdasarkan kebarangkalian Ralat Jenis I bagi kes kadar ralat berkumpulan (FWER) dan kadar penemuan palsu (FDR) dijalankan berdasarkan simulasi Monte Carlo. Didapati, bagi kumpulan min yang sama iaitu {0,0} bagi kesemua keadaan, Ralat Jenis I bernilai sifar. Hal ini kerana kesemua ujian gagal menolak hipotesis nol dan terbukti menyatakan kesemua hipotesis nol adalah benar. Selain itu, aras keertian 0.01 tidak sesuai digunakan bagi kesemua keadaan kerana aras keertian ini dikatakan sangat jitu. Bagi kumpulan min yang berbeza iaitu {0,1}, ujian Benjamini-Yekutieli sesuai digunakan bagi mengawal kadar penemuan palsu (FDR) kerana dapat meminimumkan Ralat Jenis I dengan baik berbanding dengan ujian lain. Manakala bagi kadar ralat berkumpulan (FWER), ujian Hommel sesuai digunakan berbanding dengan ujian lain. Hal ini kerana ujian ini dapat mengawal dengan baik dan meminimumkan Ralat Jenis I.

Kata kunci: Kadar penemuan palsu; kadar ralat berkumpulan; ujian Benjamini-Hochberg; ujian Bonferroni; ujian Holm

ABSTRACT

Multiple hypothesis testing is a test that involves more than one hypothesis test which run simultaneously and is used to control group error rate (FWER) and false discovery rate (FDR) by minimizing Type I Error. This study aims to compare multiple hypothesis testing tests for t-test; test between two group samples by comparing between Bonferroni test, Holm test, Hochberg test, Hommel test, Benjamini-Hochberg test, and Benjamini-Yekutieli test according to specific conditions namely α value, number of tests, m and different types of distribution. Comparison of multiple hypothesis testing based on probability of Type I error for group error rate (FWER) and false discovery rate (FDR) was performed based on Monte Carlo simulation. It is found that for the group with that same mean {0,0} in all cases, the Type I error is zero. This is because all tests failed to reject the null hypothesis and proved that all null hypotheses were true. Also, the significance level of 0.01 is not appropriate for all situations because it is said to be very accurate. For different mean groups of {0,1}, the Benjamini-Yekutieli test is best used to control the false discovery rate (FDR) as it minimizes Type I error better than other tests. For group error rates (FWER), the Hommel test is applicable compared to other tests. This is because this test can control and minimize Type I Errors.

Keywords: Benjamini-Hochberg test; Bonferroni test; false discovery rate (FDR); group error rate (FWER); Holm test

REFERENCES

Abdi, H. 2010. Holm's sequential Bonferroni procedure. Dlm. Encyclopedia of Research Design, disunting oleh Salkind, N. Thousand Oaks, CA: Sage.

Abdul Rahman Othman. & Lai Choo Heng. 2014. Sensitivity analysis of the refinement to the Mann-Whitney Test. Sains Malaysiana 43(7): 1095-1100.

Aickin, M. & Gensler, H. 1996. Adjusting for multiple testing when reporting research results: The Bonferroni vs Holm methods. America Journal of Public Health 86(5): 726-728.

Benjamini, Y. & Yekutieli, D. 2001. The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics 29: 1165-1188.

Benjamini, Y. & Hochberg, Y. 1995. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological) 57(1): 289-300.

Bradley, J.V. 1978. Robustness. British Journal of Mathematics and Statistical Psychology 31: 144-151.

Dudoit, S., Shaffer, J.P. & Boldrick, J.C. 2003. Multiple hypothesis testing in microarray experiments. Statistical Science 18(1): 71-103.

Godfrey, K. 1985. Comparing the means of several groups. New England Journal of Medicine 311: 1450-1456.

Hochberg, Y. 1988. A Sharper Bonferroni procedure for multiple tests of significance. Biometrika 75(4): 800-802.

Holm, S. 1979. A simple sequentially rejective multiple test procedure. Scandanian Journal of Statistics 6: 65-70.

Hommel, G. 1988. A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75(2): 383-386.

Marcus, R., Eric, P. & Gabriel, K.R. 1976. On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63(3): 655-660.

Nor Aishah Ahad, Abdul Rahman Othman. & Sharipah Soaad Syed Yahaya. 2012. Performance of two-samples pseudo-median procedure. Sains Malaysiana 41(9): 1149-1154.

Pocock, S.J., Hughes, M.D. & Lee, R.J. 1987. Statistical problems in the reporting of clinical trials. New England Journal of Medicine 317: 426-432.

Perneger, T.V. 1998.  What’s wrong with Bonferroni adjustments. BMJ 316(7139): 1236-1238.

Simes, R.J. 1986. An improved Bonferroni procedure for multiple tests of significance. Biometrika 73(3): 751-754.

Sinclair, J.K., Taylor, P.J. & Hobbs, S.J. 2013. Alpha level adjustments for multiple dependent variable analyses and their applicability: A review. World Academic Press, World Academic Union 7(1): 017-020.

Smith, D.G., Clemens, J., Crede, W., Harvey, M. & Gracely, E.J. 1987. Impact of multiple comparisons in randomized clinical trials. The American Journal of Medicine 83(3): 545-550.

Wright, S.P. 1992. Adjusted P-values for simultaneous inference. Biometrics 48: 1005-1013.

*Corresponding author; email: noramuda@ukm.edu.my