Hossein Karami

University of Tehran, English Language and Literature, Faculty Member

Followers

515

Following

Co-authors

Public Views

I’m an Assistant Professor at the English Department, University of Tehran. My areas of interest include validity and fairness, especially in the context of language testing. My research has been published in various international scholarly journals including Language Testing, International Journal of Bilingual Education and Bilingualism, Educational Research and Evaluation, RELC Journal, Psychological Test and Assessment Modeling, TESOL Journal, and Asia-Pacific Education Review. My edited book, Fairness Issues in Educational Assessment, has been recently published by Routledge.

less

Interests

Uploads

Books by Hossein Karami

Fairness Issues in Educational Assessment

by Hossein Karami and Magdalena Mok

Download

Papers by Hossein Karami

The Development and Validation of a Bilingual Version of the Vocabulary Size Test

RELC Journal, 2012

This paper reports an attempt to develop and validate a bilingual Persian version of the Vocabula... more This paper reports an attempt to develop and validate a bilingual Persian version of the Vocabulary Size Test (VST). Due to the particular educational system in Iran, there is a dire need for a test that can effectively estimate English learners’ vocabulary sizes. Previous research ( Nguyen and Nation, 2011 ) has indicated that bilingual versions of the VST can be more efficient than the monolingual one. A calibration of the Persian version of the test with 190 English learners indicated that the test enjoys a high level of validity and reliability. The results of a factor analysis revealed a single construct, presumably word knowledge, is underlying the test. A one-way between-subjects ANOVA also indicated that the test can effectively distinguish between different proficiency levels. The hypothesized difficulty order was also realized in the data though it was found that clusters of 1,000 word levels provide more meaningful difficulty levels as they are less susceptible to the idi...

Developing and Validating a New Version of an EFL Multiple-Choice Reading Comprehension Test Based on Fuzzy Logic

Multiple-choice tests do not assess examinees’ knowledge in accord with reality. In fact, the par... more Multiple-choice tests do not assess examinees’ knowledge in accord with reality. In fact, the partial knowledge of examinees is not assessed. Providing a new approach to the assessment of reading comprehension in the framework of fuzzy logic, this study aims to measure this partial knowledge. In this approach, participants have to choose as many correct options as there are considering the stem. Therefore, the correct answer to each question can range from one option to all options. For the first session, an expository and an argumentative genre, and for the second session, the reading section of a TOEFL test was used. The results showed that the approach is fairer as it considers the partial knowledge of the examinees while in other common multiple-choice tests this is ignored. Also, the use of idea units as the units comprising a text gives us clues regarding the degree of difficulty of different parts of a text as well as clues about why misunderstanding may occur of the same tex...

Download

Fairness Issues in Educational Assessment

Examining the validity of the Achievement Emotions Questionnaire for measuring more emotions in the foreign language classroom

by Hossein Karami and Hossein Davari

Journal of Multilingual and Multicultural Development, 2020

Examining the psychometric features of the Teacher's Sense of Efficacy Scale in the English-as-a-foreign-language teaching context

Though the Teacher's Sense of Efficacy Scale (TSES) is widely used with English-as-a-foreign-lang... more Though the Teacher's Sense of Efficacy Scale (TSES) is widely used with English-as-a-foreign-language (EFL) teachers, these teachers constitute a cohort left out from the widespread project to validate the scale in different fields of education. In addition, the previous validation studies have mostly applied factor analysis for this purpose, which leaves some aspects of validity untouched. Thus, through factor and Rasch analysis of the data from 435 Iranian EFL teachers, the present study was undertaken to provide evidence on the validity of using the TSES with EFL teachers. Both confirmatory factor analysis and Rasch modeling confirmed the claimed three-factor structure of the TSES; however, Rasch analysis identified a misfitting item and frequent problems with the functioning of the response categories. In search of an alternative rating scale for the TSES, Rasch analysis supported the validity of a revised rating scale with five categories. Both traditional reliability and Rasch-specific indices showed that the revised rating scale was very similar to the original rating scale of the TSES in terms of measurement precision.

format_quoteCFA results support a three-factor model for the TSES, demonstrating better fit than a unidimensional structure (χ²=590, p<.0001).format_quote

Download

Validation of a bilingual version of the vocabulary size test: comparison with the monolingual version

by Hossein Karami and Mohammad Hossein Kouhpaee nejad کوهپایی نژاد

This study was set to cross-validate a bilingual Persian-English version of the Vocabulary Size T... more This study was set to cross-validate a bilingual Persian-English version of the Vocabulary Size Test (VST) against the monolingual English version and compare Iranian EFL learners' performance on the two versions. Various bilingual versions of the VST have been developed based on the assumption that bilingual versions are not affected by the grammar and reading demands of the long options in the monolingual version. To serve the purposes of the study, the Persian-English version and monolingual version of the VST were administered to 116 Iranian EFL learners. Results indicated that a single dimension was underlying both versions. Combined factor analysis indicated that both versions assessed the same construct. Further, both versions were capable of distinguishing learners of varying English proficiency levels and there was also a rough order of difficulty across frequency levels in both of them. Separate paired-samples t-tests revealed that the low-and mid-proficiency groups had differential performance on the two versions of the VST. Their L2 vocabulary knowledge was significantly underestimated by the monolingual version as shown by the mean-differences between the two versions of the VST for these two groups. In contrast, the results also revealed that the high-proficiency group in the study did not show such differential performance on the two version of the VST as the mean-difference between the two versions of the VST did not reach statistical significance for this last group. Hence, it is argued that advanced learners are competent enough so that their performance is not affected by grammar and reading demands of the long options in the monolingual version.

Download

The impact of background knowledge on test performance: A multivariate G-theory approach

Validity has been declared to be the single most important consideration in language testing and ... more Validity has been declared to be the single most important consideration in language testing and educational measurement. Messick (1989) famously identified construct-irrelevant variance as one of the major threats against test validity. Hence, test users ought to make every possible effort to make sure that test scores are not unduly affected by construct-irrelevant factors. This study applied multivariate Generalizability Theory to examine the effect of academic background on test scores obtained from the Iranian national university entrance exam. The results revealed that the relative contribution of the various sources of variance was not the same across the academic background groups. In addition, dependability indices were significantly different across the groupings Furthermore, the decision studies revealed that the groups do not necessarily need to take the same number of items so that a high reliability is obtained. Overall, the results indicate that academic background exerts a remarkable influence on the dependability of the scores.

format_quoteThe study addresses how academic background impacts score dependability of a high-stakes exam taken by over 100,000 applicants.format_quote

Download

Exploratory Factor Analysis as a construct validation tool: (mis)applications in applied linguistics research.

Factor analysis has been frequently exploited in applied research to provide evidence about the u... more Factor analysis has been frequently exploited in applied research to provide evidence about the underlying factors in various measurement instruments. A close inspection of a large number of studies published in leading applied linguistic journals shows that there is a misconception among applied linguists as to the relative merits of exploratory factor analysis and principal components analysis (PCA) and the kind of interpretations that can be drawn from each method. In addition, it is argued that the widespread application of orthogonal, rather than oblique, rotations and also the criteria used for factor selection are not in keeping with the findings in psychometrics. It is further argued that the current situation is partly due to the fact that PCA and orthogonal rotation are default options in mainstream statistical packages such as SPSS and the guidebooks on such software do not provide an explanation of the issues discussed in this article.

format_quoteOnly 35% of studies examined used sample sizes above 300, highlighting serious concerns about the validity of factor analyses.format_quote

Download

Review of “Rasch analysis in the human sciences” by Boone, W.J., Staver, J. R., & Yale, M. S. (2014)

Download

The Quest for Fairness in Language Testing

The search for fairness in language testing is distinct from other areas of educational measureme... more The search for fairness in language testing is distinct from other areas of educational measurement as the object of measurement, that is, language, is part of the identity of the test takers. So, a host of issues enter the scene when one starts to reflect on how to assess people's language abilities. As the quest for fairness in language testing is still in its infancy, even the need for such a research has been controversial, with some (e.g., Davies, 2010) arguing that such research is entirely in vain. This paper will provide an overview of some of the issues involved. Special attention will be given to critical language testing (CLT) as it has had a large impact on language testing research. It will be argued that although CLT has been very effective in revealing the ideological and value implications of the constructs of focus in language testing, extremism in this direction is not justified.

Download

Guest Editorial: Fairness Issues in Educational Assessment

by Hossein Karami and Magdalena Mok

Download