Educational evaluation

This article examines five common misunderstandings about case-study research: (a) theoretical knowledge is more valuable than practical knowledge; (b) one cannot generalize from a single case, therefore, the single-case study cannot contribute to scientific development; (c) the case study is most useful for generating hypotheses, whereas other methods are more suitable for hypotheses testing and theory building; (d) the case study contains a bias toward verification; and (e) it is often difficult to summarize specific case studies. This article explains and corrects these misunderstandings one by one and concludes with the Kuhnian insight that a scientific discipline without a large number of thoroughly executed case studies is a discipline without systematic production of exemplars, and a discipline without exemplars is an ineffective one. Social science may be strengthened by the execution of a greater number of good case studies.

Stephen M. Alessi is an Associate Professor at the University of Iowa, on Psychological and Quantitative Foundations Department. He received his Ph.D. in Educational Psychology form University of Illinois in 1979. His principal research interests are Educational Technology, Multimedia Design, Web-Based Learning and Instructional Simulation. Stephen M. Alessi is an active member of American Educational Research Association, Association for Educational Communications and Technology, European Association for Research on Learning and Instruction and System Dynamics Society.
Stanley R. Trollip is an Associate Professor at Capella University, Minneapolis. His main interests are Higher Education, Educational Assessment and Educational Technology. He has published more than 50 publications on topics like Computer based technology, Education methods and development, Classroom management, Educational Software. Stanley R. Trollip is an active member of American Educational Research Association, Association for Educational Communications and Technology

If high-stakes standardized testing were to be abolished, as the report authors might like, American society as a whole would be much worse off, and so would many individual students. Probably the most unfairly affected would be the high achievers among the poor.Wealthy families who value academics have the choice of moving to a school district where their high achieving children can excel, or sending their children to a private school. It is a waste of money and otherwise too bad that they feel they must move, but they can. Poor families are not so mobile.
  High achieving students who cannot leave a school district where academic achievement is undervalued face varied pressures that impede them: pressure to fit in and be popular; to excel at sports; to work at low-pay, dead-end jobs to earn money for cars and parties; and so on. If they study hard and excel at academics, they will be taunted; disliked; called “nerd,” “geek,” “dork”; or be accused of “acting White.”
  The report spends considerable effort worrying about the feelings of students who might fail high-stakes tests, but little if any effort worrying about the social fallout of abandoning high academic standards. High-achieving students among our poor should be considered our country’s most precious human resources.
  For a variety of reasons, our society very badly needs these students to prosper; so their gifts and ambitions should be nurtured, not discouraged. The report, however, in the effect of its recommendations, would have these students treated as pariahs and have them feel guilty for wanting to work hard and succeed. After all, if these students work hard and succeed, won’t that make other students who do not want to work hard look and feel bad?
  Abandoning the enforcement of high academic standards will not eliminate pressures and hurt feelings among our youth, however. Pressure and hurt feelings are facts of life. Abandoning academics just means the pressures will come from and the hurt feelings will be caused by nonacademic aspects of these students’ lives.
  Is that really what we want? In radical egalitarian bliss, there will be no high-stakes tests, no academic standards enforced in any meaningful way, and no academic tracking. Academic progress in every school and for every student will be slowed to the preferred pace of the least motivated student.

This is a study of distributed leadership in the context of elementary schools' adoption of comprehensive school reforms (CSR). Most CSRs are designed to configure school leadership by defining formal roles, and we hypothesized that such programs activate those roles by defining expectations for and socializing (e.g. through professional development) role incumbents. Configuration and activation were further hypothesized to influence the performance of leadership functions in schools. Using data from a study of three of the most widely-adopted CSR models, support was found for the configuration and activation hypotheses. Leadership configuration in CSR schools differed from that of non-CSR schools in part because of the addition of model-specific roles. Model participation was also related to the performance of leadership functions as principals in CSR schools and CSR-related role incumbents were found to provide significant amounts of instructional leadership. Further support for the activation hypothesis is suggested by positive relationships between leaders' professional development experiences and their performance of instructional leadership.

This article presents the theoretical and methodological considerations behind a research method which the author calls ‘phronetic planning research’. Such research sets out to answer four questions of power and values for specific instances of planning: (1) Where are we going with planning? (2) Who gains and who loses, and by which mechanisms of power? (3) Is this development desirable? (4) What, if anything, should we do about it? A central task of phronetic planning research is to provide concrete examples and detailed narratives of the ways in which power and values work in planning and with what consequences to whom, and to suggest how relations of power and values could be changed to work with other consequences. Insofar as planning situations become clear, they are clarified by detailed stories of who is doing what to whom. Clarifications of that kind are a principal concern for phronetic planning research and provide the main link to praxis.

Taken together, the works of Jurgen Habermas and Michel Foucault highlight an essential tension in modernity. This is the tension between the normative and the real, between what should be done and what is actually done. Understanding this tension is crucial to understanding modern democracy, what it is and what it could be. It has been argued that an effective way of making democracy stronger is to strengthen civil society. This article contains a comparative analysis of the central ideas of Habermas and Foucault as they pertain to the question of democracy and civil society. More specifically, the discourse ethics of Habermas is contrasted with the power analytics and ethics of Foucault evaluating their usefulness for those interested in understanding, and bringing about, democratic social change.

Learning objects offer increased ability to share learning resources so that system-wide production costs can be reduced. But how can users select from a set of similar learning objects in a repository and be assured of quality? This article reviews recent developments in the establishment of learning object repositories and metadata standards, and presents a formative reliability analysis of an online, collaborative method for evaluating quality of learning objects. The method uses a 10-item Learning Object Review Instrument (LORI) within a Convergent Participation evaluation model that brings together instructional designers, media developers, and instructors. The inter-rater reliability analysis of 12 raters evaluating eight learning objects identified specific items in LORI that require further development. Overall, the collaborative process substantially increased the reliability and validity of aggregate learning object ratings. The study concludes with specific recommendations including changes to LORI items, a rater training process, and requirements for selecting an evaluation team.

In this longitudinal study, we examined the effects of the use of English as the language of instruction in the first 4 years of secondary education in The Netherlands on the students' language proficiency in English and Dutch, and achievement in subject matters taught through English. Compared to a control group in regular secondary education, students attending this form of bilingual education showed higher scores for their English language proficiency in terms of oral proficiency and reading comprehension. No effects have been found for receptive word knowledge and no negative effects have been found with respect to the results of their school leaving exams at the end of secondary education for Dutch and subject matters taught through English.

This article provides an answer to what has been called the biggest problem in theorizing and understanding planning: the ambivalence about power found among planning researchers, theorists, and students. The author narrates how he came to work with issues of power and gives an example of how the methodology he developed for power studies—phronetic planning research—may be employed in practice. Phronetic planning research follows the tradition of power studies running from Machiavelli and Nietzsche to Michel Foucault and Pierre Bourdieu. It focuses on four value-rational questions: (1) Where are we going with planning? (2) Who gains and who loses, and by which mechanisms of power? (3) Is this development desirable? (4) What should be done? These questions are exemplified for a specific instance of Scandinavian urban planning. The author finds that the questions, and their answers, make a difference to planning in practice. They make planning research matter.

We examined 2 hypotheses regarding why some young women do not maintain their espoused occupational aspirations in male-dominated fields from late adolescence through young adulthood. The first hypothesis concerns attitudes towards math and science; the second concerns the desire for job flexibility. The sample of young women (N ¼ 104) was taken from a larger longitudinal investigation of approximately 1,000 young women from a midwestern metropolitan area in Michigan, USA, who were followed from age 18 (in 1990) to age 25 (1997). Findings suggest that desire for a flexible job, high time demands of an occupation, and low intrinsic value of physical science were the best predictors of women changing their occupational aspirations out of maledominated fields. These results suggest that despite the women's movement and more efforts in society to open occupational doors to traditional male-jobs for women, concerns about balancing career and family, together with lower value for science-related domains, continue to steer young women away from occupations in traditionally male-dominated fields, where their abilities and ambitions may lie.

This study examined the continued gender imbalance in mathematics participation in senior high, which has implications for adolescents' future careers. It confirms persisting greater male participation in maths-related educational and occupational choices among this sample of Australian youth. Gendered course-taking in maths, along with participation intentions for maths-related careers, were explained using the Expectancy-Value framework of Eccles and colleagues (Eccles . The influences of maths-related self-perceptions, intrinsic and utility values, and perceived task difficulty were assessed using longitudinal data from a sample of 442 adolescents spanning Grades 9 through 11 in Sydney, Australia. Gendered mathsrelated self-perceptions and intrinsic values were the major influences on gendered educational participation in senior high maths, which subsequently predicted maths-related career aspirationsover and above prior mathematical achievement. Utility value showed a curvilinear relationship with maths-related occupational intentions moderated by gender, whereby girls with the highest utility values planned for highly maths-related careers, while boys with mid through high utility values planned similarly highly maths-related careers. Recommendations focus on ways to enhance participation in maths, especially for girls.

This paper describes an experience where the Technology Acceptance Model (TAM) has been adapted for use in the evaluation of methodological and technological innovations determined by the introduction of a new e-learning system in an Italian online university. While the original TAM allows one to assess acceptance and adoption of a new technology, in this case there was also a need to consider all the phases of use of the system (course design, running and evaluation), all the users of the system (students, teachers and e-learning management), and all the system’s components (the e-learning platform, the learning resources and mostly the underlying pedagogical approach). The resulting model, which is an extension of the original TAM, is a three-dimensional one (phases of use, users and components), with three aspects to be considered on each axis. For each of the 27 combinations of these aspects, indicators of usefulness and ease-of-use have been identified. When available, data concerning actual use (derived from the tracking functions of the platform) and effectiveness (based on teachers’ adoption of new tools and students’ learning outcomes) have also been used to complement the data.

Rates of employee absences and the effects of absences on productivity are topics of conversation in many organizations in many countries. One reason is that high rates of employee absence may signal weak management and poor labor-management relations. A second reason is that reducing rates of employee absence may be an effective way to improve productivity. This paper reports the results of a study of employee absences in education, a large, labor-intensive industry. Policymakers' concern with teacher absence rests on three premises: (1) that a significant portion of teachers' absences is discretionary, (2) that teachers' absences have a nontrivial impact on productivity, and (3) that feasible policy changes could reduce rates of absence among teachers. This paper presents the results of an empirical investigation of the first two of these premises; it discusses the third premise. We employ a methodology that accounts for time-invariant differences among teachers in skill and motivation. We find large variation in adjusted teacher absence rates among schools. We estimate that each 10 days of teacher absences reduce students' mathematics achievement by 3.3 percent of a standard deviation.

If we want to empower and re-enchant organization research, we need to do three things. First, we must drop all pretence, however indirect, at emulating the success of the natural sciences in producing cumulative and predictive theory, for their approach simply  does not work in organization research or any of  the social sciences (for the full argument, see Flyvbjerg 2001). Second, we must address problems that matter to groups in the local, national, and global communities in which we live, and we must do it in ways that matter; we must focus on issues of context, values, and power, as advocated by great social scientists from Aristotle and Machiavelli  to Max Weber and Pierre Bourdieu. Finally, we must effectively and dialogically communicate the results of our research to our fellow citizens and carefully listen to their feedback. If we do this – focus on specific values and interests in the context of particular power relations – we may successfully transform organization research into an activity performed in public for organizational publics, sometimes to clarify, sometimes to intervene, sometimes to generate new perspectives, and always to serve as eyes and ears in ongoing efforts to understand the present and to deliberate about the future. We may, in short, arrive at organization research that matters.

The present meta-analysis examined the effect of grade retention on academic outcomes and investigated systemic sources of variability in effect sizes. Using multi-level modeling, we investigated characteristics of 207 effect sizes across 22 studies published between 1990 and 2007 at two levels: the study (between) and individual (within) levels. Design quality was a study-level variable. Individual level variables were median grade retained and median number of years post retention. Quality of design was associated with less negative effects. Studies employing middle to high methodological designs yielded effect sizes not statistically significantly different from zero and 0.34 higher (more positive) than studies with low design quality. Years post retention was negatively associated with retention effects, and this effect was stronger for studies using grade comparisons versus age comparisons. Results challenge the widely held view that retention has a negative impact on achievement. Suggestions for future research are discussed.

Niccolò Machiavelli, the founder of modern political and administrative thought, made clear that an understanding of politics requires distinguishing between formal politics and what later, with Ludwig von Rochau, would become known as Realpolitik. No such distinction has been employed in the study of rationality. Yet I will argue that distinguishing between formal rationality and real rationality is as important for the understanding of rationality and planning as the distinction between formal politics and Realpolitik has been for understanding politics.

Course evaluations can be an effective tool for students to appraise a course and instructor; however, they can also be a communicative space for students to write comments instructors perceive as hurtful, need to make sense of and reconcile with their instructional practice. Instructors turn to others to help make sense of and cope with comments. This study applies a message-centered approach to consider what social support is in a specific context, types and functions of social support in that situation, and effects of that support based on unique cultural and institutional macrostructures of the communication event. The study’s purpose was to identify support providers and messages women instructors receive when coping with hurtful comments. Collegiate women instructors (N = 81) received support from friends, family, colleagues, and administrators, including traditional forms (informational, network, emotion, esteem) as well as social undermining messages such as minimization and negative appraisal. Participants found network, situational appraisal, and esteem to be the most helpful types of support, and minimization, teaching recommendations, and negative appraisal to be the least helpful. Findings offer implications for women instructors seeking support for hurtful course evaluation comments and recommendations for how to offer support to those instructors.

This study explores the relationship between socioeconomic status (SES) and the computer-use profile of 1241 school students in Flanders, the northern region of Belgium. More specifically, the article examines whether varying patterns of computer access, attitudes, competencies and uses can be seen as constituting differences in cultural capital. Additionally, gender was included in the survey as an important background characteristic in digital divide research. Path analysis was used to model the complex relationships between the influencing factors upon the ICT-related variables. What emerged from the analyses was that SES affects the computer-use profile only moderately. No relationship between SES and computer ownership was found. Moreover, the acquisition of ICT competencies can no longer be attributed to computer ownership. Apart from a small effect on ICT use (a higher SES tends to be associated with more ICT use), SES does not seem to affect the computer-use profile of young people in Flanders.

This article examines the range of eighth-grade mathematics learning opportunities in the U.S. drawing on data gathered for the Third International Mathematics and Science Study (TIMSS). Sources of variation in the provision of learning opportunities are identified and patterns of eighth grade mathematics course offerings are compared across schools. Comparison of students' learning opportunities includes consideration of the specific course in which they were enrolled, the type of textbook employed for the course, and the proportion of time teachers devoted to teaching specific topics. Analyses revealed a mismatch between the mathematics course title and the textbook employed in the course for nearly 30 percent of U.S. eighth grade students. Course-textbook combinations demonstrated significant relationships with the time teachers devoted to specific topics and the international topic difficulty score. Some differences in mathematics learning opportunities were found on the basis of a school's location (urban, rural, suburban), size, and percent minority enrollment. The authors contend that the significant relationships found were not consistent enough to provide a satisfactory explanation for the observed variation and that individual student differences, an often implicit rationale for tracking, also represents an inadequate explanation for the observed diversity in curricular opportunities. They conclude that recognizing the multiple definitions for students' classroom mathematics learning experiences is an important step in reform and policy discussions.

As the United Nations Decade of Education for Sustainable Development (2005–2014) draws to a close, it is timely to review ways in which the sustainable development initiatives of higher education institutions have been, and can be, evaluated. In their efforts to document and assess collaborative sustainable development program outcomes and impacts, universities in the North and South are challenged by similar conundrums that confront development agencies. This article explores pathways to symmetrical evaluations of transnationally partnered research, curricula, and public-outreach initiatives specifically devoted to sustainable development. Drawing on extensive literature and informed by international development experience, the authors present a novel framework for evaluating transnational higher education partnerships devoted to sustainable development that addresses design, management, capacity building, and institutional outreach. The framework is applied by assessing several full-term African higher education evaluation case studies with a view toward identifying key limitations and suggesting useful future symmetrical evaluation pathways. University participants in transnational sustainable development initiatives, and their supporting donors, would be well-served by utilizing an inclusive evaluation framework that is infused with principles of symmetry.

'Assessment' and 'feedback' are inherently embedded in a course curriculum of engineering education settings. These components are indispensable for the teaching-learning processes. It is observed that engineering faculty members do not require any 'teacher-training' to join the engineering institutions across the globe. Hence, they may not have adequate experience in assessing students' performances and providing feedback to students. Only a few research studies have been carried out on assessment and feedback from the engineering education context. Therefore, this paper attempts to critically analyse the literature pertaining to learning through different types and methods of assessment practices in the engineering education settings. Further, it examines the significance of qualitative feedback in assessment and the principles of good feedback practice. It highlights the implications of assessing student performance and providing feedback from the engineering education perspective. Finally, the paper offers some recommendations on assessment and feedback practices in the engineering education settings.

The doctoral dissertation is defined as an original contribution to a field. By definition, this makes the dissertation a creative product, and the result of a creative process. The creative process of doctoral work has historically not been highlighted. The same is true for education as a whole. While there is an increasing call for greater creativity in education, they remain aspirational. In this article we describe the underlying premises and some of the practices of a doctoral degree that has been designed with the intention of foregrounding the creative process.

The present study compared the relative effects of 2 educational programs on kindergarten children. The experimental group took part in a numeracy-specific program, which focused on conceptual knowledge. Children were taught basic numerical skills such as understanding and handling numbers and their relations as well as counting principles. The control group participated in a more general program which involved training in procedural skills.

Although the relationship between attitudes toward reading and reading achievement has been well documented, the causal relationship between these constructs remains unclear.Using longirudinal covariance strucrure modeling. this srody tested the hypothesis that 3 reading-related constructs in the primary grades (2nd-3rd grade) -reading attirude, behavior, and achievement -would predict reading achievement in the 7th grade. Results showed that primary attirude was not correlated with primary achievement yet both had causal paths to 7th-grade achievement, described as a "temporalinteraction" model. The resulting model suggests that while reading attitude and achievement may appear unrelated at the early stages of reading they become more closely linked over time, developing into important causal determinants of reading achievement by early adolescence.

It is a widely held assumption that learning style is a useful model for quantifying user characteristics for effective personalized learning. We set out to challenge this assumption by discussing the current state of the art, in relation to quantitative evaluations of such systems and also the methodologies that should be employed in such evaluations. We present two case studies that provide rigorous and quantitative evaluations of learning-style-adapted e-learning environments. We believe that the null results of both these studies indicate a limited usefulness in terms of learning styles for user modeling and suggest that alternative characteristics or techniques might provide a more beneficial experience to users.

The article describes and critically analyzes how Russian education researchers approached the topic of quality evaluation in education between 1990 and 2014. Evaluation and quality have grown into major policy issues in education across the world, simultaneously acting as powerful steering mechanisms on national and transnational levels. Russia is no exception to this global phenomenon, but little is known about how Russian education researchers discuss the topic in national academic journals. This article discusses four major periods, each characterized by a shift in the focus of discussion and/or the introduction of a completely new agenda. These periods capture the dominant themes, titled “effective management and customization of education,” “the rise of broad-scale assessments,” “systemic approach to quality evaluation,” and “toward a more nuanced usage of evaluation data.” We interpret the findings within two intertwined conceptual frameworks: governance at a distance and New Public Management. How these frames help us understand the academic discussion on quality evaluation in Russian school education is also discussed.

The cover photograph of school children in Ethiopia is by Aminata Maiga-Touré. The requirement for faster development of the new nations in Africa. .. is more education and training at all levels-a more generally literate working force, more skilled artisans, more members of the learned professions, more entrepreneurs, more skilled government administrators. Until the human resources of the new African nations are more fully developed-and no huge injection of money can greatly accelerate that process-the opportunities for the wise and effective utilization of foreign investment will necessarily remain limited.

The Code of Fair Testing Practices in Education (Code) is a guide for professionals in fulfilling their obligation to provide
and use tests that are fair to all test takers regardless of age, gender, disability, race, ethnicity, national origin, religion,
sexual orientation, linguistic background, or other personal characteristics. Fairness is a primary consideration in all
aspects of testing. Careful standardization of tests and administration conditions helps to ensure that all test takers are
given a comparable opportunity to demonstrate what they know and how they can perform in the area being tested.
Fairness implies that every test taker has the opportunity to prepare for the test and is informed about the general
nature and content of the test, as appropriate to the purpose of the test. Fairness also extends to the accurate reporting
of individual and group test results. Fairness is not an isolated concept, but must be considered in all aspects of the
testing process.

This article identifies spatial dimensions of educational outcomes using maps of the 2016 Grade 5 reading results for Australia's National Assessment Program–Literacy and Numeracy for all Australian schools. A geographical information system (GIS) was used to overlay schools' results onto suburbs' advantage or disadvantage to visualise spatial patterns. We then examined the extent to which school results “cluster” in socio‐economically advantaged and disadvantaged suburbs and considered the consistency of spatial patterns for results across major cities. That work illustrates both how GIS can foreground educational inequality and how “the spatial” is more than corollary for student socio‐economic status. Results show substantial differences between urban and remote areas and towns of different size. Maps of cities visualise spatial “clustering” patterns of school results, with most schools in advantaged suburbs having high results and almost no schools in disadvantaged suburbs having high results. Educational outcomes strongly align to local socio‐demographic characteristics, and parallel host communities’ levels of advantage or disadvantage. Differences between public and private schools are less significant than within‐sector differences for schools in advantaged or disadvantaged locales. Patterns in all cities are consistent—schools in advantaged suburbs predominantly have high results, whereas non‐government schools generally perform better than government schools in disadvantaged suburbs. Most concerning is the persistent and increasing trajectory of results in advantaged, and more so in disadvantaged suburbs, of all cities since the first National Assessment Program–Literacy and Numeracy in 2008. Ameliorating spatial inequality between primary schools is one of the greatest challenges for Australians.

In this study, the results of ®ve training studies evaluating the effects of a coaching program for use in Dutch primary and secondary schools are described. The coaches involved in these studies were: (1) school counsellors, (2) school principals, (3) mentors of beginning teachers, (4) mentors of teachers-in-training and (5) secondary school teachers. Coaching is a form of inclass support to provide (novice) teachers with feedback on their functioning and thereby stimulate the self-re¯ection and self-analysis needed to improve instructional effectiveness. Based on the pre-and post-training ratings of coaching conferences, a signi®cant treatment effect was found for the coaching skills concerned with the development of autonomy (empowerment), feedback and business-like attitude. The pre-and post-training ratings from the coached teachers themselves showed a signi®cant treatment effect for the coaching skills concerned with the planning of observational data collection. In addition, the teachers coached by trained coaches valued the coaching skills demonstrated signi®cantly more than those teachers coached by untrained coaches. Inspection of the interactions for the adjusted post-test scores revealed different training effects for the ®ve studies.

Abstract
The Evaluation Self-Efficacy (ESE) scale was designed as an outcome measure for evaluation research courses in social work. A revised version of the Social Cognitive Theory–based ESE (ESE-II) was developed and evaluated in the current study including both new and revised items. The ESE-II was evaluated in a final sample of 168 masters level students using a pretest–posttest design. Exploratory factor analysis revealed a single-factor structure underlying the 14 self-efficacy items at both assessment points. Cronbach’s alphas for the ESE-II were high at pre- and posttest. An argument underpinning content validity was developedand convergent validity was demonstrated. The ESE-II was also sensitive to change over time both at the item and scale level. The current study provides evidence supporting select psychometric properties of the ESE-II and the flexibility of self-efficacy as an outcome measure for social work education.

Principals experience a range of feelings and emotions while practicing social justice leadership with implications on their leadership. This article presents a qualitative case study of an elementary school principal in an urban setting and how she led to create a more inclusive school. Sensemaking theory was applied to examine the principal's experiences leading for social justice and how successes, challenges, and failures influenced her leadership. Findings have implications for theory, future research, and principal selection, preparation, and training.

University students are too often challenged by their limited skills in application, investigation, relational thinking, and communication of ideas. In this study, we have combined 3 tools that potentially can support and foster students' development in the above mentioned areas through student collaboration, concept mapping, and electronic technologies. The participants in this study were 26 students in two intact classes in learning theories. In groups of 3 to 5 students, they were asked to generate 3 concept maps and accompanying prose over the term on 3 major issues in the ®eld of learning. Through the use of interviews, questionnaires, and student generated concept maps, students reportedly enjoyed concept mapping for its organizational and relational properties but preferred sharing their concept maps and dialoguing with one another in a synchronous mode where immediate feedback and¯ow of thinking could be maintained when involved in constructing maps. Moreover, they did not like the redundancy offered by both prose and concept map outputs, suggesting that while concept mapping can be an arena for generating and generally structuring ideas, prose can be a means of communicating such ideas in a form that is common to most people. This is particularly important for teachers and students who have dif®culty navigating through maps alone.

In recent years there has been a global call for more scientific knowledge about education as a basis of governance. This means that exact descriptions of the reality of schooling should inform decisions about what works in education. In this article, evaluation and testing are analysed as cartography, the art of mapping educational spaces, which bot creates and confuses our sense of educational reality. By using elements from cultural studies of cartography as well as sociology and the philosophy of science, this article claims that the analogy of cartography and evaluation can open novel vistas for contemplating the relationship between the world of education and its scientific representation. As a case in point, the article uses the construction of Finnish primary school reform. The analysis shows how evaluation as the mapping of the reality of education brings distant objects near, onto a homogeneous, stable plane. It also makes certain things visible while leaving others out of sight. Furthermore, evaluation as cartography is not only passive representation; it actually creates new spaces. This way, evaluation practices can profoundly affect how we think and act upon schooling.

191 S. HOWIE & COLLEAGUES: EFFECT OF MULTILINGUAL POLICIES ON READING LITERACY IN SOUTH AFRICAN SCHOOLS The effect of multilingual policies on performance and progression in reading literacy in South African primary schools Sarah Howie, Elsie Venter, Surette ...

Students' feedback is an effective mechanism that provides valuable insights about teaching-learning process. Handling opinions of students expressed in reviews is a quite labour-intensive and tedious task as it is typically performed manually by the human intervention. While this task may be viable for small-scale courses that involve just a few students' feedback, it is unpractical for large-scale cases as it applies to online courses in general, and MOOCs, in particular. Therefore, to address this issue, we propose in this paper a framework to automatically analyzing opinions of students expressed in reviews. Specifically, the framework relies on aspect-level sentiment analysis and aims to automatically identify sentiment or opinion polarity expressed towards a given aspect related to the MOOC. The proposed framework takes advantage of weakly supervised annotation of MOOC-related aspects and propagates the weak supervision signal to effectively identify the aspect categories discussed in the unlabeled students' reviews. Consequently, it significantly reduces the need for manually annotated data which is the main bottleneck for all deep learning techniques. A large-scale real-world education dataset containing around 105k students' reviews collected from Coursera and a dataset comprising of 5989 students' feedback in traditional classroom settings are used to perform experiments. The experimental results indicate that our proposed framework attains inspiring performance with respect to both the aspect category identification and the aspect sentiment classification. Moreover, the results suggest that the framework leads to more accurate results than the expensive and labour-intensive sentiment analysis techniques relying heavily on manually labelled data.

The study focused on students' perceptions in order to find out the study behavior put forward by them prior to and during the exam in each of both the open-book and the closed-book exam alternatives. Participants were 181 students, comprising a 72.8% response rate, who experienced both exam alternatives; they were asked to assess 39 statements referring to study behaviors on a 5-point scale (1 = not at all present to 5 = present to a very great extent) for both exam options. The results indicated that students preparing for a closed book examination tend to postpone their study at the end of the semester, focus on the assigned texts and memorize information. Students preparing for an openbook examination tend to consult various sources and interrelate the information acquired; when taking the exam, they work creatively while, at the same time, they probe deeply into the knowledge gained. When the 39 statements were factor-analyzed, four factors emerged having to do with (a) mastering the course content; (b) involvement in the learning process; (c) using the knowledge gained in a creative way; and (d) facing the exam with optimism. For all four factors, the composite score was higher for the openbook exam option than for the closed one. The implications of the study as well as its major flaws are also discussed.