The Design Principle Documentation Project has followed the development of thirty digital badging... more The Design Principle Documentation Project has followed the development of thirty digital badging projects as they moved from their intended practices outlined in their initial proposals to their enacted practices constrained by their audience and the badging platform. The DPD Project aimed to capture the knowledge that emerged in this shift before it evaporated. The result was the derivation of ten design principles for designing assessments in digital badge systems that is relevant beyond badging systems. A scholarly review of the literature related to the design principles revealed aspects of assessment functions that badging projectsor anyone designing assessments for a learning systemshould consider as they make decisions about the kind of learning they wish to foster.
Erin Knight is the senior learning director at the Mozilla Foundation and co-creator of Mozilla's... more Erin Knight is the senior learning director at the Mozilla Foundation and co-creator of Mozilla's Open Badges Infrastructure. Erin is leading the effort at the Mozilla Foundation to support the dynamic community that has emerged around digital badges.
The Design Principles Documentation Project traced the practices of thirty digital badge projects... more The Design Principles Documentation Project traced the practices of thirty digital badge projects from conception to implementation in order to create design principles that may inform future design and implementation of digital badge systems. This paper uses collective case study methods to reflexively examine the specific practices for recognizing learning with digital badges as they relate to the broader design principles. While the projects implemented some of the same recognition principles, the specific recognition practices differed. Juxtaposition of the three cases confirms the absence of "best practices." Rather, there are many possible recognition practices that appear appropriate in some contexts, and that then have implications for assessing, motivating, and evaluating learning with digital badges.
This paper describes a method for studying programs that issue Open Badges to recognize learning.... more This paper describes a method for studying programs that issue Open Badges to recognize learning. The Design Principles Documentation (DPD) Project followed the development of 30 educational programs that planned to issue open digital badges to recognize "lifelong learning" accomplishment. The DPD Project's aim was to formulate general design principles based on the practices observed among the 30 research subjects. Analysis yielded 37 principles across four researcher-selected functions of digital badge systems: recognizing learning, assessing learning, motivating learning, and studying learning. This work describes this research methodology and its affordances for uncovering relationships between different elements of badge system design and between those elements and the larger project contexts in which they operate.
There is broad interest in the use of open digital badges to enhance learners' motivation. These ... more There is broad interest in the use of open digital badges to enhance learners' motivation. These web-enabled tokens of learning and accomplishment have the potential to induce excitement and elicit powerful forms of engagement and learning. Researchers and developers, however, appear divided on the role of digital badges in motivating learners. Our paper addresses the skepticism and promise that surrounds badges and presents the design principles for motivating learning found among 30 digital badge projects and aligns them with research. In doing so, we consider how contextual factors-such as how badges are used recognize learning and how that learning is assessed-may play out to influence learner motivation in badge-oriented learning ecosystems.
Open digital badges are web-enabled tokens of learning and accomplishment. They operate in an env... more Open digital badges are web-enabled tokens of learning and accomplishment. They operate in an environment of explicit (rather than tacit) trust; open badges provide issuers the ability to include specific claims and associate those claims with detailed supporting evidence. Earners are encouraged to share their badges over social networks, email, and websites, and the information they contain is expected to circulate readily in these spaces. Building upon current concepts and theories from the Information Sciences and Learning Sciences, this paper shows how the informational affordances of digital badges are transforming education and learning more generally, and more particularly by transcending conventional paradigms of academic credentialing & educational assessment.
An assessment-oriented design-based research model was applied to existing inquiryoriented multim... more An assessment-oriented design-based research model was applied to existing inquiryoriented multimedia programs in astronomy, biology, and ecology. Building on emerging situative theories of assessment, the model extends prevailing views of formative assessment for learning by embedding ''discursive'' formative assessment more directly into the curriculum. Three twenty-hour curricula were designed and aligned to content standards, and three levels of assessments were developed and used to assess and enhance learning for each curriculum. These assessments included three or four informal ''activity-oriented'' quizzes and discursive formative feedback rubrics supporting collective discourse, a ''curriculum-oriented'' examination of individual conceptual understanding, and a ''standards-oriented'' test measuring aggregated achievement of targeted standards. After two designresearch cycles, worthwhile scientific argumentation and statistically significant gains were attained for two of the three packages on the exam and test. Achievement gains were comparable to or larger than those of students in comparison classrooms. Many existing innovations could be enhanced and evaluated in this fashion; designing these strategies directly into innovations could have an even greater impact on discourse, understanding, and achievement. ß
Evaluating the impact of instructional innovations and coordinating instruction, assessment, and ... more Evaluating the impact of instructional innovations and coordinating instruction, assessment, and testing present complex tensions. Many evaluation and coordination efforts aim to address these tensions by using the coherence provided by modern cognitive science perspectives on domain-specific learning. This paper introduces an alternative framework that uses emerging situative assessment perspectives to align learning across increasingly formal levels of educational practice. This framework emerged from 2 design studies of a 20-hr high school genetics curriculum that used the GenScope computer-based modeling software. The 1st study aligned learning across (a) the contextualized enactment of inquiry-oriented activities in GenScope, (b) "feedback conversations" around informal embedded assessments, and (c) a formal performance assessment; the 2nd study extended this alignment to a conventional achievement test. Design-based refinements ultimately delivered gains of nearly 2 SD on the performance assessment and more than 1 SD in achievement. These compared to gains of 0.25 and 0.50 SD, respectively, in well-matched comparison classrooms. General and specific assessment design principles for aligning instruction, assessment, and testing and for evaluating instructional innovations are presented.
This study investigated the effects of external rewards on fifth graders' motivation, engagement ... more This study investigated the effects of external rewards on fifth graders' motivation, engagement and learning while playing an educational game. We were interested in exploring whether the feedback-rich environment of the game could mitigate the predicted negative effects of external rewards. Data of students' engagement and learning were collected and analyzed at multiple levels. A quasi-experimental design was used to examine the effect of external rewards in one group (n ¼ 50) compared to a control group without such rewards (n ¼ 56). According to the results, the external rewards did not undermine students' motivation (e.g., at proximal and distal levels), however they did not foster disciplinary engagement. On the other hand, students in the reward condition showed significantly larger gains in conceptual understanding (proximal) and non-significantly larger gains in achievement (distal). These results suggest that the predicted negative consequences of external rewards may be addressed in this new generation of learning environments. Future research and contributions of the study are provided.
Relating Narrative, Inquiry, and Inscriptions: Supporting Consequential Play
Journal of Science Education and Technology, 2007
In this paper we describe our research using a multi-user virtual environment, Quest Atlantis, to... more In this paper we describe our research using a multi-user virtual environment, Quest Atlantis, to embed fourth grade students in an aquatic habitat simulation. Specifically targeted towards engaging students in a rich inquiry investigation, we layered a socio-scientific narrative and an interactive rule set into a multi-user virtual environment gaming engine to establish a virtual world through which students learned about science inquiry, water quality concepts, and the challenges in balancing scientific and socio-economic factors. Overall, students were clearly engaged, participated in rich scientific discourse, submitted quality work, and learned science content. Further, through participation in this narrative, students developed a rich perceptual, conceptual, and ethical understanding of science. This study suggests that multi-user virtual worlds can be effectively leveraged to support academic content learning.
Enhancing Inquiry, Understanding, and Achievement in an Astronomy Multimedia Learning Environment
Journal of Science Education and Technology, 2006
As an example of design-based research, this study refined an assessment strategy for simultaneou... more As an example of design-based research, this study refined an assessment strategy for simultaneously enhancing inquiry-based learning and supporting achievement on conventional assessment measures. Astronomy Village ® : Investigating the Universe™ is a software program designed to engage secondary science students in authentic and inquiry-based learning over core topics in astronomy. The software was enhanced with a 20-hour curriculum and three levels of assessment to ensure successful inquiry experiences and high-stakes achievement. The first year implementation of Astronomy Village® yielded significant gains on a curriculum-oriented exam but not a standards-oriented test, and provided useful design insights that were integrated into the second year implementations. Significant gains were obtained on the test during the second year as well. It is expected that many existing inquiry-oriented science curricula might be similarly enhanced, and is suggested that a large-scale effort to do so might have a lasting impact on science education.
Balancing varied assessment functions to attain systemic validity: Three is the magic number
Studies in Educational Evaluation, 2006
Erratum to: Relating Narrative, Inquiry, and Inscriptions: Supporting Consequential Play
Journal of Science Education and Technology, 2010
In this paper we describe our research using a multi-user virtual environment, Quest Atlantis, to... more In this paper we describe our research using a multi-user virtual environment, Quest Atlantis, to embed fourth grade students in an aquatic habitat simulation. Specifically targeted towards engaging students in a rich inquiry investigation, we layered a socio-scientific narrative and an interactive rule set into a multi-user virtual environment gaming engine to establish a virtual world through which students learned about science inquiry, water quality concepts, and the challenges in balancing scientific and socio-economic factors. Overall, students were clearly engaged, participated in rich scientific discourse, submitted quality work, and learned science content. Further, through participation in this narrative, students developed a rich perceptual, conceptual, and ethical understanding of science. This study suggests that multi-user virtual worlds can be effectively leveraged to support academic content learning.
Enhancing Inquiry, Understanding, and Achievement in an Astronomy Multimedia Learning Environment
Journal of Science Education and Technology, 2006
Argumentation: A strategy for improving achievement and revealing scientific identities
International Journal of Science Education, 2008
Relating Narrative, Inquiry, and Inscriptions: Supporting Consequential Play
Engaged Participation: A Sociocultural Model of Motivation With Implications for Educational Assessment
Educational Assessment, 2005
Classroom Discourse as a Tool to Enhance Formative Assessment and Practise in Science
Uploads
Papers by Daniel Hickey