Skip to main content

Jiangang Hao

Educational Testing Service, Sadapr, Sr. Research Scientist

Followers

11

Following

97

Co-author

1

Public Views

Uploads

Papers by Jiangang Hao

Large mosaic multiCCD camera is the key instrument for modern digital sky survey. DECam is an ext... more Large mosaic multiCCD camera is the key instrument for modern digital sky survey. DECam is an extremely red sensitive 520 Megapixel camera designed for the incoming Dark Energy Survey (DES). It is consist of sixty two 4k×2k and twelve 2k×2k 250-micron thick fully-depleted CCDs, with a focal plane of 44 cm in diameter and a field of view of 2.2 square degree. It will be attached to the Blanco 4-meter telescope at CTIO. The DES will cover 5000 square-degrees of the southern galactic cap in 5 color bands (g, r, i, z, Y) in 5 years starting from 2011. To achieve the science goal of constraining the Dark Energy evolution, stringent requirements are laid down for the design of DECam. Among them, the flatness of the focal plane needs to be controlled within a 60-micron envelope in order to achieve the specified PSF variation limit. It is very challenging to measure the flatness of the focal plane to such precision when it is placed in a high vacuum dewar at 173 K. We developed two image based techniques to measure the flatness of the focal plane. By imaging a regular grid of dots on the focal plane, the CCD offset along the optical axis is converted to the variation the grid spacings at different positions on the focal plane. After extracting the patterns and comparing the change in spacings, we can measure the flatness to high precision. In method 1, the regular dots are kept in high sub micron precision and cover the whole focal plane. In method 2, no high precision for the grid is required. Instead, we use a precise XY stage moves the pattern across the whole focal plane and comparing the variations of the spacing when it is imaged by different CCDs. Simulation and real measurements show that the two methods work very well for our purpose, and are in good agreement with the direct optical measurements.

format_quoteThe DES specification allows a focal plane non-flatness of 60 microns, with significant implications for image precision in galaxy shape measurements.format_quote

Analyzing Process Data from Game/Scenario-Based Tasks: An Edit Distance Approach

Students' activities in game/scenario-based tasks (G/SBTs, hereafter) can be characterized by... more Students' activities in game/scenario-based tasks (G/SBTs, hereafter) can be characterized by a sequence of time-stamped actions of different types with different attributes. For a subset of the G/SBTs where only the order of the actions are of great interest, the process data can be well characterized as a string of characters (action string, hereafter) if we encode each action name as a single character. In this paper, we report our work on evaluating students' performances by comparing how far their action strings are from the action string that corresponds to the best performance, where the proximity is quantified by the edit distance between the strings. Specifically, we choose the Levenshtein distance, which is defined as the minimum number of insertion, deletion and replacement needed to convert one character string to another. Our results show a strong correlation between the edit distances and the scores obtained from the scoring rubrics of the WELL task from NAEP T...

On Convergence of Cognitive and Non-cognitive Behavior in Collaborative Activity

We present results from a pilot study to investigate the evidence for convergence and synchrony i... more We present results from a pilot study to investigate the evidence for convergence and synchrony in cognitive and noncognitive behavior of dyads engaged in a collaborative activity. Our approach utilizes multimodal data including video and participant action log files retrieved from the collaborative activity, an online educational simulation on science topics. The log files captured cognitive behavior including frequency and content of chat messages between dyads, as well system help requests. The video data recorded participant nonverbal behavior that was processed on a frame-by-frame basis using automated facial expression classifiers and coded by trained human raters on high-level noncognitive behaviors including: affect display gestures, engagement, anxiety and curiosity. The data were analyzed at individual and dyad levels and results using hierarchical clustering analysis demonstrate evidence of cognitive and noncognitive behavioral convergence among dyads.

Collaborative Problem Solving: Innovating Standardized Assessment

In this symposium, we present the overall design, data, and scientific findings from the ETS Coll... more In this symposium, we present the overall design, data, and scientific findings from the ETS Collaborative Science Assessment Prototype (ECSAP). We are opening our data to the CSCL community and introducing the procedures to request access to the data. ECSAP was developed to explore the assessment of collaborative problem solving (CPS) competency through a large-scale and standardized approach. The goal of this symposium is to examine research questions that are of interest to the CSCL community, such as how CPS skills and collaborative patterns interact with performance outcomes, and how prior content knowledge and personality of team members affect the collaboration process and outcomes. In our study, we collected both individual and collaborative responses (~1500 responses) to the ECSAP instruments. We present our study findings that used new methodologies in psychometrics and followed the best practices of psychometrics and statistics.

format_quoteIntroduced format_quote

Assessing Collaborative Problem Solving with Simulation Based Tasks

Assessing collaborative problem solving (CPS) is an integrated part of the computer-supported col... more Assessing collaborative problem solving (CPS) is an integrated part of the computer-supported collaborative learning (CSCL). We present some preliminary results from a project developed for assessing the CPS using web-based simulation. In the simulation, two participants collaborate via a chat box to complete a task on volcano science. By comparing the responses from 486 individuals and 278 teams (dyads) recruited from Amazon Mechanical Turk, we found the performance from the teams (dyads) is significantly higher than that from individuals. We also find that the item difficulty in the simulation affects both the processes and outcomes of the collaboration.

Ghost in the Machine: A Symposium on Collaboration Between Human and Computerized Agents in Educational Contexts

The importance of collaboration is growing in an era where knowledge-based tasks are increasingly... more The importance of collaboration is growing in an era where knowledge-based tasks are increasingly accomplished by teams of people with complementary roles and expertise, as opposed to individuals doing isolated work. Moreover, the nature of collaboration is shifting to a more sophisticated skillset that includes accomplishing tasks through mediated interactions with peers halfway across the world or even computer-generated agents. The 4 papers presented in this symposium focus on this new opportunity of introducing computer-generated agents in collaboration as well as the challenges entailed. The presentations will cover a broad range of topics ranging from the assessment of collaborative problem-solving skills to the use of computer-generated agents in intelligent tutoring systems. In addition, they will illustrate the validity and practicability of various state-of-the-art approaches to introduce non-human collaborators into human collaboration. Overall focus of the symposium Coll...

format_quoteThe study aimed to explore if PISA 2015 CPS tasks reflected actual student interactions with agents, paving the way for future assessments involving technology.format_quote

An Item Response Theory Analysis of Problem-Solving Processes in Scenario-Based Tasks

Psychological test and assessment modeling, 2017

Advances in technology result in evolving educational assessment design and implementation. The n... more Advances in technology result in evolving educational assessment design and implementation. The new generation assessments include innovative technology-enhanced items, such as simulations and game-like tasks that mimic an authentic learning experience. Two questions that arise along with the implementation of the technology-enhanced items are: (1) what data and their associated features may serve as meaningful measurement evidence, and (2) how to statistically and psychometrically characterize new data and reliably identify their features of interest. This paper focuses on one of the new data types, process data, which reflects students' procedure of solving a problem. A new model, a Markov-IRT model, is proposed to characterize and capture the unique features of each individual's response process during a problem-solving activity in scenario-based tasks. The structure of the model, its assumptions, the parameter space, and the estimation of the parameters are discussed in this paper. Furthermore, we illustrate the application of the Markov-IRT model, and discuss its usefulness in characterizing students' response processes using an empirical example based on a scenario-based task from the NAEP-TEL assessment. Lastly, we illustrate the identification and extraction of features of the students' response processes to be used as evidence for psychometric measurement.

Collaborative Problem Solving Skills versus Collaboration Outcomes: Findings from Statistical Analysis and Data Mining

With the aid of educational data mining and statistical analysis, we investigate the relationship... more With the aid of educational data mining and statistical analysis, we investigate the relationship between collaboration outcomes and collaborative problem solving (CPS) skills exhibited during the collaboration process. We found that negotiation skill contributes positively to the collaboration outcomes while purely sharing information does the opposite.

format_quoteThe revised responses from the simulation task provide direct insights into the impact of collaboration on individual performance.format_quote

The Evidence Trace File: A Data Structure for Virtual Performance Assessments Informed by Data Analytics and Evidence‐Centered Design

ETS Research Report Series, 2018

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its pr... more Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.

format_quoteEvidence-Centered Design (ECD) provides a framework for VPA creation, aiding communication and assessment efficiency.format_quote

Characterizing Interactive Communications in Computer-Supported Collaborative Problem-Solving Tasks: A Conditional Transition Profile Approach

Frontiers in Psychology, 2019

Communication in a collaborative problem-solving activity plays a pivotal role in the success of ... more Communication in a collaborative problem-solving activity plays a pivotal role in the success of the collaboration in both academia and the workplace. Computer-supported collaboration makes it possible to collect large-scale communication data to investigate the process at a finer granularity. In this paper, we introduce a conditional transition profile (CTP) to characterize aspects of each team member's communication. Based on the data from a large-scale empirical study, we found that participants in the same team tend to show similar CTP compared to participants from different teams. We also found that team members who showed more "negotiation" after the partner "shared" information tended to show more improvement after the collaboration while those who continued sharing ideas while their partners were negotiating tended to improve less.

CPS-Rater: Automated Sequential Annotation for Conversations in Collaborative Problem-Solving Activities

ETS Research Report Series, 2017

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its pr... more Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.

format_quoteCPS-rater was developed using chat data from the ECSAP, with each team generating around 80 conversational turns during tasks.format_quote

EPCAL: ETS Platform for Collaborative Assessment and Learning

ETS Research Report Series, 2017

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its pr... more Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.

Interdisciplinary research agenda in support of assessment of collaborative problem solving: lessons learned from developing a Collaborative Science Assessment Prototype

Computers in Human Behavior, 2017

Taming Log Files From Game/Simulation-Based Assessments: Data Models and Data Analysis Tools

ETS Research Report Series, 2016

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its pr... more Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service.

The Dark Energy Survey and operations: Year 1

SPIE Proceedings, 2014

The Dark Energy Survey (DES) is a next generation optical survey aimed at understanding the accel... more The Dark Energy Survey (DES) is a next generation optical survey aimed at understanding the accelerating expansion of the universe using four complementary methods: weak gravitational lensing, galaxy cluster counts, baryon acoustic oscillations, and Type Ia supernovae. To perform the 5000 sq-degree wide field and 30 sq-degree supernova surveys, the DES Collaboration built the Dark Energy Camera (DECam), a 3 square-degree, 570-Megapixel CCD camera that was installed at the prime focus of the Blanco 4-meter telescope at the Cerro Tololo Inter-American Observatory (CTIO). DES started its first observing season on August 31, 2013 and observed for 105 nights through mid-February 2014. This paper describes DES "Year 1" (Y1), the strategy and goals for the first year's data, provides an outline of the operations procedures, lists the efficiency of survey operations and the causes of lost observing time, provides details about the quality of the first year's data, and hints at the "Year 2" plan and outlook.

DECam Integration Tests on Telescope Simulator

Physics Procedia, 2012

The Dark Energy Survey (DES) is a next generation optical survey aimed at measuring the expansion... more The Dark Energy Survey (DES) is a next generation optical survey aimed at measuring the expansion history of the universe using four probes: weak gravitational lensing, galaxy cluster counts, baryon acoustic oscillations, and Type Ia supernovae. To perform the survey, the DES Collaboration is building the Dark Energy Camera (DECam), a 3 square degree, 570 Megapixel CCD camera which will be mounted at the Blanco 4-meter telescope at the Cerro Tololo Inter-American Observatory. DES will survey 5000 square degrees of the southern galactic cap in 5 filters (g, r, i, z, Y). DECam will be comprised of 74 250 micron thick fully depleted CCDs: 62 2k x 4k CCDs for imaging and 12 2k x 2k CCDs for guiding and focus. Construction of DECam is nearing completion. In order to verify that the camera meets technical specifications for DES and to reduce the time required to commission the instrument, we have constructed a full sized telescope simulator and performed full system testing and integration prior to shipping. To complete this comprehensive test phase we have simulated a DES observing run in which we have collected 4 nights worth of data. We report on the results of these unique tests performed for the DECam and its impact on the experiments progress.

Orientation bias of optically selected galaxy clusters and its impact on stacked weak-lensing analyses

Weak-lensing measurements of the averaged shear profiles of galaxy clusters binned by some proxy ... more Weak-lensing measurements of the averaged shear profiles of galaxy clusters binned by some proxy for cluster mass are commonly converted to cluster mass estimates under the assumption that these cluster stacks have spherical symmetry. In this paper, we test whether this assumption holds for optically selected clusters binned by estimated optical richness. Using mock catalogues created from N-body simulations populated realistically with galaxies, we ran a suite of optical cluster finders and estimated their optical richness. We binned galaxy clusters by true cluster mass and estimated optical richness and measure the ellipticity of these stacks. We find that the processes of optical cluster selection and richness estimation are biased, leading to stacked structures that are elongated along the line of sight. We show that weak-lensing alone cannot measure the size of this orientation bias. Weak-lensing masses of stacked optically selected clusters are overestimated by up to 3-6 per cent when clusters can be uniquely associated with haloes. This effect is large enough to lead to significant biases in the cosmological parameters derived from large surveys like the Dark Energy Survey, if not calibrated via simulations or fitted simultaneously. This bias probably also contributes to the observed discrepancy between the observed and predicted Sunyaev-Zel'dovich signal of optically selected clusters.

format_quoteAt low cluster masses, average ellipticity shows a strong prolate shape, declining to spherical symmetry at higher masses (log M200 ≈ 14.1).format_quote

Measuring the flatness of focal plane for very large mosaic CCD camera

SPIE Proceedings, 2010

Large mosaic multiCCD camera is the key instrument for modern digital sky survey. DECam is an ext... more Large mosaic multiCCD camera is the key instrument for modern digital sky survey. DECam is an extremely red sensitive 520 Megapixel camera designed for the incoming Dark Energy Survey (DES). It is consist of sixty two 4k×2k and twelve 2k×2k 250-micron thick fully-depleted CCDs, with a focal plane of 44 cm in diameter and a field of view of 2.2 square degree. It will be attached to the Blanco 4-meter telescope at CTIO. The DES will cover 5000 square-degrees of the southern galactic cap in 5 color bands (g, r, i, z, Y) in 5 years starting from 2011. To achieve the science goal of constraining the Dark Energy evolution, stringent requirements are laid down for the design of DECam. Among them, the flatness of the focal plane needs to be controlled within a 60-micron envelope in order to achieve the specified PSF variation limit. It is very challenging to measure the flatness of the focal plane to such precision when it is placed in a high vacuum dewar at 173 K. We developed two image based techniques to measure the flatness of the focal plane. By imaging a regular grid of dots on the focal plane, the CCD offset along the optical axis is converted to the variation the grid spacings at different positions on the focal plane. After extracting the patterns and comparing the change in spacings, we can measure the flatness to high precision. In method 1, the regular dots are kept in high sub micron precision and cover the whole focal plane. In method 2, no high precision for the grid is required. Instead, we use a precise XY stage moves the pattern across the whole focal plane and comparing the variations of the spacing when it is imaged by different CCDs. Simulation and real measurements show that the two methods work very well for our purpose, and are in good agreement with the direct optical measurements.

<title>Status of the dark energy survey camera (DECam) project</title>

Ground-based and Airborne Instrumentation for Astronomy III, 2010

The Dark Energy Survey Collaboration has completed construction of the Dark Energy Camera (DECam)... more The Dark Energy Survey Collaboration has completed construction of the Dark Energy Camera (DECam), a 3 square degree, 570 Megapixel CCD camera which will be mounted on the Blanco 4-meter telescope at CTIO. DECam will be used to perform the 5000 sq. deg. Dark Energy Survey with 30% of the telescope time over a 5 year period. During the remainder of the time, and after the survey, DECam will be available as a community instrument. All components of DECam have been shipped to Chile and post-shipping checkout finished in Jan. 2012. Installation is in progress. A summary of lessons learned and an update of the performance of DECam and the status of the DECam installation and commissioning will be presented.

<title>Testing the Dark Energy Camera on a telescope simulator</title>

Ground-based and Airborne Instrumentation for Astronomy III, 2010

The Dark Energy Camera is a new prime-focus instrument to be delivered to the Blanco 4-meter tele... more The Dark Energy Camera is a new prime-focus instrument to be delivered to the Blanco 4-meter telescope at the Cerro Tololo Inter-American Observatory (CTIO) in 2011. Construction is in-progress at this time at Fermilab. In order to verify that the camera meets technical specifications for the Dark Energy Survey and to reduce the time required to commission the instrument while it is on the telescope, we are constructing a "Telescope Simulator" and performing full system testing prior to shipping to CTIO. This presentation will describe the Telescope Simulator and how we use it to verify some of the technical specifications.