Research & Data Practices
last updated 2/17/2017
Lumen Learning seeks to advance learning science by yielding insights about learning and how to improve learning efficacy using data collected through courseware as well as related learner data from institutions. We will adhere to transparent, responsible and ethical practices around data ownership, sharing and use. Lumen Learning is also committed to compliance with institutional, state and federal policies regarding appropriate handling and use of learner data.
The foundation of Lumen Learning’s research and data practices is the Asilomar Convention for Learning Research in Higher Education, which asserts the following tenets for learning research:
- We are committed to advancing the science of learning for the improvement of higher education.
“The science of learning can improve higher education and should proceed through open, participatory, and transparent processes of data collection and analysis that provide empirical evidence for knowledge claims.” (Asilomar Convention)
- We are committed to sharing data, findings and technologies within the learning research community in order to extend the project’s contributions to learning science.
“Maximizing the benefits of learning research requires the sharing of data, discovery, and technology among a community of researchers and educational organizations committed, and accountable to, principles of ethical inquiry held in common.” (Asilomar Convention)
Improvement in Student Learning
Improving student learning is the single most important purpose of Lumen Learning projects. Our courseware will use data to provide visibility for learners and instructors about student performance, as well as when and how to learn more successfully through factors learners and instructors can control. By design, our courseware will use data and technology not to replace student-instructor interaction, but rather to enhance and strengthen this relationship in support of student learning.
Lumen Learning maintains a strong commitment to research, building from and extending evidence-based practices when appropriate and possible.
Consent for Use of Data
In order to function fully and effectively, Lumen Learning courseware captures and uses a variety of learner data within the system, for which no explicit permission is required beyond the institution’s agreement to use the courseware. We pursue a policy of openness about the data captured within the system and how it is used. The courseware uses learners’ own data to help them understand their learning process and progress. The courseware also shares learner data with instructors to provide visibility into student behaviors and performance, to help instructors facilitate more effective learning.
We seek consent from students and faculty to use learning data for research and analytical purposes, following best practices established by Carnegie Mellon University’s Open Learning Initiative (OLI). Implemented with process oversight from Brigham Young University’s Institutional Review Board (IRB) and Carnegie Mellon University’s Institutional Review Board (IRB), this approach uses an opt-in/opt-out form to confirm user consent for authorized researchers and research communities to use their de-identified data in research studies. Students may opt in or opt out repeatedly, allowing them to change their minds about participation at any point. Lumen Learning’s use of learner data from outside the courseware is governed by agreements with participating institutions that contribute additional learner data to Lumen Learning projects.
Data Capture and Inventory
Lumen courseware captures comprehensive data that is meaningfully contextualized with semantic markup. These measures strengthen data quality, analytical capability and searchability. Contextualized data allows us to conduct a full spectrum of analyses and discovery in support of student learning. It future-proofs the project by providing the means to explore new questions or develop new data models as our understanding of the courseware and learning science evolves.
Data points to be captured for Lumen courseware functionality and research purposes are listed in a Data Inventory available upon request, along with source(s) for gathering each data point.
Lumen’s Waymaker courseware captures and uses personally identifiable information (PII) such as name, email, telephone and social media contact information to facilitate a variety of functions within the courseware (e.g. personalized teaching interactions). For research purposes, we may seek the ability to track a single student’s progression taking courses over successive terms in order to analyze learning efficacy of Lumen’s courseware over time. In this case we seek an optimal approach for tracking a student using available data, in ways that protect learners’ privacy.
Uses of Data, Data Models and Analysis
Lumen Learning seeks to identify the most important feedback loops around improving student learning, including the use of data to encourage metacognition and to expose and reinforce good behaviors. Data capture, research and analysis focus on:
- Feedback loops for students
- Feedback loops for faculty
- Feedback loops for course design
Current learning science informs our initial hypotheses about these feedback loops. Over time our hypotheses and research strategies evolve along with the courseware and our understanding of its impact on learning. Data analyses drive continuous iterative improvements, accompanied by success measurements to gauge efficacy.
Lumen researchers compile research questions to drive data collection, the research plan and data model development. To address these questions, we apply a variety of analytical techniques to better understand and improve student learning including techniques such as:
- Learning analytics: Analytics that validate learning has taken place
- Engagement analytics: Analytics documenting measurable activities such as levels of interaction with content, what happens in the classroom, personal interaction, etc.
- Progression analytics: Analytics that gauge movement through a course and/or an education program over time
Wherever possible and appropriate, courseware design and research data models support variability and divergent pathways for students to achieve success, rather than “one size fits all.” We employ multiple data models such as:
- Cognitive models: How are students learning effectively?
- Predictive models: What are we trying to predict? How can we use predictions?
- Adaptation models: What approaches and practices will better support the learner? The instructor? Courseware efficacy?
- Assessment models: What types of assessments are most effective at demonstrating mastery of learning objectives?
- Iterative improvement models: What is most effective in facilitating continuous improvement in the courseware?
We are committed to making research findings publicly available in order to broaden the impact of this work on education and learning science, while maintaining privacy protections for learners.
Lumen Learning seeks to generate insights and provide evidence for practices that reduce inequalities in access to education and students’ ability to succeed.
Security, Risk and Liability
Lumen Learning and its partners employ best practices around information security to ensure courseware, integrations and personally identifiable data remain secure. These practices impact courseware architecture and functionality as well as the behaviors of learners, instructors, institutional staff and the courseware provider.
Data Architecture, Systems and Technologies
Lumen courseware uses one or more open data stores, like the PAR Framework data warehouse, to make de-identified data available to authorized learning research communities for the purposes of broadening impact on education generally. We reuse and borrow any tools and assets that can reasonably be reused or borrowed.
Continuous Consideration of Research and Data Practices
As our work progresses, we will review strategies, policies and practices around data and research at least annually to assure that they align with recommended standards among researchers, learners and educational institutions. We also update the published Data Inventory at least annually.