Concept
Question, learning context, intended decision, and feasibility
STUDY PORTFOLIO
Studies With Visible Research Status
These records cover learning, teaching, and research practice. Completed evidence records show what was reviewed and what it can support. Planned studies show methods, safeguards, and the decisions still ahead.
Portfolio status
- Portfolio records
- 6 Two completed evidence records and four planned studies
- Completed
- 2 Public evidence and methods syntheses
- Planned
- 4 No recruitment or data collection
- Last reviewed
- 28 August 2026 Public status register
CONTROLLED LIFECYCLE
A Stage Is a Claim That Must Be Supported
Each record states what was reviewed or what would be tested, which learning outcomes matter, and where uncertainty remains. Completed syntheses do not become intervention findings, and planned methods do not become results before observation.
Protocol drafting
Design, outcomes, implementation, and analysis decisions
Review
Scientific, education, ethics, privacy, accessibility, and security determinations
Registration and execution
Time-stamped record, recruitment, and data collection
Analysis and reporting
Results, uncertainty, limitations, artifacts, and corrections
COMPLETED RECORDS
Evidence already reviewed
These records synthesize public sources and methods guidance. They report bounded conclusions, not new participant outcomes.
SL-005Stage 05: Analysis and reporting
Completed evidence synthesisWhat higher-education AI tutoring studies actually measure
Research question
What can controlled studies tell educators about AI tutoring, learning time, and learner experience in higher education?
Synthesis method
Completed rapid evidence synthesis of publicly available controlled higher-education studies, with a focal randomized crossover trial and supporting research standards.
Target population or material
Published higher-education studies of AI tutoring and active learning. No Santaros participants, recruitment, or proprietary learner data were used.
Comparison frame
AI tutoring conditions compared with active-learning or instructor-led teaching in the reported source studies. Design differences were retained rather than pooled.
Main validity limits
Single focal randomized trial, one course context, intervention specificity, outcome alignment, unmeasured educator effects, and publication or selection bias.
Public record
Completed evidence brief, extraction table, source links, limitations register, and a bounded replication question list.
Synthesis outcomes
- Learning performance
- Time on task
- Engagement and motivation
- Study design quality
- Transfer and generalizability limits
SL-006Stage 05: Analysis and reporting
Completed methods synthesisTeaching reproducible research as a learning outcome
Research question
Which teaching and reporting practices make reproducibility teachable, assessable, and useful to scientific learners?
Synthesis method
Completed cross-standard methods synthesis of education research, data stewardship, evidence-synthesis, and research-reporting guidance.
Target population or material
Public standards and guidance relevant to research-methods, data-science, and research-software teaching. No human participants or learner records were analyzed.
Comparison frame
Outcome, provenance, uncertainty, and sharing requirements compared across standards. This record makes no causal comparison between curricula.
Main validity limits
Standards are normative guidance rather than learner-effect estimates; terminology, discipline, licensing, and institutional requirements vary.
Public record
Completed teaching checklist, cross-standard comparison, implementation prompts, and a planned learner study specification.
Synthesis outcomes
- Outcome definition
- Provenance completeness
- Environment recoverability
- Evidence traceability
- Learner transfer requirements
PLANNED STUDIES
Questions still to be tested
These prospective records are not findings. Recruitment, ethics determinations, and registration will be shown when they exist.
SL-001Stage 02: Protocol drafting
Planned intervention studyScientific reasoning instruction with AI assistance
Research question
Can structured AI-assisted instruction improve how graduate learners generate, test, and revise scientific hypotheses?
Candidate method
Proposed counterbalanced learning study using matched hypothesis-generation, critique, and transfer tasks.
Target population or material
Graduate learners and early-career researchers in participating science programs. Disciplines, eligibility, and sample-size rationale are not yet specified.
Candidate comparison
Matched instructional tasks completed with a structured AI tutor and with instructor-authored resources, with task order counterbalanced where feasible.
Main validity risks
Task equivalence, prior AI experience, instructor effects, disciplinary heterogeneity, rater reliability, and benchmark contamination.
Planned public artifacts
Public protocol, lesson specification, scoring rubric, de-identified learner responses where permitted, and analysis code.
Candidate outcome measures
- Hypothesis testability
- Evidence-to-claim alignment
- Confidence calibration
- Delayed transfer
- Inter-rater agreement
SL-002Stage 02: Protocol drafting
Planned intervention studyReproducible analysis training with AI assistance
Research question
Can learners produce a more reproducible scientific analysis when AI assistance is embedded inside explicit provenance instruction?
Candidate method
Proposed blinded reproduction challenge embedded in a graduate quantitative-methods module using open datasets and containerized environments.
Target population or material
Graduate learners, research software trainees, and early-career analysts. Expertise thresholds and scientific domains are not yet specified.
Candidate comparison
Provenance-first AI-assisted workflow instruction compared with conventional worked examples and documentation.
Main validity risks
Dataset familiarity, environment drift, instructor assistance, reproducer expertise, tolerance definitions, and hidden manual steps.
Planned public artifacts
Teaching protocol, provenance schema, reproducibility checklist, benchmark tasks, reference containers, and model-configuration record.
Candidate outcome measures
- Reproduction success
- Trace completeness
- Decision justification
- Delayed transfer
- Environment portability
SL-003Stage 02: Protocol drafting
Planned intervention studyTeaching evidence synthesis under scientific disagreement
Research question
Can structured AI-assisted instruction improve how learners represent consensus, uncertainty, contradiction, and evidence quality?
Candidate method
Proposed expert-annotated learning benchmark using source packets with supporting, null, conflicting, indirect, and lower-certainty evidence.
Target population or material
Graduate learners and early-career researchers in evidence-intensive disciplines. Domains and prerequisite knowledge are not yet selected.
Candidate comparison
Structured AI-assisted synthesis instruction compared with a conventional critical-appraisal worksheet using the same source packets.
Main validity risks
Source-selection bias, expert disagreement, domain knowledge, benchmark leakage, rubric sensitivity, and construct-irrelevant writing ability.
Planned public artifacts
Teaching protocol, licensed source packets, expert annotations, learner rubric, error taxonomy, and benchmark harness.
Candidate outcome measures
- Citation support
- Contradiction detection
- Evidence certainty
- Selective omission
- Delayed transfer
SL-004Stage 01: Concept and scoping
Planned field studyMentored team learning in shared AI research workspaces
Research question
How do shared AI workspaces affect teaching, feedback, coordination, and accountability in research teams?
Candidate method
Proposed prospective mixed-method field study with baseline, adoption, and follow-up periods across mentored research teams.
Target population or material
Small research teams that include a mentor or instructor and graduate or early-career researchers. Recruitment and institutions are not yet specified.
Candidate comparison
Within-team baseline and workspace-use periods, with staggered adoption or comparison teams considered during protocol development.
Main validity risks
Tool novelty, self-selection, group-level clustering, mentor style, confidential work, maturation, and observer effects.
Planned public artifacts
Study protocol, event-log specification, mentoring observation guide, governance template, and de-identified implementation case reports.
Candidate outcome measures
- Feedback latency
- Decision traceability
- Researcher learning
- Duplicate work
- Appropriate escalation
GOVERNANCE
Human-Participant Work Requires a Documented Determination
Sample sizes, recruitment channels, participating institutions, and review bodies are not named because they have not been verified for publication.
Before any human-participant work begins, the responsible institution or authorized review body must document the required review, exemption, waiver, consent, privacy, security, and data-sharing conditions.
CONTRIBUTE TO A STUDY
Bring Domain Expertise Into Protocol Development
We welcome scientific collaborators who can improve tasks, outcome definitions, validity checks, and real-world constraints before a protocol is finalized.
Participation, authorship, data access, and governance are documented before research begins.