Evaluation Plans

The following sample evaluation plans and timelines are included in this section:
Comprehensive Evaluation Plan, Data Collection Timeline, Project Time Table; for conceptual models of CISE REU sites, refer to the Logic Model section of the toolkit.

Evaluation Plan Template

Goals & Desired Outcomes	Activities	Example Measures	Data Collection
Goal 1: Immerse students into research lab culture Desired Outcomes: Students participate for whole year Students feel accepted and feel as if contributing to their lab	Expectations for REU student to contribute to lab Lab meetings & social activities Doctoral student mentor reaching out to REU student	Student participation Student reflections Attitude scale that assess knowledge & feelings	Pre-post test Interviews & focus groups Student interviews Longitudinal data Collection (Years 2 and 3)
Goal 2: Students understand computing research methodologies Desired Outcome: Students publish technical reports and peer-reviewed articles	Lab activities Guidance from faculty and doctoral students REU cohort activities Attend graduate research seminar series	No. of student papers submitted and accepted Quality of student participation in formal and informal research presentations	Pre-post test Interviews & focus groups Student Research Conference Student and mentor interviews Longitudinal data Collection (Years 2 and 3)
Goal 3: Greater student awareness of computing research disciplines Desired Outcome: Students able to articulate the research focus of their lab and of the labs hosting other REU Teams.	Student’s weekly reports in lab meetings Participation in Student Research Conference Visit other labs and interact with REU cohort	Student knowledge of computing research areas	Pre-post test Student Research Conference Interviews & focus groups Longitudinal data Collection (Years 2 and 3)
Goal 4: Student support Desired Outcomes: Students interact easily with peer and faculty mentors Students feel they know where to go to get information and help	Provide doctoral student mentor and faculty advisor Provide interaction with McNair Scholars Monthly development seminars	Mentoring satisfaction scale Identity development scale Social, academic, & psych support scale Sense of belonging scale	Pre-post test Interviews & focus groups Student interviews Longitudinal data Collection (Years 2 and 3)
Goal 5: Graduate school preparation Desired Outcomes: Students take and score well on the GRE Students apply to and are accepted into graduate programs	GRE prep course Monthly development seminars	No. of students taking GRE Student GRE scores Students intent to apply to grad school Students application and acceptance to grad school	Pre-post test Interviews & focus groups Longitudinal data Collection (Years 2 and 3)

Data Collection Time Table Template

Instrument

Aug

Sept

Oct

Nov

Dec

Jan

Feb

Mar

Apr

May

Jun

Jul

Student

Student Pre-Survey

Analyze

Instrument development/refinement

Conduct

Student Post-Survey

Conduct

Focus Groups (2)

Transcribe

Analyze

Protocol development/refinement

Conduct

Individual Interviews (4)

Conduct

Faculty & GA Mentors

Faculty/Mentor Survey

Analyze

Instrument development/refinement

Conduct

Milestones

NSF Fast Lane Report

Due

Executive Summary

Due

IRB renewal

Due

Project Time Table Template

Feb – April	Develop recruiting materials, begin recruiting, interview and select students Develop/modify summer experience based on prior year feedback
June – August	Students participate in an 8 week summer experience (approximately 36 hours of research and 4 hours of GRE prep course, weekly) Pre and post summer experience assessments and formative and summative evaluations conducted at beginning and end of 8 weeks. Student Research Conference, in conjunction with McNair Scholars, in August
Sept – May	Students continue research during academic year experience (approximately 10 hours per week). Pre and post academic year experience assessments and formative and summative evaluations conducted at beginning and end of academic year.
May	Annual project evaluation and reporting.