Evaluation Plans

The following sample evaluation plans and timelines are included in this section:
Comprehensive Evaluation Plan, Data Collection Timeline, Project Time Table; for conceptual models of CISE REU sites, refer to the Logic Model section of the toolkit.

Evaluation Plan Template

Goals & Desired Outcomes Activities Example Measures Data Collection
Goal 1: Immerse students into research lab culture
Desired Outcomes:

  • Students participate for whole year
  • Students feel accepted and feel as if contributing to their lab
  • Expectations for REU student to contribute to lab
  • Lab meetings & social activities
  • Doctoral student mentor reaching out to REU student
  • Student participation
  • Student reflections
  • Attitude scale that assess knowledge & feelings
  • Pre-post test
  • Interviews & focus groups
  • Student interviews
  • Longitudinal data Collection (Years 2 and 3)
Goal 2: Students understand computing research methodologies
Desired Outcome: Students publish technical reports and peer-reviewed articles
  • Lab activities
  • Guidance from faculty and doctoral students
  • REU cohort activities
  • Attend graduate research seminar series
  • No. of student papers submitted and accepted
  • Quality of student participation in formal and informal research presentations
  • Pre-post test
  • Interviews & focus groups
  • Student Research Conference
  • Student and mentor interviews
  • Longitudinal data Collection (Years 2 and 3)
Goal 3: Greater student awareness of computing research disciplines
Desired Outcome: Students able to articulate the research focus of their lab and of the labs hosting other REU Teams.
  • Student’s weekly reports in lab meetings
  • Participation in Student Research Conference
  • Visit other labs and interact with REU cohort
  • Student knowledge of computing research areas
  • Pre-post test
  • Student Research Conference
  • Interviews & focus groups
  • Longitudinal data Collection (Years 2 and 3)
Goal 4: Student support
Desired Outcomes:

  • Students interact easily with peer and faculty mentors
  • Students feel they know where to go to get information and help
  • Provide doctoral student mentor and faculty advisor
  • Provide interaction with McNair Scholars
  • Monthly development seminars
  • Mentoring satisfaction scale
  • Identity development scale
  • Social, academic, & psych support scale
  • Sense of belonging scale
  • Pre-post test
  • Interviews & focus groups
  • Student interviews
  • Longitudinal data Collection (Years 2 and 3)
Goal 5: Graduate school preparation
Desired Outcomes:

  • Students take and score well on the GRE
  • Students apply to and are accepted into graduate programs
  • GRE prep course
  • Monthly development seminars
  • No. of students taking GRE
  • Student GRE scores
  • Students intent to apply to grad school
  • Students application and acceptance to grad school
  • Pre-post test
  • Interviews & focus groups
  • Longitudinal data Collection (Years 2 and 3)

Data Collection Time Table Template

Instrument Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun Jul
Student
Student Pre-Survey Analyze Instrument development/refinement Conduct
Student Post-Survey Conduct
Focus Groups (2) Transcribe Analyze Protocol development/refinement Conduct
Individual Interviews (4) Conduct
Faculty & GA Mentors
Faculty/Mentor Survey Analyze Instrument development/refinement Conduct
Milestones
NSF Fast Lane Report Due
Executive Summary Due
IRB renewal Due

Project Time Table Template

Feb – April
  • Develop recruiting materials, begin recruiting, interview and select students
  • Develop/modify summer experience based on prior year feedback
June – August
  • Students participate in an 8 week summer experience (approximately 36 hours of research and 4 hours of GRE prep course, weekly)
  • Pre and post summer experience assessments and formative and summative evaluations conducted at beginning and end of 8 weeks.
  • Student Research Conference, in conjunction with McNair Scholars, in August
Sept – May
  • Students continue research during academic year experience (approximately 10 hours per week).
  • Pre and post academic year experience assessments and formative and summative evaluations conducted at beginning and end of academic year.
May
  • Annual project evaluation and reporting.