Division of Student Affairs
Washington State University College Assistance Migrant Program
Peer Evaluation Report
Prepared by:
Amas Aduviri
December 8, 2017
A complete PDF version of this report can be requested by contacting the Division of Student Affairs via email or call (509) 335-4531.
Introduction
This evaluation report is the result of a request by Lucila Loera, CAMP Interim Director
and Assistant Vice-President, Office for Access, Equity & Achievement, Washington
State University (WSU), College Assistance Migrant Program (CAMP), for an onsite
peer evaluation. The external peer evaluator mainly examined specific aspects of the
program related to student file review and financial files review as requested by the
program. The external peer evaluator visited the program for two and half days.
According to its website, WSU was established in 1890 and the main campus is located
in Pullman, with other campuses in Spokane, Tri-Cities, Vancouver, and Everett. WSU
is a comprehensive four-year university that offers strong professional career programs
designed specifically to meet the needs of its students. With over 20,000 students
enrolled on the main campus and another 10,000 students on other campuses, about 30
percent are multicultural. According to its fact sheet, WSU fulfills its mission as
Washington’s land-grant research university by delivering knowledge and the benefits of
research activities to people throughout the state. WSU’s ultimate goal is to become
recognized as one of the nation’s 25 public research universities, preeminent in research
and discovery, teaching and engagement by 2030.
The WSU CAMP program has been in existence since 2006. Since the inception of the
first grant, the WSU CAMP program has served over 500 students prior to receiving the
funding for this grant cycle this year. By the end of the third grant cycle, more than 250
students will be served by 2022. It’s important to point out that WSU is one of the fewer
four year residential institutions in the country that continues to serve more than 50 new
CAMP students each year. The overarching goal of the program is to help students break
the cycle of poverty by breaking down educational barriers and facilitating their progress
toward post-secondary degree attainment. Most of the participants are recruited from the
Yakima valley all the way to Brewster in the Okanagan valley. According to the WSU
grant proposal, most of the agricultural work in Washington focuses primarily on picking
apples, cherries, pears, red raspberries, and hops. Also, Washington is second only to
California in the number of agricultural commodities produced, including the wine
industry.
WSU CAMP regularly conducts evaluations because it was included in the proposal
narrative (Quality of Program Evaluation, p.22-25). The WSU CAMP program is
currently staffed with the following staff members: Lucila Loera, Interim Director and
Assistant Vice-President, Office for Access, Equity & Achievement; Rafael Benavides
Pruneda, Academic Coordinator & Retention Specialist; Jesus Enriquez, Recruiter; Maria
Barrera and Adi Villanueva, Technical Assistants; and Ian Norland, Program
Coordinator. For this evaluation period, the program is serving 52 students out of 50
funded to serve.
Common characteristics of WSU CAMP program:
Outreach and recruitment:
The WSU CAMP provides outreach and recruitment of
potential participants in the state of Washington, specifically in the following target areas
where most of the farm workers are concentrated: Brewster, Bridgeport, Mabton, Othello,
Sunnyside, Toppenish, Wapato, Warden, Wahluke, Wenatchee, and Yakima area. The
Recruiter spends all year-round building connections with local feeder school districts,
and state and community agencies. Constant follow up to potential students is done in
order to meet the numbers of students served by the program.
Orientation: The WSU CAMP Orientation typically lasts two days and is intended to
help prepare recruited students and parents get acquainted with the program and staff.
The bilingual student and parent orientation occurs in May where students are informed
about the financial aid packages and enroll in classes for fall semester. This is a
collaboration effort with the university’s summer orientation programs and the
Multicultural Student Services (MSS) program. In addition, the program provides a
student and staff retreat in the fall as an activity for group bonding, helping students
develop trust in staff, and emphasizing goal setting and overcoming obstacles. These
orientation activities also provide an opportunity for staff to assess each student’s
readiness to participate fully in the program and allow students to learn about the
expectations of the program.
Tutoring: WSU CAMP participants are offered tutoring services for the duration of the
time that the students are in the program. The tutoring service mainly focuses on math,
chemistry, biology, economics, and social studies. Participants can receive additional
tutoring through the Multicultural Student Services Office.
Advising: WSU CAMP commonly provides academic advising and personal and career
counseling to all participants. This includes providing assessment of career interests,
aptitudes, and motivation. The Academic Coordinator & Retention Specialist is the main
staff to provide academic advising.
Other Support Services: Other services include CAMP workshops, mentoring through
the Multicultural Student Services, and transition and follow-up services as described in
their grant proposal. Also, counseling in the institution is available to students to assist
them in dealing with personal problems, mental health issues, substance abuse, and other
issues. In addition, WSU CAMP provides participants with a stipend for expenses not
provided by the institution.
Methods of Evaluation
In order to fully investigate the characteristics and outcomes of the WSU CAMP
program, the external peer evaluator used a variety of methods including surveys,
interviews with participants and staff, observations, document reviews, and meetings
with program staff. WSU CAMP provided access to all relevant records and set up
interviews with appropriate staff. Although interviews were confidential, anonymity is
not assured.
Student Survey: The external peer evaluator developed and administered an online
survey using Qualtrics to survey current students. The survey asked participants about
their motivations for participating in CAMP, satisfaction with the program, and some
demographic information. The response rate was 63.4% (33/52) of current enrolled
students in CAMP. This was a great response rate by the group.
The pages that follow provide a more detailed observation as the external peer evaluator
saw and heard from the various interviewees with whom he met. These elements serve as
a prelude to reflect on the program's strengths and recommendations made at the
conclusion of this report.
Areas Reviewed
I. GPRA Results
Observations:
- For the current 2016-17 academic year, WSU CAMP program was funded to
serve 50 students; however, the program served 52 students in fall semester.
There is no GPRA results for this cohort since this is the first year of the five-year
grant cycle. The GPRA results will be reported in November 2018. - Number of students served – During the last five years, the WSU CAMP program
served 256 new participants. For the GPRA reports, the number of participants
served was 289 which includes persisters/returners. WSU CAMP has a strong
recruitment plan and is able to consistently meet the number of students to be
served as outlined in the grant proposal. - GPRA 1 - The peer external evaluator examined the GPRA results of the last five
years. This measure is calculated by dividing the number of CAMP first
academic year completers (the number of CAMP eligible students who
successfully completed their first academic year of college by the end of the
current reporting period) by the total number funded, as per the approved
application by OME, or the total number actually served (whichever is greater),
minus the number of persisters. The GPRA I for all the five years was 89.2%.
The national target for GPRA I is 86%. Consequently, the program attained with
high marks, the national GPRA target of 86%. - GPRA II – The GPRA II is calculated by dividing the number of students in
GPRA I by number of students who enrolled for their second year at an institution
of higher education. The WSU CAMP GPRA II for all the five years was
99.57%. Therefore, the program successfully attained the national GPRA II target
of 85 percent. - To validate its GPRA I target, the Director submits the CAMP GPRA I
Documentation Form to the Registrar’s Office to verify and attest that all students
were enrolled as full-time and completed their academic year as defined by
policies. For GPRA II, the Academic Coordinator & Retention Specialist verifies
the enrollment of all returning students, including those who are persisters by
using the university’s enrollment database system.
II. Student File Review
With the permission of the Program Coordinator, eleven student files were randomly
selected for file review. From the provided list, the external peer evaluator selected
every 4th student file (4th, 8th, 12th, 16th, etc.) to perform the student file review.
Each student information is well organized in a folder that is divided in different
sections that include: Eligibility, Application & Intake, Academic Services, Financial
Aid Services, Monthly Meetings (contacts) and Exit Plan.
Observations:
- The program's student files are located in the Program Coordinator’s area. The
files are easy to browse through, and there is a cover page that indicates where
students’ forms are located. The student files are well organized and appear to
have complete demographic information, academic progress, financial
information, monthly meetings, and copies of eligibility documentation. - It appears all eleven students clearly met the eligibility requirements for the
program. Nine students received eligibility under the MEP eligibility guidelines
and thus, a copy of the qualifying COE was included in each of their files to attest
to student eligibility. The qualifying dates varied from 2002 to 2016. Two
students met eligibility through the seasonal eligibility guidelines. The amount of
days worked in seasonal work for one student was 80 days and for the other
student was 113 days. Both seasonal eligibilities were verified by a CAMP staff.
All supporting documentation was included for these two students. Therefore, all
eleven participants meet the eligibility established by the Office of Migrant
Education (MEP). - All eleven students appear to be in need of academic support and consequently in
need of a program such as CAMP. Each student file includes a matrix form that
ranks each student’s academic needs. None of the students were admitted under
the university’ special admit criteria. The average GPA for all the eleven students
is 3.26, with 2.65 being the lowest and 3.93 being the highest GPA. For a
research institution such as WSU, the average 3.26 GPA demonstrates that WSU
CAMP students are in need of academic support and other student services. - All eleven students demonstrated need for federal financial aid assistance. The
eleven students’ Estimated Family Contribution (EFC) was from $0 to $832. One
student had a EFC of $19,125. Given the demonstrated EFC, the program is
serving students with high financial needs. Also, the student survey reveals that a
high percentage (35.3%) of students felt insufficient finance as a challenge in
school. Therefore, providing financial services to students in high need is crucial
for these students. - All eleven students were enrolled full time for 2017 fall semester. Each student
file includes the monthly meeting report which highlights the classes and the
number of credits being attempted by each student. The average number of
credits for all eleven students is 14.18 credits per semester with 12 credits being
the lowest and 17 credits being the most attempted. - The Academic Services section include documentation for the 1:1 meetings with
the Academic Coordinator & Retention Specialist. It appears most of the monthly
meetings forms are complete, but in some student files, the monthly meeting
report is missing. This may be because the meeting is being scheduled or the
filing is still in the process of being completed. - The Exit Plan form was not completed because this section is done at the end of
the academic year, during the spring semester.
III. Financial Management
The external peer evaluator examined the financial management of the program. The
Program Coordinator provided the financial files for 2016-17 and 2017-18. The actual
receipts of the grant expenses are filed by month in each folder. The grant expenses can
be tracked using the Administrative Information Systems. In addition, the up to date
expenses are tracked by the Program Administrator through an internal system.
Observations:
- According to the Program Coordinator, a copy of purchases and invoices are sent
to the Business Office for record keeping, as required by federal grants. The
copies of the receipts and invoices are maintained in the Program Coordinator's
office for reconciliation purposes as well as for auditing purposes. A goal for the
Program Coordinator is to keep all files electronically. - All program expenditures are up to date in the university system and the Program
Coordinator can generate an expense report within minutes. The Program
Coordinator is very knowledgeable in navigating from one system to another in
order to monitor all grant expenses, as well as the invoices for expenditure
justification. - The actual expenditures for reviewed year is as follow (rounded to the nearest
dollar): Salaries: $213,338; Benefits: $72,781; Travel: $26,011; Supplies: $3,492,
Contractual Services: $0,00; and Other: $30,289. Total Direct: $345,911. Indirect
at 8%: $27,173. Student stipends and other support: $58,500. Total grant
expenditure for the reviewed FY: $431,584. The total carryover balance for the
next FY is: $975. The program has expended the proposed budget accordingly. - The CAMP program does not receive additional funds from the institution. All
program expenses are solely relied on grant funds. The state of Washington
Office of Superintendent of Public Instruction (OSPI) provides approximatively
$3,750 for additional financial support to WSU CAMP, including other CAMP
programs in the state of Washington. - It is clear there are several tiers of expenditure control to ensure grant funds are
being expended under federal and state procedures and regulations. For example,
there are different steps and cross checking before a check payment is being sent
out to participants and vendors. - According to the reviewed files, for all the grant expenses, an invoice or receipt
was attached for the specific purchase. The external peer evaluator did not find
any missing receipt or an expenditure without a receipt. All filing was done
sequentially following the calendar days of each month. - There were two or three expenses that needed additional justification for those
expenses. The CAMP program did provide the Office of Sponsored Programs
additional information and justification for those expenses. The Program
Assistant receives training and meets with the Business Office to discuss
budgetary issues when needed. - The external peer evaluator found one questionable expenditure. An invoice was
made to rent a space for CAMP participants to play soccer. If the activity was
just to have a fun soccer activity, then the expense might be unallowable.
However, the expense can be justified depending on the initial intention of the
activity. - The program is currently in the process of hiring staff to fill vacated positions.
The positions are expected to be filled in January or February. Therefore, there
may be some substantial carryover balance in the current budget, unless some of
these funds are allocated to other activities related to the completion of the
intended objectives.
IV. Participant’s Online Survey
An online survey was sent to students a week of the onsite visit. Thirty three out of 52
students (63.4%) responded to the survey. In addition, the peer external evaluator
interviewed four students. The following is the feedback provided by current and past
participants:
- According to the online survey, most (35.3%) of the current students learned
about CAMP through the Recruiter. Another group (23.5%) learned about CAMP
from the high school counselor or other staff. These results demonstrate that the
CAMP Recruiter is being effective in recruiting students, as well as informing
counselors about WSU CAMP in the high schools. The outcomes of this
comprehensive recruitment is that each year, the CAMP program has served more
students than the number of students approved in the grant. - According to the survey, most (25.3%) of the students appear to be motivated and
eager to learn. This also was very clear from the interview with the students. Part
of the motivation comes from their willingness to change their lives for better
through the education. - Like many first generation and low-income students, CAMP students face barriers
in getting their post-secondary degree. For many of the WSU CAMP students,
financial barriers and the difficulty of classes seem to be a few of the barriers they
need to overcome. Interviewed students mentioned that the CAMP staff has been
an important factor to continuing their education because of their support and the
services the program provides. - The majority of the CAMP students feel extremely satisfied and satisfied (83.9%)
with their experience with CAMP. Many of them are likely and quite likely
(83.9%) to recommend the program to other students interested in the program. - When asked about two most favorite experiences in CAMP, WSU CAMP
students found meeting with Anita and Ray (previous CAMP staff), and the
student retreat activity very helpful. These are seen as very important to them;
one service is used on a daily basis and the other is a one-time event, but being
important for cohort building. - When asked about two least favorite experiences in CAMP. Students felt
saddened and unhappy with the change of staff, specifically with Anita and Ray
leaving the program. This demonstrates that current students built a strong
relationship with the two staff members. Also, not many, but fewer students
disliked the Critical Literacies Achievement and Success Program (CLASP)
activity. - The last question asked was about how their experience can be improved in
CAMP. Students’ answers varied from keeping the previous staff in the program
to improving tutoring. Many students also mentioned the need for more activities
for bonding and cohort building.
V. Program Management
The external peer evaluator did not examine the details of the program
management, academic resources and services, collaborations, and follow up
services. The focus of the visit was student file review and financial
management. However, the external peer evaluator met with the recruiter for a
short meeting.
Outreach and Recruitment
- The WSU CAMP recruitment effort focuses primarily in the central region of the
state. Most of the feeder school districts are within a 2-4 hour drive from where
the recruiter is stationed. All incoming first year students are required to live on
campus since the institution has a mandatory living on campus policy for first
year students, including CAMP students. After their first year, students can live
off campus in the nearby apartment complexes. - The number of students served for this evaluation period (2016-17) is 54 students.
According to the grant proposal, the goal is to serve 50 students each year.
However, the program is achieving the number of participants to be served
because of the Recruiter’s experience working for several years, and also, because
WSU CAMP is known among the feeder schools. Also, in 2015-16, the program
met its approved number of participants to be served. - According to the survey, many students acknowledged the support and help of the
Recruiter for making their enrollment at WSU CAMP possible. Also, the student
survey indicates that many (27.45%) of the students hear about the program via
word of mouth and through their siblings.
VI. Summary of Strengths and Recommendations
Strengths:
The WSU CAMP program has a presence on the campus and can be recognized for the
following specific strengths:
- Most importantly, the WSU CAMP has consistently meets the national GPRA
measures. This is a result of the ultimate program support to its participants. The
majority of the participants are completing the minimum 24 credits and are in
good academic standing. The average five-year GPRA performance is higher
than the national targets. - The program continues to successfully meet or surpass the number of students
being served each year on campus. The program proposed students to be served
according to their grant proposal is 50 student each year. This is a result of a
great recruitment plan and the ability of the recruiter to connect well with the
feeder schools and other key partners that provide support with recruitment. It’s
important to point out that fewer residential institutions in the country that host
CAMP programs have served 50 students or more each year. - According to the student satisfaction survey, 38.7% and 48.4% of students
strongly agree and agree that WSU CAMP study space is adequate for their study
needs. Also, 45.1% and 38.7% of the students strongly agreed and agreed that the
tutoring support provided by CAMP is adequate. In addition, 35.4% and 41.9%
of the students strongly agreed and agreed that the CAMP bilingual orientation
was helpful to them and their families as they began their education experience at
WSU. - The program continues to provide needed financial support to participants.
Students receive semester stipends as well as other financial support for
emergencies when funds are available. A high number of students mentioned in
the survey that the CAMP staff helped them understand their financial situation
and options. An 80.6% of the students were satisfied with the financial assistance
received through CAMP. The financial support through stipends and other forms
of financial support are important to the success of first generation, low-income
students. Some of the students interviewed were highly motivated and at the
same time grateful for the support of the program. - It appears all grant expenditures are within the guidelines of EDGAR and the
Program Coordinator meets on a regular basis with the Office of Sponsored and
Research Programs that oversees the grant. No unallowable grant expenses have
been recorded during this FY. - The CAMP students’ files are easy to browse and the participant’s support is well
documented in the files. Each file is clearly divided into different sections such as
Eligibility, Application & Intake, Academic Services, Financial Aid Services,
Monthly Meetings (contacts) and Exit Plan, which facilitates the browsing of
students’ files. All reviewed files were complete and well organized and all
eligibility documentation is in place. - All reporting (Annual Performance Report and Revised Budget) was sent within
the timeline requested by the Office of Migrant Education and also made readily
available for the peer external reviewer, including the grant proposal and the
budget narrative.
Suggestions:
The WSU CAMP is a well-established program; however, the program continues to
improve the services to its participants.
- Students in the interview and the satisfactory survey mentioned the need for more
group activities to build a strong cohort. Some feel they are already disconnected
from the other students. They felt the group activities at the beginning of the year
such as the retreat, were strong, but lately there have been almost no cohort
building activities. Perhaps the staff should organize small get-togethers
throughout the year, such as a welcome gathering during the spring semester. - It appears the monthly stipend provided to each of the students is credited directly
to each student’s account. For example, if a student has an outstanding balance,
the stipends reduces the balance because the WSU Financial Aid Office sees the
stipends as part of financial aid support to students. The CAMP Non-Regulatory
Guidance in K3 mentions, “While the Department has no regulatory definition of
the term “stipend”, for the purposes of the CAMP one may consider a stipend to
be project funds provided directly to participating students, or on their behalf, to
offset living or educational expenses that are not included in other financial
support. Stipends should help to ensure the student’s continued participation in
the project and ultimate completion of the first academic year of a program of
study at the IHE. A stipend could, for example, be given to offset the cost of
transportation to and from classes for a commuter student in a CAMP project, or
to help the student purchase necessary books and supplies not otherwise provided
by the project. A commuter CAMP project might also provide its students with a
stipend to purchase meals while on campus for classes.” Therefore, maybe the
program can consult with the Financial Aid Office to find ways to have the
stipends be directly given to each student. - Three main staff no longer work in the program. Many students felt somewhat
frustrated with the changes and some students felt abandoned. I believe this is a
natural frustration from the students because previous staff had built a great
relationship with most of the them. As stated in the grant, the program should
begin the process to hire replacement for those vacancies. The process has
already started and it is expected new staff will be in place at the beginning of
spring semester. - The grant financial recordkeeping is in place. The Program Coordinator is
knowledgeable of the grant management. He will soon transition to only
managing one grant. There is an extensive grant financial procedures manual that
appears to have some outdated information since the creation of the manual. I
suggest the financial procedures manual be revised and simplified to make it more
useful for an everyday use. - As the new staff are hired and added to the program, they should organize a staff
retreat in the summer to work on program development, plan activities, work on
the timeline for the program, and continue to become more knowledgeable about
the grant's goals and objectives, especially since a new leadership will be in place
and new staff members will be hired by the program. - Program staff should start developing the progress of the program in moving from
“knowledge” of evidence to the “skill” of building evidence as being required by
the Office of Migrant Education. These steps should include: developing the
program’s Logic Model, the program’s intervention and relevant outcomes,
determining and setting up an equivalent comparison group, the program’s impact
study which includes data collection during program implementation and a
program’s evaluation timeline, and finally the impact of the study design.
Conclusion
Based on the review of the different areas of the WSU CAMP program, the external peer
evaluator believes that the program is working to be an effective academic support
program and believes certain student support services can be more effective with the
implementation of the suggestions from this report. It is important to note that the
program peer evaluation was intended to assist program staff in examining and improving
their services for students from migrant and seasonal migrant backgrounds. Overall, the
WSU CAMP program is well established as evidenced by the fact that the program has a
strong support from the institution, and is meeting or exceeding most of their grant goals
and objectives.
Finally, program staff should carefully analyze the student survey responses to find cues
where the program can improve their services. When possible, staff should let
participants know of activity changes due to their responses. As a result, participants will
feel listened to and more identified with the program.
The external peer evaluator welcomes any questions or concerns regarding these
recommendations. The external evaluator appreciates the cooperation provided by all
participants in this evaluation. Everyone who was interviewed was very cooperative and
forthcoming with information and documents requested.