GTE Links Virginia for Literacy

GMU Segment

Curriculum Project Evaluation Report

November 2, 2001

Curriculum Project Evaluation Report

Introduction

In the spring of 1998, the GTE Corporation, in consultation with the National Institute for Literacy, created and funded a partnership of four Virginia universities which was designed to promote workforce development through improving the technology use and technology skills of undereducated adults.  This project, "GTE Links Virginia for Literacy," included George Mason, James Madison, Old Dominion, and Virginia Commonwealth Universities.  Its specific goals were as follows:

·        To promote better understanding of the basic technology skills that employees need at work.

·        To use technology to link adults who need basic skills to education and training programs.

·        To expand opportunities for adults to use technology to improve work-related basic and technical skills. 

·        To strengthen adult education services through teacher training and technology as a tool for literacy learning.

Each of the four university partners committed to develop a particular project theme.  George Mason University agreed to focus on curriculum development and to assist in this effort, recruited regional collaborators including Northern Virginia Community College (Manassas branch), Arlington Public Schools, and Prince William County Schools.  Although activities at all of these sites included curriculum development and piloting, Prince William County Schools Department of Adult Education in cooperation with the Virginia Employment Commission in Woodbridge conducted the primary pilot of new curriculum.  At this site, between March 2000 and June 2001, a six-week, thirty-hour-per-week curriculum for unemployed and educationally disadvantaged workers was implemented with three groups of learners.  The curriculum had been  developed in an earlier phase of the GMU segment of the Links project.  The remainder of this report describes insights gained from this activity and from the students who participated in it. 

Nature of the Curriculum

The “GTE curriculum” developed under the GMU segment of the Links project consists of three integrated strands: job readiness, GED preparation, and basic technology skills.  Each strand was taught by a different teacher, but all teachers worked closely together to optimize the connections between strands (resumé work for job readiness parallels resumé developed in an earlier phase of the GMU segment of the Links project creation on the computer, for example).  Focus on these three curriculum themes involved direct instruction by the three teachers, but also involvement in VEC employment preparation activities and occasional field trips to community resources.  Classes met six hours each weekday for six weeks.

Student Participation and Rules for Involvement

Ten individuals enrolled in and eight completed the first pilot course.  Ongoing evaluation by instructors and solicited student feedback resulted in refinements to the curriculum for two successive classes graduating nine (of eleven) and seven (of twelve) participants, respectively.  Students who did not adhere to the commitment policy they agreed to at the outset (maximum of two tardy arrivals or two absences; participation throughout the full six weeks of instruction) were dismissed for noncompliance.  Across groups, the completion rate (a constant issue in the complicated and often precariously balanced lives of adult basic education participants) was 75%.  Strong evidence exists that students who persevered experienced the six-week course of instruction as intense, and academically, personally, and socially beneficial—even life changing. 

                                                Participant Profile

Admission Criteria

Initial qualification of potential participants was determined by TABE (Test of Adult Basic Education) testing.  The screening process included several other criteria that were balanced against each other to determine which students were best prepared to benefit from the planned curriculum.  For example, non-native English speakers with a lower TABE score but high evidence of motivation and goal orientation were considered qualified.  Attachment A (Admissions Intake Survey Form), Attachment B (Student Acceptance Form), and Attachment C (Applicant Interview Form) illustrate the attempt to enroll only highly motivated, committed students using transparent admission criteria and requisite written agreements.  An admissions committee anonymously reviewed applicants based on the following criteria and components.

·         TABE Test scores (Reading and Math) of 9.0 grade level or higher

·         Baseline Writing Sample

·         Predictable Match between Program Goals and Individual Needs

·         Oral Interview

·         Written Survey

·         High Level of Commitment for Participation and Attendance

·         High Level of Interest in Education

 Languages/Cultures Represented

Collectively, non-native English speaking students represented a majority in each of the classes; in all, numerous languages and even more cultures were represented, as Spanish speakers derived from multiple countries of origin.  Four participants emigrated from several African countries, one from the Philippines, one from Korea, one from Taiwan, and the rest from Spanish speaking countries.

Baseline Skill Levels

Skill levels on program entry in the primary competency areas (reading, writing, and computer literacy) indicate a broad span.  Using results from the TABE tests, initial reading scores ranged from 5.1 to 12.9 (average, 9.4), and writing scores from 5.8 to 12.9 (average, 9.2).  Entering students who were computer literate prior to instruction (defined as sufficient prior exposure to demonstrate basic keyboarding and word processing skills, and basic navigation on a Windows platform and on the Internet) comprised from .33 to .50 of participants in each class, .44 overall.  Computer literacy in this case does not imply adequate skills for functioning in an office or college environment. 

 Information Gathering Procedures

Evidence of student satisfaction was documented in participant reaction surveys collected from program participants at the conclusion of each course, a group interview format (July 2001), individual interviews of selected students (August 2001), and continuous informal instructor assessment. 

Exit Survey

All students were asked to complete a two-page evaluation form on completing the program to determine how well served they felt by the experience, to isolate most and least helpful aspects, and to gather recommendations for improvements.  (See Attachment D: Sample Feedback Instrument.) 

A 5-point Lickert scale (5-excellent; 1-poor) was used to evaluate reaction to instructors and to rate general class impressions.  In both categories, exceptions to "5" ratings were very rare; none was lower than "4".  These included reactions to teacher competence, preparation, organization, friendliness, helpfulness, and quality of interactions with students.  Students also rated materials and class usefulness in improving skills and meeting personal goals.  Typical examples of responses to open-ended questions in specific categories are provided in the Findings section. 

Interviews

Group Interview

On July 8, 2001, a small group of individuals from the first and second classes gathered at the home of Instructor Debby Cargill in Woodbridge, VA.  Renewing friendships, catching up on each other's lives, and enjoying a social event constituted a significant part of the agenda but this gathering also provided an opportunity for former students to discuss their program experience in a group setting.  The group included two instructors and two representatives of George Mason University (GMU).  Participants knew in advance that they would be asked to discuss/evaluate the program as part of the evening's agenda.  All expressed delight at being reunited, enthusiasm about their future possibilities, and hope that the program would continue to benefit others. 

Both instructors and GMU representatives probed student reactions to the program.  Students spoke freely.  Questioning pursued student perceptions in these general areas: 

·          Employment (current and anticipated changes)

·          Goals (fulfilled by program and new or future goals prompted by the program experience)

·          Program Assessment (most and least valued elements; recommended changes)

·          Program Influence (on student's family, attitudes, goals, competencies; personal changes). 

See Attachment E for examples of the kinds of question prompts that spurred the conversation (not comprehensive or strictly adhered to).

Individual Interviews

On separate occasions in August, several students not able to attend the group interview were individually questioned, using the same basic format and questions described above.  Their comments portrayed impressions similar to those expressed in the group setting and were consistent with opinions and recommendations expressed in the written exit surveys.  Enthusiasm for the benefits of the course were expressed without solicitation, but differing levels of confidence about future prospects surfaced.  These differences can be accounted for in part by differences in family circumstance, personality, and especially English language proficiency.  (See Findings for specific results of interviews.)

                                                        Findings

Exit Survey Data

Expectation Fulfillment

A large majority declared that the program had satisfied either specific individual participation motivations or general satisfaction with the overall program.  Success with program elements cited as ways their personal expectations were fulfilled included GED preparation, English, typing, problem-solving skills, computer, writing, job readiness, and math.  Also named were appreciation for being able to work in teams and learning " what it takes to get a job."  At least four persons said "all" their expectations were met and several more said they learned much more than they had expected to at the outset.  One student acknowledged that her "writing tremendously improved;" another said she "can do math I never thought possible." 

Most Helpful/Meaningful

The role of the instructors was crucial to the way students received the program.  Teachers perceived as friendly, caring, helpful, motivating, and professional came up repeatedly as the overwhelmingly most positive aspect of the classes.  Some students recognized that [teachers] "wanted to improve our self-esteem," or acknowledged that they had improved their social as well as academic skills as the result of the "family atmosphere," the "class unity," or the opportunities to "work as a team."  Teachers were respected for their professionalism, but were much appreciated for being "caring", even "loving," and paying attention to "all the little details" that expressed that—like bringing food and treats.  One student volunteered as most important lesson, "I learned that no matter where we come from, we are all equal." 

All aspects of the curriculum were mentioned at least once as being the "most helpful," e.g., resumé creation; interview skills; job readiness; and math skills, or even "all" curriculum areas.  Learning computer skills was most frequently indicated as the greatest program benefit.

Recommended Changes

Respondents were virtually unanimous in their assertion that the program should be longer in order to take full advantage of the learning possibilities or to work on areas of perceived individual deficits.  The suggestions ranged from increasing the course by a week or two to extending it to six months—in one case, a year.  Curriculum areas specifically singled out as needing more attention were computers, math, GED studies, and ESL. 

Other suggestions for improving the program included making it possible to take the course a second time (two people), taking the GED test while still in the program, and dividing the time allocated for each class more equally on the same day (vs. longer blocks on the same subject).  One person recommended more academic tests to check interim progress; another thought Internet access was needed.  Providing a second scanner and a designated lunchroom were also suggested as potential improvements.  Two people expressed the desire for some ongoing relationship with their classmates, e.g., coming back together occasionally for a social occasion or attending future graduations as a group. 

Commitment to Change

Changes in student attitudes, sense of self-empowerment, and motivation to succeed are reflected in the numbers who on exiting the program indicated a commitment to enroll in further education or training or to upgrade employment.  Many graduates indicated their intentions to continue with their educational pursuits.  Examples of their choices range from enrolling in an ESL class, to attending computer training courses in the private sector, to pursuing computer certification, to attending Northern Virginia Community College. 

Employment

Some students signaled changes in employment status.  One student enlisted for military service.  One indicated a position upgrade that included benefits; one was promoted to a supervisory role in a floral shop; one reported a pending interview for an AT&T job.  One student secured a job with the US Postal Service; two indicated a desire to start their own businesses.  Some students remained in current employment (security guard; office administrator; bank teller; retail, grocery, or auto industry worker).  One student who obtained employment as a caseworker in a family social services agency after completing the program attributed her successful resume and interview to skills she learned in the class.  Some, especially homemakers with minor children, were not seeking employment, but indicated intentions to continue with studies now or later or to work outside the home when their family responsibilities changed. 

In addition to this evidence of individual goal satisfaction, the following statements give a sense of the magnitude of impact on individual lives. 

·        "I am going to get out of here a whole different person than when I came in!" 

·        "The computer program has given me a whole new world to look forward to." 

·        "This program makes me step forward for my future."

·        "I am no longer fearful of a computer.  I now have the confidence and knowledge to take further computer courses, and I definitely plan to do so."

·        "I was completely committed to this program.  I now have the desire to continue…to learn more, more, more."

·        "I think it was one of the best experiences of my life—sharing time with your teachers and partners, and learning at the same time.  I feel like I am a new person, and ready to go forward."   

Interview Data

Themes arising from the group interviews indicate these achievements: 

·        Strong gains in confidence

·        Resumé-building skills valued (can describe who I am for potential employer) 

·        Greater sense of control over personal life

·        Greater expression of initiative, optimism, looking to future

·        Less apologetic, less reclusive, less intimidated by new challenges

·        Interest in, and commitment to future self-improvement steps (e.g., taking credit-bearing courses at NOVA or elsewhere, re-taking failed portions of GED test; taking a job in bank after conquering math phobia)

·        Sense of liberation (mind opening insights) based on program experiences

·        Continuing obstacles in personal lives (family obligations, financial limitations, employment instability, language deficits, social isolation) will challenge future advancements

·        Strong support of cohort community (teachers and students) extremely important to sense of possibility, willingness to take risks, dream new dreams

·        Documentation of learning (passing GED score) important to self-concept

Interviewees expressed support for much longer classes, and some ongoing social contact with their fellow students after graduation.  One student suggested that graduates could provide motivation and support for future cohorts by volunteering to speak to them.  Another recommended making a video of program participants giving one-minute presentations about their individual program experiences.  Such a video could be used to attract future funding so that other students of similar needs would more likely be served.    

A particularly articulate student spelled out how her attitudes and future prospects had changed as a result of what she had learned.  She feels more self-confident, knows how to access resources she may need, is able to stimulate and satisfy her own curiosity through reading and research, and can better self-evaluate.  She sees more choices for her life and career, and more opportunity for personal and professional growth.  

Conclusion

The collective goals of this assessment were (1) to document the differences that the program has made in participants’ lives over time, (2) to document how well served participants feel by the program experience, and (3) to contribute to the exploration of how transferable the curriculum might be to other settings. 

Based on direct participant feedback, it appears that the experimental curriculum developed in this GTE-funded project and refined with three groups of students has had significant impact on participants’ short-term academic performance and long-term emotional and psychological factors such as self-confidence and self-esteem.  Together, these advancements predict greater career potential and satisfaction over time.  Participant enthusiasm is apparent in students’ praise for instruction, instructors, and the program experience.  Their desire for "more of the same" is a clear indication that the curriculum is addressing and meeting real needs.

The results of this evaluation report suggest that the GTE/GMU curriculum has significant potential to help other students develop their personal and employment potentials.  As a result, we are pleased to share it at statewide conferences (as we done several times to date) and through the Virginia Adult Learning Resource Center (VALRC) website.

 Attachment A

ADMISSIONS INTAKE SURVEY FORM

Name___________________________________________Date____________________

Address_________________________________________Phone__________________

Please answer the questions below.

·              What was the last grade you completed?

·              What factors were involved in your decision to leave school?

·              Have you had any further education or training since then?

·              Have you ever take the GED test before or studied for it?

·              What if you don't attain your GED on your first attempt?

·              Why do you want to participate in this program?

·              What will you do to make this program work for you?

·              Daily attendance and punctuality are required. How will you achieve this?

·               What are your goals after completing this program?

 Attachment B

PRINCE WILLIAM COUNTY SCHOOLS

ADULT EDUCATION

P.O. Box 389

Manassas, Virginia 20108

(703) 791-7357

ACCEPTANCE FORM

_______________                  I accept placement in the GTE Links Virginia for Literacy GED Program, which will start on October 10, 2000.

_______________                  I do not accept placement in the GTE Links Virginia for Literacy GED Program for the following reason:

________________________________________________________________________

________________________________________________________________________

COMMITMENT POLICY: Students will commit to this program as if accepting a new job.  Regular attendance is necessary for the success of this program and absences and coming late to class will not be accepted. We realize that there are uncontrollable reasons for absences, but it is our policy to dismiss students who have missed more than two days and/or who have been late to class two times or more.

________________                I agree to the Commitment Policy and will adhere to daily attendance and punctuality.

________________                I do not agree to the Commitment Policy. (If you cannot agree to the policy, then you are subject to dismissal from the program.)

Applicant Signature_______________________________________________________

Date___________________________________________________________________

Note: This form must be signed and brought to class on the first day, October 10, 2000.  If you decide that you cannot sign the Commitment Policy, then you must call the Project Coordinator at (703) 791-7358.  We will give you guidance in selecting another program that may be right for you.

Attachment C

PRINCE WILLIAM COUNTY SCHOOLS ADULT EDUCATION

GTE LINKS FOR VIRGINIA GED COURSE

APPLICANT INTERVIEW

NAME__________________________________________DATE__________________

EMPLOYMENT STATUS______ Employed______ Unemployed______ Not Working

WORK ETHIC__________________________________________________________

PLANS MADE TO ALLOW FOR STUDY TIME_____________________________

PLANS MADE TO ALLOW FOR COURSE PARTICIPATION_________________

LEISURE ACTIVITIES, HOBBIES, ETC.___________________________________

FAMILY SETTING/STRUCTURE_________________________________________

GREATEST ACCOMPLISHMENTS_______________________________________

CONCEPT OF SELF_____________________________________________________

PERSONAL GOALS_____________________________________________________

_______________________________________________________________________

COMMITMENT TO PROGRAM                            High            Average          Low

OVERALL IMPRESSION:  

MENTALITY                                     Quick        Average        Slow

DEGREE OF INTEREST                  High          Average        Low

DEGREE OF MATURITY                High          Average        Low

GENERAL APPEARANCE_______________________________________________

SCORES      TABE LEVEL D____________ Total Reading____________ Total Math

                         BASELINE WRITING SAMPLE____________________ SCORE (1-6)

RECOMMENDATIONS                               _______________________ Highly Qualified

                                                                        _____________________ Qualified

                                                                        _____________________ Not Qualified

COMMENTS____________________________________________________________

________________________________________________________________________

SIGNATURE___________________, GTE Project Coordinator ________________DATE____________

Sample Exit Survey Form

(Available hard copy only.)

GROUP INTERVIEW DISCUSSION PROMPTS

(General guidelines only)

·        Are you currently working?

·        Have you been working regularly since you left the program?

·        Are you experiencing any difficulties in your present job?  With transportation or child care related to your job?  With job expectations?  With supervisor or peer relationships? 

·        Do you think the cohort helped you to prepare for your current employment?  In what ways?  What would have helped more?

·        What were the most valuable parts of the cohort to you?

·        Are there any parts of the curriculum we should leave out?

·        Are there elements in which you would have liked more instruction?

·        What are some specific ways the class helped you?

·        What suggestions do you have for improving the learning experience for other students?

·        How did you feel about the very diverse nature of the class?

·        Have you changed your career goals since you left the cohort?

·        Have you fulfilled (made progress toward) the career goals you had when you entered the program?

·        Has your family been supportive of your career or school goals?

·        What would you like to be doing two years from now?  Five years from now?

·        Have you met new friends as the result of new employment or training experiences?

·        How much do you think the class affected your chances for better employment?

·        Do you think the class changed your attitude about your future?  In what ways? 

·        Are you glad that you participated in the class?

·        Would you recommend it to others?

·        Would you be interested in getting together with other cohort members occasionally?

Table of contents