Outline of Evaluation Project

My evaluation project will be a needs analysis, with a mixed mode method for data collection. I will be evaluating introducing elearning tasks into upcoming engineering courses.

Needs Assessment -·
Who users are,
type of users·
cost-effective·
where the gaps are.

Reason for a 'needs analysis'
I believe that a review of current practice can reveal that little elearning is encompassed within the teaching of engineering. At the best there are electronic notes and assignments made available to students on an electronic filing cabinet. I would like to see more dynamic use of elearning in the form of quizzes and possible group discussions.
Students that study engineering tend to be kinesthetic learners and already use computers in some engineering courses like computer aided design, so they have a familiarity with computers. I suspect that the main reason elearning instruction is not more widely used is due to those that teach engineering not having the skill or knowledge to do so.
The strategic plan of this polytechnic is to provide a state of the art facility and learning environment. That must include the use of modern techniques (such as elearning) where appropriate. And to be student focused. So there is a directive from above that encourages elearning to be developed into our courses. Since we have the job of running the courses for the first time, we have the opportunity to do this.
There is a ‘need for analysis’ to find out if what I believe could be true and if elearning would be aligned with the strategic plan.

What is to be evaluated.
It is intended that elearning be introduced slowy, as instructors get up to speed with the technology. So a pilot is proposed, one which includes the use of interactive learning material with follow up quizzes, and a site which will allow instruction form f2f for review.
I would also like to check the viability of using a discussion board, I not sure how this could be used inside of the engineering curriculum but there may well be instances where it would be appropriate.

How the evaluation will work
It will use a 'multiple methods' model to collect data and information. Methods I aim use are:
Student survey –To find student demographics and learner types.
Focus groups – Get students to trial a quiz and give their opinion. I will endeavor to test if learning outcomes can be met. I will give them a small assessment before the trial and another assessment after the trail, to try and measure effective change to learning outcomes. Both test with a range of outcome levels and should give some quantifiable results. I am hoping to have 10 to 15 students in my focus group.
A separate survey after they have completed the trial to see what their feelings were on using elearning as part of their learning. This survey would have open ended questions and the results would have to be considered with consultation of other instructors.
Expert review - Peers to trial a quiz and be interviewed on their opinion.

Summary - Needs assessment by Rossett

I picked this article on the basis that I intend to do a needs evaluation. I found Allison Rossett easy to read and the scenarios she used, I could relate to.

Needs Assessment By Allison Rossett - Summary

Rossett give a general overview of needs assessment with the emphasis on occupational instructional training. She give a number of example that are corporate based – one that runs through the whole discussion is of soft and limp French fries in a fast food joint.
She defines it as “needs assessments gather information to assist professionals in making data-driven and responsive recommendations about how to solve the problem or introduce the new technology. While needs assessment may lead to the development of instruction, it does not always do so. The important role that needs assessment plays is to give us information, at the beginning of the effort, about what is needed to improve performance. She defines five purposes for doing needs assessments

1. To define Optimal performance
2. To find out what is happening with actual performance
3. To know how learners feel about
4. To find the cause of performance problems
   a. Lack the skill or knowledge
   b. The environment is in the way
   c. There are no or improper incentives
   d. Lack of motivation
5. To find solutions
   

Rossett then talk about how to conduct a needs assessment giving 5 steps

1. Determine purpose based on Initiators
   a. Performance problems
   b. New stuff/technologies
   c. Mandates from above
2. Identify sources
3. Select tools
   a. Interviewing
   b. Observing performance
   c. Examine records and outcomes
   d. Facilitating groups
   e. Surveying through questionnaires
4. Conduct the needs assessment in stages
5. Use findings for decision making.
   

In her conclusion she remarks the
“Deciding to observe interview or survey isn't the hard part. The challenge is to determine sources, plan stages of assessment , and frame questions that will enable a professional to move crisply form a general problem with limp French fries to a series of actions and recommendations that will guarantee yummy ones. “