Final version -Proposed evaluation



Title:              Evaluation of the Introduction of e-learning into engineering mechanics



 



Introduction: - great introduction




This a front end or needs assessment evaluation for the introduction of e-learning into the mechanical engineering course at the Bahrain Polytechnic. We Staff in the ....School/Department aim to trial some interactive web based resources into the Engineering Mechanics paper, which is a level 5 paper, carrying 15 credits and will run in the second semester of the new academic year. The actual e-learning resource has not yet been developed but the Merlot website has an interactive resource along the lines that is intended to be introduced into the engineering mechanics paper.



In the words of Allison Rossett (Rossett, 1995) a needs assessment seeks to



    Gather information to assist professionals in making data driven and responsive recommendations about how to solve the problem or introduce the new technology.



This needs evaluation is to provide information to help make decisions about the introduction of new e-learning resources. 



This evaluation will include two questionnaires to students, one to ascertain their learning styles, and the second to find out how they feel about merlot resourceWe are using the merlot website to mimic our resources, from which information about student’s opinions and feelings can be gained with some relevance.  The evaluation will also include interviews with teaching staff and prospective employers to determine that the resource is in line with teaching requirements and relevant to industry.



  a sentence about what else is in the plan would be good at the end of this paragraph. -Bronwyn 6/3/09 3:24 PM 







Background: - excellent description of the reasons for your evaluation.




 



What is to be evaluated?

It is intended that web based e-learning be introduced into engineering learning resources.  A pilot resource is to be introduced into Engineering mechanics in the form of a interactive resource along the lines of a resources that has been developed and available in the public domain from the Merlot website namely MecMovies on this address: http://web.mst.edu/~mecmovie/ For the sake of this evaluation we will use the MecMovies resources to get some idea of how students react to the resource.  The actual resource will need to be developed to meet the requirements of our Mechanical engineering course.



 



Why



In the Mechanical engineering course that are currently being developed at Bahrain polytechnic, e-learning take the form of electronic notes and assignments made available to students on a web based filing system.  A more dynamic use of interactive e-learning in the form of quizzes, video recordings of lessons and possible group discussions is proposed



 



It will take some time to introduce e-learning into the engineering course, the intention is not to replace face to face learning but supplement it with additional resources i.e. to develop resources towards an appropriate blended delivery.  The introduction of e-learning is envisaged to increase as students and tutors become more aware and familiar with types and capabilities of tools available.



 



Bahrain Polytechnic is committed to providing a modern facility that is internationally recognized with student focused teaching methods.  E-learning offers another form of teaching which may appeal to students who require a more time flexible platform and those students who are at different skill levels.  With this in mind e-learning is to be introduced as an additional teaching resource.  David Haffaker (Huffaker (2003) concluded



Elearning application can be personalized to the learners needs and still provide communication that foster collaborative, pro-social work.  Since e-learning offers an anytime-anywhere transfer of information, it cultivates learning both inside and outside the classroom.” 



Which is supportive of the benefits of a flexible platform, While Mason and Rennie (Mason & Rennie, 2006) pointed out that



..blended approaches can encourage participants to make better use of face to face contact in the knowledge that preparations and follow up can be conducted online”. .



Which supports the use of blended learning. 



This needs assessment is being conducted to is to try and clarify issues that may arise for both students and teachers and to ensure that the resources used are appropriate for the learner in terms of current usage and for their future prospect employment.



 



Purposes: good



To investigate appropriate methods to introduce online interactive e-learning into a engineering mechanics course if it was offered by blended delivery? 



 



Audiences: 



 



The stake holders are



Students present and future



Engineering tutors at the polytechnic



Future employers of graduates



Degree Accreditations boards



 



Decisions: good



Estimation of how much time, effort and money should to be spent into establishing e-learning resources in the engineering course based on recommendations from the needs assessment.



 



Questions: great guidelines and questions



There are two guidelines taken from Massey university elg web site (Massey University, 2006) Guidelines for the support of e-learning in New Zealand tertiary institutions” as follows with sub questions for this evaluation.



 



TD3              How does the e-learning encourage a realistic progression towards self direction and recognises varied starting points in levels of confidence and motivation?



This guideline looks at student’s familiarity with technology and whether they are capable of using the e-learning without the need to up skill at the same time as they are trying to learn the course material. This is relevant to the introduction of e-learning as if the new resource is not pitched at the correct level, or with the support required the e-learning can be contrary to the learning objectives.



Sub Questions




  • Will students be suitably skilled to use the technology?


  • How do students feel about using web based interactive learning resources?


  • Will there be sufficient technology support?


  • Will tutors embrace the new technology and the use of e-learning for teaching?



 



SD3               Do students gain knowledge relevant to employment and/or current thinking in their field?



One of the mandates for the existence of Bahrain polytechnic is that their graduates have relevant skills for their chosen vocation.  There has been a common complaint by employers that graduating students are not work ready.  That may well be an unreasonable request by employers but Bahrain Polytechnic’s aim is that courses will give students skills that are aligned with industries requirements .



Sub Questions




  • How important is online learning for preparing graduates for e-engineering in their careers?


  • Is investment into e-learning resources of this type justified in term of relevance to the working environment?



 



 



Methods: this is really clearly explained.



 



How the evaluation will work.



This evaluation will use a 'multiple methods' model to collect data and information. The paradigm used is the Eclectic-Mixed Methods-Pragmatic Paradigm.  This methodology allows for a number of different ways to look at the same question.  As Professor Tom Reeves (Reeves) (2003) suggested that this paradigm is most useful  “Because it is the one approach most capable of handling the complexity that is the hall mark of contemporary society and technology”.  Instructional design is complex and does not lend itself to large volumes of data collection or being able to isolate any one variable.  This evaluation uses both surveys and interviews, with some quantitative data collection for students current learning styles and technology usage, and some open ended questions aimed to get students, tutors and industrial experts opinions and comments.



 



Types of data collection devices that will be used and the sample



Student learning styles surveyTo find student demographics and learner types.  Prospective students currently enrolled in the foundation engineering course are to be given a learner style’ questionnaire – this will help establish how many students will benefit from the use of e-learning resources.  The Learning styles questionnaire in on line http://www.engr.ncsu.edu/learningstyles/ilsweb.htm it is a 44-item questionnaire that can be submitted and automatically scored on the Web.



 



Student survey on a trial resource.  A number of students, volunteers from the group of currently enrolled engineering students, approximately 6-8, will trial the “mecMovies resource and fill out a survey as to their opinion and feelings about the use of the on line interactive learning resource.  This survey would have open ended questions encouraging students to give opinion and possible recommendations. 



Student interviews on following a videoed trial after being videoed.  Video record students while they do use the trial software, and then interview them using the recording to prompt them.  This tool  will not be used in this evaluation due to time and equipment constraints.



Expert review Conduct an Interview with engineering tutor/show many? -Bronwyn 6/3/09 4:04 PM . as to their view on e-learning resources. In respect of difficulties and advantages they see in the implementation of e-learning.



Conduct an interview of one relevant industry person (preferably a prospective employer) to see if the resource will compliment skills that are required in the work force, i.e. that the technology skills student acquire while using on line learning resources are relevant and of future use to students when they gain employment.




A matrix of the instrumentation used to gather data to answer the evaluation questions is shown in Table 1.


 

Table 1: insert caption -Bronwyn 6/3/09 4:07 PM 






































































Matrix Evaluation tools



Vs



Questions




Student learning style survey




Student technology skill survey




Students trial resource commentsno need to include this if it is part of the survey -Bronwyn 6/3/09 4:05 PM 




Peer expert interview




Industry expert interview




[1]Student interviews on the trial after being videoed




Will students be suitably skilled to use the technology?



 




X




X




 




X




 




X




How do students feel about using web based interactive learning resources.



 




X




X




X




 




 




X




Will there be sufficient technology support



 




 




 




 




X




 




 




Will tutors embrace the new technology and the use of e-learning for teaching




 




 




 




X




 




 




How important is online learning for preparing graduates for e-engineering in their careers?




 




 




 




X




X




 




Is investment into e-learning resources of this type justified in term of relevance to the working environment?




 




 




 




X




X




 




 



 



Instrumentation: 




  • The trial software to be used as part of the needs assessment is that on Merlot MecMovies”url address  http://web.mst.edu/~mecmovie/


  • Learner style questionnaire “Index of Learning Styles Questionnaire”  developed by Barbara A. Soloman and Richard M. Felder of North Carolina State University

    Raleigh, North Carolina url address http://www.engr.ncsu.edu/learningstyles/ilsweb.htm


  • Student technology usage and triail software survey  Attached appendix 1


  • A questionnaire for interviews of teachers and prospective employers Appendix 2



 



Limitations: 



The small numbers of staff and students involved gives a limited view for the results.



The actual e-learning resources which we will end up use ing in the course may vary from the one we trail.spelling -Bronwyn 6/3/09 4:11 PM 



 



Logistics: 



Learner style survey, trial software “MecMovies” and questionnaire to be conducted outside of class time.



Organize students trial – in their own time outside of class



Conduct a student review on the trail software (Questionnaire as per appendix 1)



Prepare an informal interview with peer tutor and prospective employer



 



Time Line: looks achievable all going to plan. :) 



Preparation of evaluation questionnaires  and trail assessments – Complete by               21 may



Carry out the trail with students in week                                                                                                  24 to 28 May



Interview with tutor and industry expert also in week                                                                       4 to 28 May



Collation of results                                                                                                                                            31may to 5June



Compilation of report                                                                                                                              26 June



 



Budget:   good budget



Staff time                                                                                                   48 hours



Preparation of proposed evaluation plan



Preparation of surveys and interview



Completion of surveys and interview



Collation of results



Compilation of report



Cost Staff time 48 hours at $65/hr                                                        cost               $ 3120



Incidentals                                                                                                                $   350



Total Cost                                                                                                                $ 3470



 



Pradeeps EVALUATION PLAN                                                                                     17 | Page              May 09



 



 





Appendix 1 - very comprehensive questions. :)



E-learning Participants Questionnaire



Thank you for taking the time to fill in this questionnaire. This questionnaire is being used to investigate the online technology you use and your experiences.



Responses to this questionnaire will be absolutely confidential and no other participants will be able to see your data. The answers to this questionnaire will go only to the project researcher (Pradeep Nathoo) and any information which could potentially identify participants will not be disclosed.



Please tick the relevant boxes and be as full and comprehensive as possible with your other answers.



Section A: Information about your technology use





Q1: Approximately, for how many years have you been using a computer for?



(Please enter a numerical value only)



Q2: I normally use a computer (please tick one)

















 



Every day



 









A few times a week









Occasionally









Rarely/never









 



Q3: I have access to broadband (please tick all that apply)













At home/student residence




At work




At University/College/Learning Centre




 






Other Location (please state)



 



Q4: I normally access the Internet to look for information(please tick one)

















 



Every day



 









A few times a week









Occasionally









Rarely/never









 







Q5: Approximately, how many hours a week do you spend at home or somewhere else on the Internet including using the internet(for work, recreational and educational purpose)?              (Please enter a numerical value only)


 
yes no does not seem to fit this question and some others further on -Bronwyn 6/3/09 4:14 PM 

















Yes



 




 






No



 




 




Q6: When I use a computer, I customise it to suit my personal preferences e.g. background colours, icon sizes, mouse buttons, menu items, size of print on screen:



If yes, please briefly list the changes you most often make:



 



 



 















 



Yes



 









 



No



 









 



Use of online tools



Q7: I use social networks (E.g. MySpace, Flickr, Facebook)



 



If yes:which do you use?



 



 



 



Q8  How often do you use synchronous (At the same time) chat tools? (please tick one)

















 



Every day



 









A few times a week









Occasionally









Rarely/never





















 



Yes









 



No









 



Q9: I use messaging and discussion tools (E.g. Email, forums, phone texting)



If yes which do you use?





 



How often do you use messaging and discussion tools? (please tick one)

















 



Every day



 









A few times a week









Occasionally









Rarely/never









 















 



Yes



 









 



No



 









Q10: I play online games or use virtual worlds and talk to other players (E.g. World of War Craft, Battlefront 2, Sims Online, Second Life)



b) Which do you use?



 



 



How often do you play online games or use virtual worlds (please tick one)

















 



Every day



 









A few times a week









Occasionally









Rarely/never









 















 



Yes



 









 



No



 









Q11: Do you have a personal website/blog?



If yes: which do you use?





 



How often do you use your online personal space? (please tick one)

















 



Every day



 









A few times a week









Occasionally









Rarely/never





















 



Yes



 









 



No



 









 



Q12: I use other social networking and communication tools online (E.g. Online dating, Friends Reunited, ebay)



If yes:which do you use?



 



 



   c) How often do you use your other social and communication tools online? (please tick one)

















 



Every day



 









A few times a week









Occasionally









Rarely/never









 



Q13. Overall, how confident are you in the use of computer-based and Internet-based communication and information methods?.



 

















Extremely unconfident









unconfident 









confident









extremely confident













Section B  Survey on Trial interactive learning resource



With regards to the MecMovies interactive learning resource that you trialled in class can you answer the following questions.  Please be honest and as critical as you wish,  The results are fully confidential. Your comments are most valuable



1              The instructions for using the resource were easy to follow


















 



Strongly agree



 









Neutral









Strongly disagree









Comment



 



 



 



 



 




 



2              To navigate around the resource was easy


















 



Strongly agree



 









Neutral









Strongly disagree









Comment



 



 



 



 



 




 



3              the resource was a helpful way to learn.


















 



Strongly agree



 









Neutral









Strongly disagree









Comment



 



 



 



 



 




 



4              I feel that I learnt something from the resource


















 



Strongly agree



 









Neutral









Strongly disagree









Comment



 



 



 



 



 




 



5              I would like to use more of this type of learning resource as part of my course


















 



Strongly agree



 









Neutral









Strongly disagree









Comment



 



 



 



 



 




 



6. what sort of support do you believe you will need if you are asked to use interactive learning resources (such as the one you trialed) regularly in your course?


















 




 




 




 




 




 




Comment



 



 



 



 



 




 



 



 



7. what type of skills do you believe you need for eLearning (online- Internet, multimedia, computers)


















 




 




 




 




 




 




Comment



 



 



 



 



 




 







Section C: Personal Details



Name:





Age:  (Please enter a numerical value only)





Gender:               Male                                           Female



 



 



 



Which program do you intend to study



Engineering Electrical               Engineering mechanical



 



 



 



 



Employment/education Status:  



In full time education                            In part time education                                Not in education



 



 



 







Appendix 2                            Interview of tutors and industry expert



Interview questions



This interview is for the purpose of investigating the best method to introduce elearning into a mechanical engineering course that is to have a blended delivery.  You opinion is confidential and valued



For the industry expert



 



Skill levels  For the following table, place a tick in the skill level you feel that an engineer must have in today’s modern engineering concern. for each of the items listed



 











































































































 




 




Very skilled




Skilled




Unskilled




Very unskilled




Do not use




Software applications




Word processing




 




 




 




 




 




Excel spreadsheet




 




 




 




 




 




Presentation software (power Point




 




 




 




 




 




Creating and editing video/audio




 




 




 




 




 




CAD CAM




 




 




 




 




 




ICT




Email




 




 




 




 




 




Web Forums




 




 




 




 




 




Professional social networking




 




 




 




 




 




Wikis blogs




 




 




 




 




 




Digital information




Information retrieval -= searching on the internet




 




 




 




 




 




Ability to use on ine library resources




 




 




 




 




 




 



Do you feel it is necessary for engineers to be able to be able to program in a software language?add ? to all these questions. -Bronwyn 6/3/09 4:16 PM 



 



Do graduate mechanical engineers use web based software



 



What forms of software are used in industry.  Ie CAD CAM Mat lab etc



 



Can these skills be developed while working – is this preferable?



 



How much web based / internet usage would an engineer expect?



 



 



Questions for tutors



Do you feel you have the skills, time to use e-learning software



What is your opinion on the introduction of elearning into engineering programs



How much e-learning should there be  %?



What type?



How should it be used



Do you feel that students will enjoy using elearning



Will it require students to up skill?



Will it require teacher up skilling



What drawback or difficulties do you foresee



What advantages are there



Are you or other staff members able to develop  and modify electronic interactive learning software for appropriate use and level in the classroom



Do you feel that there is sufficient technology support to be able to deliver e-elearning in a blended course?



 



 



Bibliography - good list and using APA referencing - best to put this before the Appendix



Huffaker, D. (2003). reconnecting theclassroom:Elearning pedagogy in US public high schools. Australian Journal of education Technology , 19 (3), 356-370.



Mason, R., & Rennie, F. (2006). Elearning the key concepts.



Massey University. (2006, june). Guidelines for the support of e-learning in New Zealand tertiary institutions. Retrieved may 7, 2008, from elg: http://elg.massy.ac.nz



Rossett, A. (1995). Needs Assessment. In G. Anglin (Ed), Instructional technology: Past present and future (2nd Ed.) (pp 183-196) Englewood CO: Libraries unlimited.. Englewood Co libraries unlimited.



Tom, Reeves, T. (n.d.). Retrieved feb 5, 2006, from http:/www.educationau.edu.au/archives/Cp/refs/reeves_paradigms.htm



 



 



 



 



 










[1] This tool is not to be used in this evaluation