Pixel International Conferences

The purpose of this study is to examine the usability of simulations in an educational portal that is supported by Ministry of National Education and is extensively used by many teachers and students in Turkey. This study also aims to determine sufficiency in evaluating simulations’ usability. Candidates of educational technologist as CEIT (Computer Education and Instructional Design) students taken CBI (Computer Based Instruction) courses in their bachelor are expected to evaluate training materials professionally.

Methodology of this research is designed as a case study in three phases: evaluating simulations according to usability criteria, determining impediments when using simulations and determining sufficiency of educational technologists in the field of evaluating simulations. In the first phase, 14 criteria have been generated by literature review to determine simulation evaluating criteria and then interactive practices of the portal have been analyzed to remark simulations. In the second phase, 28 simulations have been assigned to 53 participants and each of these simulations has evaluated at least two participants. Determining evaluation consistency of education technologist candidates is tested using assessment results. In the last phase, all of the simulations have been used by 27 participants. The participants have taken screenshot when they have faced a problem in simulation, and prepared reports for researchers after discussing among them.

Content analysis has been applied to these reports to detect the problems encountered by participants in using simulations. In order to determine internal consistency of assessments of participants, Fleiss Kappa parameter has been used by using above mentioned criteria  to evaluate over a total of 28 points: yes (2), partially (1), no(0).

According to content analysis, participants have indicated several impediments at the reports: friendly user interface, compatibility with internet browsers, taking note and manipulating facilities. Participants have suggested that simulations are to be improved using plugins to make the offered information by simulation permanent. Inadequacy in guiding and presentation of the instructions and uncertainties in the end of the simulation cause deficiencies in offered instructions. According to another finding, internal consistency in educational technologist candidates’ evaluation has been determined lower than expected (K=0,21).

The findings in this study show that simulations should be improved in terms of usability. On the other hand, curriculum of programs to educate education technologists should be updated from the point of software evaluation.

Keywords: sufficiency of simulation evaluation, usability, education technologist candidate