f Competencias TIC del profesorado universitario
 

METHODOLOGY

 

MENU

 

Based on the objectives set out in this proposal, we consider that our work has two fundamental parts: a first part in which will be carried out an exploration and analysis of the indicators currently used, at a national as well as international level, for measuring the ICT competences of higher education teachers, and a second part in which will be drawn up a proposal of improvement (based on these indicators) of the ICT competences of higher education teachers and a telematic instrument for self-evaluation.

Consequently, we have set the work around two general phases, broken down in a series of tasks more specific:

First phase: Indicators of evaluation of ICT competences in higher education teachers.

  1. Exploration of indicators (nacional and internacional) used currently for the measurement of ICT competences in higher education teachers
  2. Analysis and discrimination of the indicators by experts: clasification depending on objetictivy and subjectivity criteria, and also on its impact and relevance in the global analysis (weighting)
  3. Desing, validation and application of an evaluation instrument based on the objective indicators and a proposal of integration of subjective indicators in this instrument
  4. Second phase: Proposal of impovement of the quality of teaching.

  5. Questionnaire online for the evaluation of ICT competences of Spanish higher education teachers. It will be applied to all participants universities.
  6. Improvement proposal of the quality of higher education teaching about ICT competences and a catalogue of indicators of good educational practices in ICT use.
  7. 6. Desing and implementation of a self-evaluation instrument online for higher education teachers interested in measuring their levels in ICT competences

The tasks here represented will be carried out consecutively, just as is indicated in the next work schelude:


All the process of work is made explicit below:

First phase: Indicators of evaluation of ICT competences of higher education teachers.
Indicadores de evaluación de competencias TIC del docente universitario.


The first phase of our project will be focused on the exploration of the indicators used currently for the measuring of the ICT competences of higher education teachers, at a national, European and international level. After this first exploration done using primary sources (documentation and information of the nacional agencies and universities) and secondary sources (scientific publication related), will be carried out a process of differentiation between objective and subjective indicators. This phase will completed with the development of an instrument of evaluation based on the objective indicators and with the proposal of integration of subjective indicators in this instrument. Next we are going to explain in detail the process of work that will be carried out in each one of the tasks of this first phase.

Task 1: Exploration of the national and international indicators used currently for the measuring of ICT competences in Spanish higher education teachers.

Inside this task we can highlight two activities clearly differentiated

On the one hand will be carried out the exploration of the national indicators and, on the other hand and through different techniques, iwill be carried out the exploration of international indicators. Both tasks will be done at the same time.

  • Activity 1.1 : Exploration of the indicators of quality in ICT competences for the teaching at a national level.

The exploration of national indicators will be done resorting to the primary information sources (real testimonials and first hand data), compiling all the existing information in each public Spanish university and in the quality agencies of each autonomous community ( hereafter CCAA), in case of existing

For this task, all the researchers who participate in the project will be involved , among them will be distributed the universities and quality agencies of which obtain indicators about ICT competences.

The list of universities to explore will be obtained of the Web of the Ministry of Sciencie and Innovation (Ministerio de Ciencia e Innovación). Therefore, the sample with which we count will be the total of the public Spanish universities, which amounts to a total of 49 ( according to the data of 6th of March of 2009, obtained at the official Web of the MICIN > http://ciencia.micinn.fecyt.es/univ/jsp/plantilla.jsp?area=ccuniv).For its part, the information relating to the quality agencies of each one of the communities is obtained of the official Web of the ANECA(http://www.aneca.es/servicios/link_ccaa.asp), where currently appear a total of 11 agencies, belonging to these CCAA: Andalucía, Aragón, Islas Baleares, Islas Canarias, Castilla la Mancha, Castilla-León, Cataluña, Galicia, Madrid, País Vasco and Valencia.

The development of the definitive list of the universities and quality agencies which will take part in the sample will be carried out from Murcia (Mª del Mar Sánchez Vera). This task will be done during the first week of the month of May.

Once the sample is chosen, for the distribution of the same will be used the criteria of proximity of the researchers with the corresponding university and quality agency, because in this way, the researchers will be more connected with the places in where they have to look for information and the task, therefore, will be more agil in its execution (reason by which it has been tried to count with a number enough representative of researchers situated in differents points of the Spanish geography)

Each one of the researchers must conclude their exploration with the elaboration of a list in which it will be included all the indicators of quality used for measuring ICT competences which they have been, and the list has to specify the source from which the information comes, so in case it is neccesary, it could resort to the archive of the project. This list must be sent to a researcher of the University of Murcia (Isabel Gutiérrez Porlán), who will take care of unify in one common list all indicators. This task will be done during the month of May and the three first weeks of the month of June. The common list of indicators will be done in the last week of June.

  • Activity 1.2: Exploration of indicators of quality and ICT competentes for Higher Level Teaching at internacional level.

The exploration of international indicators will be carried out by the research group situated in Murcia. For this will be resorted to an exhaustive exploration of secondary sources ( scientific publications and technical reports of research and evaluation of teaching competences) at European and international level.

Specifically in this task are going to participate: Linda Castañeda Quintero, Isabel Mª Solano Fernández, Mª del Mar Sánchez Vera, Patricia López Vicent and Isabel Gutiérrez Porlán. The task will conclude with the elaboration of a list of indicators (which has to include also the sources from where the indicators come), and it will be included in the list of national indicators. This task will be done simultaneosly with the previous tasks (May and June)

Task 2: Discrimination by experts of the indicators (objective and subjective) for the analysis of ICT competences of higher education teachers.

Once the complete list of indicators is done, the task 2 will consist in the diferentiation between them, according to its objective or subjective character.

From the research group situated in the University of Murcia, will be carried out the first discrimination of indicators through the analisys of content to distinguish between those objective indicators (measuribles and quantifiables) and subjective. In this first diferenciation will take part the researchers Linda Castañeda Quintero, Isabel Mª Solano Fernández, Mª del Mar Sánchez Vera, Patricia López Vicent and Isabel Gutiérrez Porlán. Each one of them will elaborate individually two lists of indicators (objective and subjective), giving in the neccesary cases the appropiate observations. Once the individual lists are elaborated, will be hold a meeting, coordinated by Mª Paz Prendes Espinosa, in which will put in common the work of each researcher, looking for a consensus in the definitive list, taking into acount the objective indicators as well as the subjective ones.

This task will be carried out during the firsts two weeks of the month of July.

Once the lists are elaborated (objective indicators and subjective indicators), will proceed to the definitive differenciation of indicators by experts through the technique known as “Discussion Group”, which will be carried out in an on-site way, and in which will participate just a part of the researchers of the project. In this same process, experts will be asked about the weighting of the indicators, which allows us to judge the relative weight that every indicator will have in the final evaluation of the ICT competence of teachers.

So, three groups of discussion will be constituted and will meet simultaneosly in three differents points of Spain, to validate the lists of indicators. The groups of discussion will be made up of members of the research group of the project, who fulfill also the characteristic of being well-known professionals in the fields of ICT and education.

Through the group of discussion, the experts will meet for debating and talking about the proposed list of indicators, coordinated by a moderator of the University of Murcia. The three groups will count, as work document, with the list of indicators elaborated from Murcia and they will work on the validation and improvement of it. The porpuse of working with three different groups simultaneosly is to allow us triangulate the results of each one of the work documents. The meetings will be held the last week of the month of July. To facilitate the displacement, these groups of experts will meet according the geographic proximity of the CCAA, like this:

Grupo A: Castilla León, Extremadura y País Vasco.

Grupo B: Cataluña, Islas Baleares, Comunidad Valenciana y Murcia.

Grupo C: Andalucía, Madrid, y Canarias.

Each meeting will conclude with the elaboration of a new work document which will be collected by the coordinator of each group. Once in Murcia with the three reports, these will be triangulated with the data, and will be carried out an analysis of its content, so it will be done the final work document. This task will be carried out during de second week of September.

Task 3: Design, validation and application of an instrument of evaluation based on the objective indicators and the proposal of integration of the subjective indicators in this instrument

Inside this section, we can highlith two activities clearly different: on one side the design, validation and application of an instrument to collect the information, based on the objective indicators, and on the other side, the proposal of integration of the subjective indicators in an evaluation instrument.

  • Activity 3.1: Design, validation and application of an instrument to collect the information.

DESIGN:

Based on the list of objective indicators will be carried out the design and production of an instrument of evaluation. The main objective of this instrument will be to know the ICT competences present in the teachers of higher education. The instrument to collect the information will be a questionnaire online, so it can facilitate and make more dynamic its sending and reception.

The questionnaire will be elaborated from the group of research of Murcia, specifically the responsable for elaborating it and coordinating the validation process and its application, will be Linda Castañeda Quintero.

VALIDATION:

Once the questionnaire is done, it will be validated through the tecnique known as “Panel of Experts”. In this panel of experts will participate different members of the group of research proposed for this project, situated in different Spanish universities. The validation of the questionnaire will be carried out through work online, so it can garantee the presence of the higher number of experts and more contributions and suggestions of improvement will be collected.

This methodology technique can be defined as a group of specialists who are independiente from each other and renowned at least in one of the fields referred, which gathers to emit a colective judgement in consensus about a particular one.

We consider that, following recomendations of the European Commission, referred to the research methodologies (CE, s/f), this technique is very appropiate for the validation among other reasons because:

  • The topic is well defined.
  • It requires the opinion of experts with high level in the field, when also, we count with a research group with well-known experts in these topics
  • The extent of the topic to analyze doesn´t justify the use of resources of higher magnitud.
  • Also it will suposed a considerable saving of time related to other evaluation tools, like the pilot study, and a limited cost in relation with other tools, thanks to this time saving.
  • Given the reputation of the experts, the conclusions will have a high level of credibility and validity.

The questionnaire will be sent by email to the experts, who will send out their suggestions in relation to it, through a closed instrument for valoration wich measures the pertinence and content of the items, understanding that the indicators included are already validated in task 1.

The questionnaire will be elaborated with an online tool to create form called “Formlogix”, wich has been already used in previous researching projects, and because its good results obtained with it, we consider that is appropiate for this project.

The process of design, validation and elaboration on the Net of the questionnaire will be done during the last week of September and the two first weeks of the month of October.

APPLICATION:

After the elaboration of the questionnaire, it starts the phase of application of it. In this phase will take part all the researchers who form part of the project, who will be the responsables of the administration of the questionnaire in their respective universities, being also the responsable of arrange the difussion procedings which will be telematic proceedings.

Each one of the researchers will send to the universities the link to the questionnaire in order to spread the questionnaire to higher number of teachers and degrees.

The process of application of the questionnaires will be carried out in the second two weeks of October, November and December. The months of November and December will be destined to the application of the questionnaire.

The questionnaire answered for the teachers will be compiled in the University of Murcia. The process of reception and organization of the questionnaire will be carried out by a grant holder hired for this purpose, during the first two weeks of January.

  • Activity 3.2: Proposal of integration of subjective indicators.

Starting from the list of subjective indicators which is elaborated in the task 2, in this phase will be done a proposal of integration of these indicators in a evaluation instrument. Will be elaborated a first proposal of integration done by the group of research situated in Murcia, which will be completed with others universities. This proposal will be elaborated in consensus and reformulated later through a strategy of group of experts on the Net (the same as in the validation of the previous questionnaire). The first proposal of integration will be carried simultaneosly to the design and validation of the questionnaire, in other words, during the first two weeks of October. The validation process from the experts will be done during the last two weeks of January.

The experts will receive a first proposal of integration of subjective indicators and they will give their suggestions and proposals of improvement. The questionnaire will be received again for the group of research situated in the University of Murcia. The responsable of this task will be Patricia López Vicente. The proposal of integration will be revised, according with the comments of the experts, and will be sent later a new version of the document to the experts. They will revise the new document, giving again their valorations.

Second Phase:
Proposal of improvement of the quality of the teaching.


In the second phase of the work will be carried out an analisys of the results obtained in the questionnaire of the previous phase, so ICT competences of the higher education teachers of the Spanish universities will be described.This last phase will conclude with the elaboration of a proposal of improvement of the quality in the higher education teaching, regarding the ICT competences, and with the design and implementation of an instrument of self-evaluation online, which measures the ICT competences of the Spanish teachers of higher education interested in knowing them.

This phase is developed in three principals tasks:

Task 4: Analysis of the results. First approximation to the current state of the ICT competences in the Spanish higher education teachers.

Once all the questionnaire are compiled, will proceed the data analysis. This analysis will be done by the grant holder hired for the project. The estimate time for the data analisys will be one month and a half (second week of January and February completely)

TTaking into account that the objective of this task is to elaborate a first approximation to the current state of the ICT competences of Spanish higher education teachers, the data analysis will be descriptive and will be focused on the use of the statistics as the average, the median and the mode.

This task will concluye with the writing of the full description of the ICT competences present in the higher education teachers who work in the Spanish public Universities. The description of the competences will be done during the first two weeks of March and will be carried out by the grant holder hired for the project, together with a member of the research group of Murcia (Trinidad Rodríguez Cifuentes).

Task 5: Proposal of improvement of the quality of higher education teaching regarding the ICT competences and a catalogue of indicators of “good educational practices in the ICT use”

Starting from the results obtained in the previous task, and taking the description of the ICT competences of Spanish higher education teacher as a base, in this last task will be carried out the proposal of impovement of the quality of the teaching.

This task will be developed by the research group located in the University of Murcia and will coordinated by the director of the project Mª Paz Prendes Espinosa and by the teacher Francisco Martínez Sánchez, who through their wide experience in the ICT use and with the results obtained in the previous phase, will elaborate a catalogue of indicators of good educational practises in the use of Information and Comunication Technologies. This task will be carried out during the last two weeks of March and during the month of April. For this task is expected the realization of several meeting-seminars among the researchers

Task 6: Design and implementation of an instrument of self-evaluation online for the higher education teachers interested in measuring their level in ICT competences.

Taking as a base the list of objective indicators of ICT competences (which is done in task 2 of the phase 1), from the researching group located in Murcia will be designed and validated an instrument of self-evaluation, a questionnaire, which will be online and open for everyone interested. This task will be done during the months of February and April.

With this questionnaire will try that, everytime someone answers it, in an automatically way, it generates a feedback about the strongest and weakest aspects related to the ICT competences of the individual and also, it works as a platform for collecting the data about the state and the worry of the teachers.

Once the indicators and criteria of this instrument of self-evaluation are elaborated, a grant holder will be put in a charge of uploading it online, automatizing the answers and organizing the data base, in order to promote its utility to obtain statistics of its use and results, keeping the anonymity of those who answer it. This questionnaire of self-evaluation of ICT competences will be active and accesible once the project is concluded.

In this way, we expect that every teacher interested in knowing her/his level in ICT competences can carry out a self-evaluation each time he/she considers that her/his level of ICT competences has increased. The instrument of self-evaluation, apart from give information about the teachers who answer it, will include a registration of the data from which we can obtain information periodically.

We consider that this instrument of self-evaluation will motivate to the teachers in their formation and permanent updating, because they will know in detail their level in ICT competences, affecting in the improvement of the quality in teaching.

Moreover the aforementioned, with this instrument we will see reflected in the practice the results of our researching.

 

RESPONSIBLES