Modelación con teoría de grafos para la unidimensionalidad de un instrumento de evaluación
Documentos PDF
Abstract
En el marco de las políticas públicas en Colombia
sobre calidad de la educación superior se
encuentran los lineamientos para que el Estado reconozca
los programas de pregrado como de alta
calidad, entre tales lineamientos se consideran los
sistemas de evaluación de los estudiantes basados
en políticas institucionales claras y transparentes
que propicien una identificación integral de las
condiciones académicas y actitudinales de los estudiantes
por lo que es imprescindible tecnificar y
modernizar los procesos de evaluación para fortalecer
la emisión de inferencias confiables sobre
el logro académico. En este artículo se describe
una aplicación de la Teoría de Grafos usando algunas
de sus definiciones y teoremas básicos con
la intención de potenciar su uso para el estudio del
supuesto de unidimensionalidad de la Teoría de
Respuesta al Ítem como alternativa para robustecer
la evaluación. La unidimensional fue entendida
en el sentido de la homogeneidad escalar y de criterio
para las relaciones de orden definidas por la
dominación tanto para ítems como para evaluados:
para los ítems conforme al número de evaluados
que los contestaron correctamente y a la dificultad;
para los evaluados respecto al número de respuestas
correctas y a la habilidad. Se aplicó la
combinación de las dos teorías a un caso de estudio
que contempla el instrumento de la prueba
diagnóstica del Examen de Clasificación de Matemáticas
Básicas en la Universidad Jorge Tadeo
Lozano, con esto se ofrece un valor agregado porque
se evidencian los resultados de forma algorítmica
y visual con ayuda de herramientas informáticas
especializadas.
Summary in foreign language
In Colombia, public policies about quality of
higher education involve student assessment
systems as a relevant characteristic. These systems
should have clear and transparent institutional
policies requiring comprehensive identification of
attitudinal and academic conditions about
examinee performance. Consequently, on the way
of continuous improvement of academic services
offered by institutions of higher education, an
assessment process adequately technified allowing
valid inferences about academic achievement is
required. In this sense, the Rasch measurement
model of Item Response Theory is a modern
alternative to strengthen educational measurement
estimating the ability of the student and the
difficulty of the item on a comparable scale.
Unidimensionality, local independence and internal
consistency are assumptions made in Rasch
measurement model. Unidimensionality assumption
has several definitions, one of them is the
occurrence of a dominant factor influencing test
performance. Another definition is considered in
the present study. Here, unidimensionality was interpreted as scalar and criterial homogeneity for
the order relations defined by domination for items
and assessed applicants. Graph Theory is an ideal
mathematical modelling approach to this
assumption inasmuch as represent intangible
interactions as required. In order to achieve this,
Graph Theory and Item Response Theory were
combined to examine the qualifying test for the
Basic Mathematics subject at Jorge Tadeo Lozano
University as a case study. This test was composed
by 45 items divided in three blocks. With several
combinations of these blocks, three virtual booklets
were obtained each one comprising 30 of them.
From the application of June 2011, a test sample of
509 responses chains was obtained. Three data
bases were processed one for each booklet, the first
spanned 175 responses chains, the second 170 and
the third one 164. To begin with the exmination on
the Rasch measurement model, the parameters
reliability, separation, Cronbach Alpha and item
residual correlation were estimated to gauge and
determine performance test for getting acceptable
values in each booklet applied. Tatsuoka was
followed to undertake the path to unidimensionality
from Graph Theory gathering a real case experience
processed with suitable software. Additionally, the
sensitivity of the order relation was verified
through: 1) ordering by number of correct
responses per item (1I order) and items difficulty
(2I order); 2) ordering by number of correct
responses per applicant (1E order) and estimated
ability (2E order). Furthermore, the linear models
were obtained collating these orders. In like
manner, three Guttman scales and their adjacency
matrices were schematized one for each booklet.
Subsequently, the respective graphs were processed
and represented using Gephi just as a specialized
tool that enables running some algorithms like
Force Atlas. Afterwards, the second power of each
adjacency matrix was found using Matlab 2014b
and domination matrices were calculated for both
items and applicants in the aforementioned orders
for a total of 12 matrices. Consistency index
developed by Cliff was computed for the
domination matrices. As a result, moderate
consistency was observed. Significant domination
for the entries of these matrices was analyzed
through McNemar test in order to have an
asymmetric dominance relation. Moreover, a
reachable matrix was calculated for each one of
these significant domination matrices as a limit of
a sequence of boolean powers. Finally, dominance
hierarchies were illustrated with vertex degrees and
compared with student maps by the means of
Winsteps 3.73. The combination of Graph Theory
and Item Response Theory allowed a deeper
comprehension of unidimensionality assumption.
Thereupon, universities can optimize their
resources offering to applicant differential
academic options per individual position in the
ability scale. The results can be used to outline
advantages for the applicants who can evidence
their position in the ability scale and identify the
different areas to improve.
Palabras clave
Evaluación; Homogeneidad; Modelo de Rasch; Teoría de Respuesta al Ítem; Teoría de Grafos; Teoría de Respuesta al Ítem; UnidimensionalidadLink to resource
http://www.ciipme-conicet.gov.ar/ojs/index.php?journal=interdisciplinaria&page=article&op=view&path%5B%5D=452&path%5B%5D=63Collections
- Año 2018 [155]
Estadísticas Google Analytics
Comments
Respuesta Comentario Repositorio Expeditio
Gracias por tomarse el tiempo para darnos su opinión.