2 Developing A Validated Instrument
2 Developing A Validated Instrument
Introduction
The teacher is an essential element of all educational systems. He has to
perform numerous tasks such as a researcher, curriculum developer, team member or
leader, professional and being an analyst (Richards and Lockhart, 1996). Teachers are
*
Dean Faculty of Education/ Chairman, Department of Education, The Islamia University of
Bahawalpur, Pakistan
**
Assistant Professor, Department of Educational Training, The Islamia University of
Bahawalpur, R.Y.Khan Campus, Pakistan. Tel: +92-332-9601095. E-mail:
[email protected]
21
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
22
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
the organization (Yusoff, Ali, & Khan, 2014). This is also considered as a total outcome
that an individual renders and that is recognized by the institution.
The excellence and performance of a teacher have been a focused apprehension
in education. The efficient job performance of a teacher is indispensable for improving
an educational system as a whole (Yusoff, Khan, & Azam, 2013). Teacher’s
performance is defined as the aptitude to fulfill the requirements and demands of the
professional development process to the required level by a homogeneous set of
knowledge, perspectives, behavior, and skills in a way to accurately display the things
(Ghasemi & Keshavarzi, 2014). In view of Werang and Lena (2014) a teacher job
performance is the quality and skills of a teacher to put together all necessary and
related variables for the increment and improvement of the educational process.
Selamat, Samsu, and Kamalu (2013) believed that teacher’s job performance is a
teacher’s method and strategies of teaching and it is associated with teacher’s
efficiency. It is a process of determining teacher’s engagement in daily routine activities
to run the school affairs properly. This is also judged as teacher’s noticeable attitudes
associated with results which are related to instructive objectives.
After determining the importance of teacher’s job performance, the second
important issue emerged is related to its measurement. A review of teacher’s work
performance literature reveals that there are some scales available for the measurement
of work performance (e.g., Stanford Center for Assessment, Learning and Equity, 2013;
Ross, Singer-Dudek, & Greer, 2005; Rebore, 1985). But, they have two clear
limitations, first, these scales are developed for West and second, most of the scales
measure employees’ performance within organizations. They are not specifically
designed for measuring teacher’s job performance. Despite a growing body of research
into teacher’s performance, there is a lack of appropriate scales for evaluating teacher’s
performance based on head’s ratings in Pakistan, which raises the significance of this
research. Considering its value, the main objective of this research is to address the
above issue by developing and validating a scale for measuring teacher’s performance
based on their heads’ ratings.
Study 1: Examining the Dimensionality of the Proposed Instrument
This study was carried out to develop and validate Teacher’s Job Performance
Scale. The development of this scale was comprised of following steps.
Step 1: Developing Items Pool
The first step of developing TJPS was the items' generation. For this purpose,
20 M.Phil students (15 male, 5 female), 20 Ph.D. students (10 male, 10 female), 10
secondary school teachers (5 male, 5 female) and 10 schools heads (5 male, 5 female)
of different subjects were requested to fill in an open-ended questionnaire to find out
their perspectives regarding the essential aspects of job performance of teachers. They
were requested to mention all those qualities, skills, behaviors and characteristics,
23
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
which they think, are crucial for teachers’ good job performance. The responses were
carefully analyzed and on the basis of them, statements were developed and arranged in
the frequency distribution. The statements with the highest frequency were retained to
make a pool of 40 items (statements). These statements were displaying different
dimensions of teachers’ job performance. Moreover, with the help of literature review,
these statements were carefully examined and scrutinized by the researcher.
Step 2: Assessing Items Suitability
In the next step, experts were asked to examine items generated in the form of
statements. Primarily, these 40 statements were given to 10 Ph.D. researchers and 03
university lecturers and they were asked to develop the categories of these statements
for teachers’ job performance. The purpose was to scrutinize items and clearly
designate items into different dimensions to check the inter-rater-reliability. On the
basis of researchers’ opinions, three facets of teachers’ job performance were developed
e.g., Instructional Qualities, Professional Qualities, and Personal Qualities. After
making these dimensions, the 40 statements were given to the 05 lecturers, 05 assistant
professors and 10 Ph.D. researchers, and they were requested to place each statement
into their respective category. The selection criterion for the items of different
dimensions was 70% consensus among the experts. The repeated statements and the
items which were not clearly related to the job performance were rejected. Therefore,
only 24 statements out of total 40 could be clearly designated into three categories of
teacher’s job performance. These statements were written with four-point Likert scale
and the scores assigned to this scale were ranging from 1 to 4. Finally, an initial form of
the scale for measuring job performance of teachers by their respective heads was
developed and was put in next step for EFA.
Step3: Pretesting
In this step, after finalization of the items, EFA was run to study the scale
structure.
Participants and Method
To further validate the TJPS and to provide an estimation of its reliability, 24-
items were given to head teachers (only the head teachers of public high and higher
secondary schools were selected because of their better understanding to evaluate their
teachers and having different management related experience and certifications). The
population of the study was head teachers of public high and higher secondary schools
situated in district Bahawalpur. Total 54 head teachers of public high and higher
secondary schools were randomly selected and requested to rate the performance of
their randomly selected 432 secondary school teachers (08 teachers per school (04
science and 04 arts). Simple random sampling technique is always preferred due to
enhanced generalizability with a wider involvement of participants (Haider & Qureshi,
2016). Of the head teachers, 27 (50%) were male and 27 (50%) were female.
24
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
Approxmately 35 (65%) heads hold master degree and 19 (35%) M.Phil. All of the head
teachers have M.Ed degree as profession qualification. The head teachers age range
from 36 – 55 years (M = 46.87, SD = 8.75) and experience range from 12 – 26 years (M
= 18.10, SD = 8.92). However, the completed questionnaire of 376 secondary school
teachers (188 male and 188 female) with age 19-48 years (M = 32.43, SD = 8.29) and
experience 1-27 years (M = 13.29, SD = 8.15) from 54 public high and higher
secondary schools were returned by their heads with a response rate of 87.03%. Data
were approximately 5 times greater than the total number of statements (Field, 2013).
We personally visited the selected schools and requested the headmaster/headmistress
to rate their teachers’ performance. In some schools, it was noted that some school
heads were hesitant in providing the true information. However, we convinced them,
informed them the rationale of the study, assured respondents the confidentiality, and
requested to complete the scale.
Exploratory Factor and Reliability Analysis
EFA with PCM followed by varimax rotation was run on 24-statements to
extract the uncorrelated items of the questionnaire. The correlation analysis revealed
that all statements correlate with at least the other variable at .30 which suggests
suitable factorability. All 24-items in the scale associated fairly and none of the
correlation coefficients were large so, there was no reason to remove any statement.
Moreover, the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy revealed that
it is very suitable for this data. The value of KMO = .826 is higher than the suggested
value of .5 (Kaiser, 1974). Furthermore, Bartlett’s Test of Sphericity was also
significant, χ² (66) = 6082.40, p < .000, demonstrating that factor analysis is a suitable
method (Bartlett, 1954). Normal distribution of all items also suggested that there is no
need to delete any item due to a high skewness (>2) or kurtosis (>7) (Finch &West,
1997). Considering this overall criteria, EFA was considered appropriate with 24-items.
To reach a meaningful factor structure, 1, 2, 3, and 4-factor solution was
examined. Finally, the whole procedure yielded 24-items divided into 3-dimensions
with each factor has more than one eigen-value and together explained 64.05% of the
common variance in the measured construct (Table 1). The final factors were comprised
of those items with factor loadings equal to or greater than 0.4 on a specific dimension,
cross-loadings not exceeding .3 and not loaded on two or more than two factors
simultaneously. These findings, therefore, present primary help for the strength of
proposed instrument (i.e., TJPS). The minimum score achieved could be 24 and
maximum 96, whereas, high score demonstrates better job performance.
25
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
Table 1
EFA Factors Loadings based on Principal Components Method
Q# Items F1 F2 F3
Q5 Teacher properly prepares and delivers his/her lectures .93
Q14 Teacher uses daily life examples to clarify concepts .91
Q1 Teacher encourages students to participate in co- .83
curricular and extracurricular activities
Q9 Teacher uses different teaching methods in classroom .79
Q19 Teacher appreciates students’ questioning and .71
classroom discussion
Q20 Teacher uses variety of teaching materials (AV aids) in .65
classroom
Q13 Teacher provides a favorable learning environment to .52
the students
Q7 Teacher constantly evaluates students’ learning .40
Q24 Teacher maintains a respectable relationship with .89
students
Q16 Teacher actively participates in school activities .81
Q15 Teacher maintains strict discipline in classroom .79
Q18 Teacher obeys rules and regulations of school .74
Q3 Teacher has a good working relationship with .61
colleagues
Q23 Teacher maintains a good relationship with .58
administrative staff
Q11 Teacher keeps contact with the parents .53
Q17 Teacher gives attention to increase his/her profession .49
knowledge
Q6 Teacher is punctual in performing duties .87
Q12 Teacher has a good sense of humor .81
Q22 Teacher speaks loudly in the classroom .77
Q10 Teacher gives proper attention to his/her work .71
Q4 Teacher performs his/her duties honestly .62
Q8 Teacher performs his/her duties according to the .51
requirement
Q2 Teacher has a good personality .44
Q21 Teacher is a responsible person .41
Eigen Value 5.45 2.61 1.30
% of Total Variance 31.42 21.77 10.86
Cronbach's Alpha .74 .83 .70
26
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
27
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
assembling new construct by including the individual statements in the construct to get
the total score.
Hypothesized Model 2
In second model, we hypothesized that 24 statements split into three first-order
factors (i.e., instructional qualities, professional qualities and personal qualities) relied
on the findings of previous study. The varimax (orthogonal) rotation in Study-1
extracted three dimensions, thus, Model 2 is considered a reasonable model for current
data.
Table 2
Hypothesized Models and Goodness-of-Fit Indices
1 First-order 3 First-order
Fit Threshold for Threshold for
factor model factor model
measure good fit acceptable fit
value value
Χ² (df) − − 993.52 (102) 243.69 (96)
Χ²/df Χ²/df < 3 3 < Χ²/df < 5 9.41 2.96
NFI .95 ≤ NFI ≤ 1 .90 ≤ NFI ≤ .95 .59 .91
GFI .95 ≤ GFI ≤ 1 .90 ≤ GFI ≤ .95 .82 .93
0 < RMSEA < .05 < RMSEA <
RMSEA .08 .05
.05 .08
TLI .95 ≤ TLI ≤ 1 .90 ≤ TLI ≤ .95 .64 .90
CFI .95 ≤ CFI ≤ 1 .90 ≤ CFI ≤ .95 .61 .92
NFI = Normed Fit Indices; GFI = Goodness of Fit Index; RMSEA = Root Mean Square
Error of Approximation; TLI = Tucker-Lewis Index; CFI = Comparative Fit Index.
In the present study, many indices were utilized to obtain the satisfactoriness of
the hypothesized models (Table 2). Since there is no agreed upon or globally tolerable
statistic as an index of ensuring model adequateness, for this reason, different indices
were measured to check the models. Moreover, chi-square and Goodness of Fit Index
(GFI) were also calculated to check the individual model. In addition, RMSEA was also
employed to calculate the lack of fit in the hypothesized models (Browne & Cudeck,
1993; Steiger & Lind, 1980). In the end, two incremental fit index measures, TLI
(Tucker & Lewis, 1973) and CFI (Bentler, 1990) were also used. Evaluating all the
indices of goodness-of-fit judged in Study-2, first hypothesized model does not give a
considerably better fit by normal standards. However, Model 2 significantly improves
all indices compared to the first model and displays good fit, as specified by the
different indices of model fit.
28
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
Table 3
Mean, SD, Correlation Matrix, and Alpha Reliability Coefficients for TJPS and Sub-
scales
Sr # Measures Mean SD 1 2 3 4
1 Instructional 2.73 0.62 1.00
Qualities
2 Professional 3.12 0.51 .608** 1.00
Qualities
3 Personal 3.04 0.53 .662** .816** 1.00
Qualities
4 TJPS 2.96 0.49 .866** .892** .918** 1.00
Cronbach's α 0.719 0.816 0.773 0.887
**p < 0.01
Pearson correlation matrix for TJPS and its dimensions are good (Table 3).
Dimensions to total scale association and inter-subscales association are significant at p
< .01 that demonstrate the internal consistency and a measure of construct validity.
Overall statistically moderate and high positive correlations were found among the
subscales of TJPS. Professional qualities has a moderate significant correlation (r =
.608, p < .01) with instructional qualities. The facet, personal qualities also has
moderate correlation with instructional qualities (r = .662, p < .01) and high correlation
with professional qualities (r = .816, p < .01). Similarly, TJPS has high correlation with
instructional qualities (r = .866, p < .01), professional qualities (r = .892, p < .01), and
personal qualities (r = .918, p < .01) respectively.
In gender-wise analysis, independent sample t-test was applied to study the
difference among male (n = 303) and female (n = 313) teachers on TJPS.
Table 4
Gender Difference on TJPS
Sr 95% CI
Scales Gender Mean SD t Sig
# LL UL
1 Instructional Male 2.75 0.85 2.327 .032 - -.063
Qualities 1.237
Female 3.40 0.24
2 Professional Male 3.25 0.47 1.274 .219 -.596 .146
Qualities Female 3.48 0.30
3 Personal Qualities Male 3.13 0.66 1.361 .190 -.827 .177
Female 3.45 0.37
4 TJPS Male 3.04 0.60 1.969 .044 -.827 .027
Female 3.44 0.23
29
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
30
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
Table 6
Job Status Difference on TJPS
Sr 95% CI
Scales Job Status Mean SD t Sig
# LL UL
1 Instructional Permanent 2.79 0.72 2.283 .029 .056 .953
Qualities
Contractual 2.28 0.58
2 Professional Permanent 3.15 0.56 3.182 .003 .192 .868
Qualities
Contractual 2.63 0.42
3 Personal Permanent 3.07 0.61 3.691 .001 .278 .958
Qualities
Contractual 2.45 0.32
4 TJPS Permanent 3.00 0.55 3.688 .001 .248 .854
Contractual 2.45 0.25
31
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
Further, CFA was applied to know the items structure and fitness in the second
study. The 3-factor-model was found much reliable with data and revealed better fit as
compared to 1-factor-model, i.e. χ²/df = 2.96, NFI = .91, GFI = .93, RMSEA = .05, TLI
= .90, and CFI = .92. In addition, the correlations between TJPS and its subscales were
also in the expected direction, maintaining the convergent validity of the TJPS. This
result also supports the assumption that the TJPS dimensions evaluate various aspects
of teacher’s job performance. It is also proposed that TJPS is internally reliable and a
valid measure for identifying job performance among school teachers.
Numerous studies have demonstrated that teachers’ gender has its effect on
teachers’ performance and effectiveness. The findings of the current study also revealed
that the difference between male and female teachers regarding performance is
significant. The performance of female teachers was much better than the male teachers
in public schools. Hanif, Tariq, and Nadeem (2011) argued that gender is a strong
predictor of teacher’s job performance. They believed that gender accounted for 15%
variance in teacher’ job performance and emerged as a major determinant of work
performance of teachers. Norlander-Case, Reagan, and Case (1999) expressed that
female teachers tend to perform better in teaching than their male counterparts. On the
contrary, Akiri and Ugborugbo (2008) found no noteworthy dissimilarity between the
performance of male and female secondary school teachers.
Moreover, the study reveals no considerable difference between urban and rural
school teachers regarding performance. The findings provide evidence that the
performance of urban and rural secondary school teachers is same. However, the high
mean score of urban teachers demonstrates better performance of urban teachers as
compared to rural teachers in public schools. This result is also supported by Akiri and
Ugborugbo (2008) described no significant difference between the performance of
urban and semi-urban teachers at secondary school level. However, they reported a
major difference between urban and rural teachers in terms of performance. The study
results also explained a significant difference between the performance of permanent
and contractual teachers in schools. Moreover, the high mean score of permanent
teachers described that they performed better as compared to contractual teachers.
Furthermore, Fyfe (2007) reported no significant difference between the performance of
permanent and contractual teachers. On the other hand, some studies demonstrated that
there is a significant difference between the performance of permanent and contract
teachers. They found contract teachers more efficient as compared to regular teachers
(Atherton & Kingdon, 2010; Hameed, Dilshad, Malik, & Batool, 2014). The low
performance of contractual teachers in the present study may be attributed to various
reasons resulted by weaker commitment, low job satisfaction, low level of
organizational justice, the absence of ethical climate, the harsh behavior of management
and strict supervision of contractual teachers.
32
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
References
Akiri, A. A., & Ugborugbo, N. M. (2008). An examination of gender’s influence on
teachers’ productivity in secondary schools. J. Soc. Sci, 17(3), 185-191.
Armstrong, M. (2007). Performance management, basic strategies and practical
guidelines. Translation N. Mirsepasi and Asmaly Kavoosi. Tehran: Cashmere
Publications.
As’ad, M. (1995). Psikologi Industri. Yogyakarta: Liberty.
Atherton, P., & Kingdon, G. (2010). The Relative Effectiveness And Costs Of Contract
And Regular Teachers In India. Institute of Education, University of London.
Aziz, M. A. (2012). Effects of Demographic Factors & Teachers' competencies on The
Achievement of Secondary School Students in Punjab. Gomal University Journal
of Research, 28(1).
Baldoni, J. (2005). Great Motivation Secrets of Great Leaders (POD): McGraw Hill
Professional.
Bartlett, M. S. (1954). A note on the multiplying factors for various χ 2 approximations.
Journal of the Royal Statistical Society. Series B (Methodological), 296-298.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological
Bulletin, 107, 238–246. http://dx.doi.org/10.1037/0033-2909.107.2.238
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A.
Bollen & J. S. Long (Eds.), Testing Structural Models (pp.136–162). Newbury
Park, CA: Sage.
Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of
performance. Personnel Selection in Organizations, 3570.
Cohen, R. J., & Swerdlik, M. E. (2010). Psychological Testing and Assessment: An
Introduction to Tests and Measurement. Singapore: McGraw-Hill Companies.
Colquitt, J., LePine, J. A., Wesson, M. J., & Wu, X. (2010). Organizational Behavior:
Essentials for Improving Performance and Commitment: Dongbei University of
Finance & Economics Press.
Field, A. (2013). Discovering statistics using IBM SPSS statistics: Sage.
Finch, J. F., & West, S. G. (1997). The investigation of personality structure: Statistical
models. Journal of Research in Personality, 31(4), 439-485.
Fyfe, A. (2007). The use of contract teachers in developing countries: Trends and
Impact. Working Paper: International Labour Office, Geneva.
33
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
34
JOURNAL OF EDUCATIONAL RESEARCH (Vol. 20 No. 1) 2017 Dept. of Education IUB, Pakistan
35