Acquiring Knowledge for Evaluation of Teachers Performance in Higher Education using a Questionnaire

In this paper, we present the step by step knowledge acquisition process by choosing a structured method through using a questionnaire as a knowledge acquisition tool. Here we want to depict the problem domain as, how to evaluate teachers performance in higher education through the use of expert system technology. The problem is how to acquire the specific knowledge for a selected problem efficiently and effectively from human experts and encode it in the suitable computer format. Acquiring knowledge from human experts in the process of expert systems development is one of the most common problems cited till yet. This questionnaire was sent to 87 domain experts within all public and private universities in Pakistani. Among them 25 domain experts sent their valuable opinions. Most of the domain experts were highly qualified, well experienced and highly responsible persons. The whole questionnaire was divided into 15 main groups of factors, which were further divided into 99 individual questions. These facts were analyzed further to give a final shape to the questionnaire. This knowledge acquisition technique may be used as a learning tool for further research work.

💡 Research Summary

The paper tackles the perennial bottleneck in expert‑system development: acquiring high‑quality domain knowledge in a systematic, cost‑effective manner. The authors focus on the specific problem of evaluating university teachers’ performance in higher education and propose a structured questionnaire as the primary knowledge‑acquisition instrument. After an initial literature review and informal consultations, they identified fifteen macro‑categories that together capture the multifaceted nature of teacher performance—curriculum design, instructional delivery, research output, student mentorship, community service, administrative duties, and continuous professional development, among others. Each macro‑category was broken down into concrete items, resulting in a questionnaire comprising 99 individual questions.

The questionnaire was distributed to 87 experts drawn from both public and private universities across Pakistan. Respondents were selected for their senior academic rank, doctoral qualifications, and an average of 15 years of teaching experience, ensuring that the collected data would reflect deep, practice‑based insight. Of the 87 invitations, 25 experts returned completed questionnaires, yielding a response rate of roughly 29 percent. Although modest, the response pool was deemed sufficiently knowledgeable for the study’s exploratory goals.

Statistical analysis was performed using standard software packages. Descriptive statistics (frequency, mean, standard deviation) provided a first‑order view of expert opinions, while exploratory factor analysis (EFA) identified underlying dimensions that explain variance across the 99 items. The EFA revealed that “curriculum design” and “student feedback utilization” accounted for the largest proportion of variance, confirming their central role in performance evaluation. Additional factors such as “research productivity” and “community engagement” also emerged as significant, albeit with lower loadings. Inter‑rater reliability was assessed via Cohen’s Kappa, indicating moderate agreement among experts and lending credibility to the aggregated judgments.

Based on these quantitative findings, the authors constructed a preliminary rule‑base for a teacher‑performance expert system. Sample rules illustrate how the system could combine multiple criteria: for instance, a teacher scoring above 4 on curriculum design and above 3.5 on student satisfaction would be classified as “high‑performing.” Such rules translate the nuanced expert judgments captured in the questionnaire into actionable decision logic that can be embedded in a knowledge‑based application.

The paper also discusses the broader methodological implications of using questionnaires for knowledge acquisition. Advantages highlighted include scalability (the ability to reach many experts simultaneously), cost efficiency (minimal travel or face‑to‑face interview expenses), and the generation of quantifiable data amenable to statistical validation. However, the authors acknowledge several limitations. First, the geographic confinement to Pakistani institutions raises concerns about cultural and institutional bias; the identified performance factors may not fully generalize to other national contexts. Second, questionnaires primarily elicit explicit knowledge, potentially overlooking tacit insights that experts might convey more naturally in open‑ended interviews or observation sessions. Third, the response rate, while acceptable for exploratory work, suggests a risk of non‑response bias—those who chose to reply may systematically differ from those who did not.

To mitigate these drawbacks, the authors propose a mixed‑methods follow‑up: supplementing the questionnaire with semi‑structured interviews, focus groups, and on‑site observations to capture richer, context‑dependent knowledge. They also recommend expanding the expert pool to include international scholars, thereby testing the cross‑cultural robustness of the identified performance dimensions. Finally, they suggest developing a prototype expert system based on the derived rule‑base, deploying it in a pilot setting, and evaluating its predictive accuracy, user acceptance, and impact on institutional decision‑making.

In conclusion, the study demonstrates that a well‑designed, domain‑specific questionnaire can serve as an effective conduit for extracting structured expert knowledge, even in complex evaluation tasks such as teacher performance assessment. By documenting each step—from factor identification and questionnaire construction to statistical validation and rule formulation—the authors provide a replicable framework that can be adapted to other domains (e.g., medical diagnosis, corporate talent appraisal, public policy analysis). The work thus contributes both substantive insights into higher‑education performance metrics and methodological guidance for future knowledge‑acquisition endeavors in expert‑system development.