Imagine that you are facilitating a subject matter expert (SME) panel to develop a job knowledge test for plastics chemists. The SME panel, after extensive discussions and examining the results of job analysis, concludes that the test content domain should include 9 areas: mathematics, statistics, physics, chemistry, marketing, plastics testing, product design, mold and die design, and plastics processing. The proposed job knowledge test would include a total of 50 multiple-choice test questions.Below is a frequency table of SME panel ratings of importance for each content area. Construct a test plan by completing the table below to show:

QUESTION

ANSWER

Test Plan for Developing a Job Knowledge Test for Plastics Chemists

Introduction

Developing a comprehensive job knowledge test for plastics chemists requires careful consideration of the content areas that are relevant to the job role. In this test plan, we will outline the mean importance ratings and proportion of content importance for each of the nine identified areas, namely mathematics, statistics, physics, chemistry, marketing, plastics testing, product design, mold and die design, and plastics processing. Additionally, we will determine the number of test items needed to cover each content area within a 50-item test.

Content Area 1: Mathematics

The mean importance ratings provided by the SME panel for mathematics are 3, 3, 4, 3, 5, 4, and 3. To calculate the mean importance, we sum up these ratings and divide by the number of ratings (7 in this case) (Bewick et al., 2003). The total sum is 25, giving us a mean importance rating of 25/7 ≈ 3.57. Consequently, the proportion of content importance for mathematics is 3.57/50 ≈ 7.14%. To determine the number of test items needed, we multiply the proportion of content importance by the total number of test items, resulting in 7.14% of 50 items, which is approximately 3.57 items.

Content Area 2: Statistics

The SME panel assigned importance ratings of 4, 4, 4, 3, 5, 4, and 4 for statistics. The mean importance rating for statistics is calculated as the sum of these ratings divided by 7, resulting in a mean importance of 4. To calculate the proportion of content importance, we divide the mean importance rating by the total number of test items (50), yielding 4/50 = 8%. Hence, for statistics, approximately 8% of the test should cover this area, which corresponds to 4 items.

Content Area 3: Physics

For physics, the importance ratings assigned by the SME panel are 1, 2, 2, 3, 2, 1, and 3. The mean importance rating is calculated as the sum of these ratings divided by 7, giving us a mean importance of 2 (Taber, 2017). In terms of proportion, physics should constitute 2/50 = 4% of the test content. Therefore, we would need approximately 2 items to cover this content area adequately.

Content Area 4: Chemistry

The SME panel unanimously assigned the highest importance rating of 5 to chemistry. With seven ratings of 5, the mean importance rating for chemistry is 5. Thus, chemistry should account for 5/50 = 10% of the test content (Brusick et al., 2020). Consequently, we should include approximately 5 items to cover this essential area adequately.

Content Area 5: Marketing

Marketing received importance ratings of 2, 2, 1, 3, 1, 1, and 1 from the SME panel. The mean importance rating is calculated as the sum of these ratings divided by 7, resulting in a mean importance of 1.57. The proportion of content importance for marketing is 1.57/50 ≈ 3.14%. Therefore, we should include approximately 1.57 items, which can be rounded up to 2 items, to cover this content area sufficiently.

Content Areas 6 to 9: Plastics Testing, Product Design, Mold & Die Design, Plastics Processing

Similar calculations are performed for these content areas based on the provided importance ratings. The mean importance ratings and proportion of content importance for each area, as well as the number of items needed, are as follows:

Plastics Testing: Mean Importance = 4, Proportion of Importance = 8%, Test Items Needed = 4 items.

Product Design: Mean Importance = 3.43, Proportion of Importance = 6.86%, Test Items Needed = 3 items.

Mold & Die Design: Mean Importance = 3, Proportion of Importance = 6%, Test Items Needed = 3 items.

Plastics Processing: Mean Importance = 4.43, Proportion of Importance = 8.86%, Test Items Needed = 4 items.

Conclusion

This test plan provides a comprehensive overview of the mean importance ratings, proportion of content importance, and the number of test items needed for each content area in the development of a job knowledge test for plastics chemists. By considering the expertise and input of the SME panel, this test plan ensures that the test content aligns with the job requirements and covers the necessary knowledge areas effectively. Developing a well-rounded and valid test will contribute to accurately assessing the knowledge and competencies of plastics chemists, enabling organizations to make informed decisions in their selection and evaluation processes.

References

Bewick, V., Cheek, L., & Ball, J. K. (2003). Statistics review 7: Correlation and regression. Critical Care, 7(6), 451. https://doi.org/10.1186/cc2401 

Brusick, D., Aardema, M. J., Allaben, W. T., Kirkland, D., Williams, G. M., Llewellyn, G. C., Parker, J. E., & Rihner, M. O. (2020). A weight of evidence assessment of the genotoxic potential of 4-methylimidazole as a possible mode of action for the formation of lung tumors in exposed mice. Food and Chemical Toxicology, 145, 111652. https://doi.org/10.1016/j.fct.2020.111652 

Taber, K. S. (2017). The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2