Providing Editing Expertise

The editing phase of newly written items offers a second opportunity to correct common item flaws. Reviewing the items in a bank is commonly done with a group of Subject Matter Experts (SMEs). To increase the clarity of items, we’ll profit from the feedback provided by item statistics to improve items.  

• We capitalize on PCS’ combined editing experience of over 20 years; quickly recognizing item flaws and correcting them with the item-writer; and
• We utilize PCS’ staff to collaboratively improve items, and, at the same time, uphold item-writing standards that yield high quality examination scores.

Training Subject-Matter-Experts (SMEs)

After our training, SMEs will be able to harness their expertise in writing high quality examination items that distinguish between qualified and unqualified candidates, and avoid common item-writing pitfalls.

• We train your SMEs to use statistics calculated for each answer choice to diagnose which answer choices are not working well;
• We show your SMEs how to recognize, and to avoid, writing items that ask two questions at once, and are needlessly confusing for candidates;
• We help your SMEs write good distractor answer choices that are plausible; but not too close to being the correct, keyed, answer; and
• We demonstrate to your SMEs how to use an item statistic that indicates if an item is distinguishing between knowledgeable and less knowledgeable candidates.  

Conducting Professional Item-Writing Workshops

When conducting a full item-writing training session, we expose attendees to bite-sized chunks of the curriculum, and then have them apply what they’ve learned by correcting flawed items in small group discussions.  

• We work face-to-face with SMEs, but we also enjoy working remotely online to conserve resources; and
• We use online classroom tools that allow small groups to breakout for a task or use something as simple as our securely accessed item writing-tool; where SMEs can upload their exam items for review.  

Working With Your Experts

Our in-house psychometrician will work with your experts to derive a current, actionable examination blueprint by analyzing the tasks and duties of jobs then mapping those onto relevant knowledge, skills, and abilities (KSAs). These KSAs then become the basis of a current, defensible exam blueprint.

• We aid in constructing a solid blueprint – an extremely important foundational document;
• We select and build the best test design for your needs;
• We improve the quality and measurement precision of your examination through our psychometric expertise; and
• We manage this entire process with you and utilize our experience to make sure that your test blueprint represents the domain of KSAs correctly.

Helping You Think Outside the Box

Trying to decided if your examination should be converted from paper and pencil to a computer-based test (CBT)? We’ll help you to analyze the implications of converting to CBT and to make a sound decision. We have helped other organizations plan and make this transition and we can help yours. Some of the benefits of using CBTs are:

• CBTs are likely to be more secure by allowing more thorough identification of candidates during registration and at the testing site;
• CBTs are able to issue scores to candidates more rapidly – increasing candidate satisfaction;
• The range of item types that can be accommodated using a CBT is much larger than that of a paper and pencil exam;
• Video can be incorporated into examination questions. Presenting scenarios or sounds or key visual elements can greatly increase the similarity of the testing task to actual tasks in the profession or occupation; and
• CBTs allow a number of test designs to be operationalized.  

Utilizing Our Psychometric Expertise

Working side-by-side with your SMEs to revise items, we aid in validating your examination and making statistical adjustments so that comparable active forms of an examination yield comparable scores, unaffected by small differences in the overall difficulty of different forms of an examination. Our philosophy is to provide quantitative analyses that include results expressed in non-technical language, suited to the audience.

• We use industry-standard tools to assemble test forms that balance content areas and statistical constraints.  Our competition may provide similar services, but what differentiates PCS is that we can use our size to treat modest testing programs as if they test thousands of candidates per year and large test programs like they’re the only customer around; and
• We scrutinize your examinations after assembly with an expert human eye, but in a bigger sense, all of our services, whether they be test assembly, picking test designs, crafting blueprints, editing questions, or training SMEs are delivered by professionals that take the time to know you, your examination needs, your candidate population, and the environment in which you operate.  

Contact PCS today for more information on any of our products and services at 877.887.9727, [email protected], or here. We’re waiting to hear from you.