An attribute analysis was developed to simultaneously assess the effects of repeatability and reproducibility on accuracy. It allows the analyst to review the responses of several reviewers if they look at multiple scenarios multiple times. It establishes statistics that assess the ability of evaluators to agree with themselves (repeatability), with each other (reproducibility) and with a master or correct value (overall accuracy) known for each characteristic – over and over again. If the test is planned and designed effectively, it can reveal enough information about the causes of the accuracy problems to justify a decision not to use attribute analysis at all. In cases where the trial does not provide sufficient information, the analysis of the attribute agreement allows for a more detailed review to inform the introduction of training changes and error correction in the measurement system. However, a bug tracking system is not an ongoing payment. The assigned values are correct or not; There is no (or should not) grey area. If codes, locations and degrees of gravity are defined effectively, there is only one attribute for each of these categories for a particular error. Yes, for example. B Repeatability is the main problem, evaluators are disoriented or undecided by certain criteria. When it comes to reproducibility, evaluators have strong opinions on certain conditions, but these opinions differ. If the problems are highlighted by several assessors, the problems are naturally systemic or procedural.

If the problems only concern a few assessors, then the problems might simply require a little personal attention. In both cases, training or work aids could be tailored to either specific individuals or all evaluators, depending on the number of evaluators who were guilty of imprecise attribution of attributes. There are many ways to make a decision within a group, for example. B a decision in the schedule, a specification or, for example, target costs. Even in a software development, the size of the residue is determined by the SCRUM team. As performing an attribute analysis can be tedious, costly and generally uncomfortable for all stakeholders (the analysis is simple versus execution), it is best to take a moment to really understand what should be done and why. In this example, a repeatability assessment is used to illustrate the idea, and it also applies to reproducibility. The fact is that many samples are needed to detect differences in an analysis of the attribute, and if the number of samples is doubled from 50 to 100, the test does not become much more sensitive. Of course, the difference that needs to be identified depends on the situation and the level of risk that the analyst is prepared to bear in the decision, but the reality is that in 50 scenarios, it is difficult for an analyst to think that there is a statistical difference in the reproducibility of two examiners with match rates of 96 percent and 86 percent. With 100 scenarios, the analyst will not be able to see any difference between 96% and 88%. The reasons why the chords (consistencys) were weak could be the reasons: unlike a continuous measurement that cannot be precise (on average), any lack of precision in an attribute system inevitably leads to accuracy problems. If the error coder is not clear or undecided on how to encode a defect, different codes are assigned to several defects of the same type, making the database imprecise.