Conflicts and disagreements are inherent in collaborative projects such as the team-driven design of diagnosis algorithms. This paper proposes a method to resolve these conflicts. The method is based on six algorithms for diagnosing a multiagent team, depending on the ways to select and compute the diagnoses.
The paper is well written and well structured. Even without being an expert in the field, a reader can understand the problem that the authors intend to solve, and acknowledge its relevance. The research questions are systematically defined and motivated, while the assumptions behind them are formally and carefully specified.
The paper does have several negatives, but they do not seriously affect its quality. First, while the authors place their findings in relation to the work by Kaminka and Tambe [1], who apply similar methods, this comparison should be made more explicit in section 7. Second, the evaluation procedure seems not to rely on standardized test units/sets, which makes a comparison with related works difficult if not impossible. In addition, the experiments and their results should be further detailed, in order to increase the relevance and significance of the results. For instance, the evaluation includes average values for experiments conducted several times, but does not mention standard error values. A formal validation of the approach is missing, as mentioned by the authors themselves.