Estimating the skills of individuals is acknowledged to be an important open problem with vast applications to optimal team formation. One difficulty with skill estimation is that the skills in question are generally considered to be “subjective” in the sense that a simple test (such as the number of words that you can type per minute) is not considered an option. Example skills may be “spring MVC programming,” “written communication in English,” “independent research in the field of graph theory,” “basketball dribbling,” and so on. Each of the skills has some depth and width associated with it, and in many cases the standards of assessment and interpretations of those assessments can change over time as well, making the task of accurate skill assessment even harder.

This paper attempts to solve this problem using an interesting approach. Rather than any assessments (whether mutual or standardized), it uses the data provided in the form of completed tasks undertaken by teams to estimate the skills of the individuals. To make the problem more amenable, they make certain assumptions, the prominent ones being: (1) a worker’s specific skill can be modeled as a deterministic value or a probability distribution function; (2) the skill of a team can be the sum or the maximum of skills of all workers (SUM and MAX aggregation functions); (3) task quality is computed by an objective quality evaluation metric and is shown as a numeric score for each task; (4) there is a correspondence between worker skills and task quality; and (5) there is sufficient data to perform these calculations (an overdetermined system). Further, by making the assumption that the quality of estimation result can be assessed by the size of the reconstruction error, they transfer the original problem to an optimization problem. Based on the above assumptions, they provide algorithms for different skill models and skill aggregation functions: (1) SUM-D, using active-set or interior-point methods in polynomial time; (2) SUM-P, using a designed local search algorithm to minimize the reconstruction error; (3) MAX-D, using max-plus algebra to compute a principal solution in polynomial time; and (4) MAX-P using a greedy algorithm but using random restarts.

The authors present experimental results of all algorithms. These results are derived using primarily two scenarios: (1) the ground truth is available (such as in the case of basketball/NBA data), and (2) no ground truth is available (such as considering a skill of “research”). In the latter case, the authors take a cross-validation type of approach for evaluation.

The overall problem of skill estimation is obviously interesting and practical from a human resources perspective. However, whether the two models (SUM and MAX aggregation functions) can be used to model the more complex world of real teams, and whether the proposed solution (estimation of skill using completed tasks) is a feasible one, are questions that merit further discussion.