Kinship Verification (Track 1)
Kinship verification is intended to determine whether or not a pair of facial images are blood relatives of a particular type (e.g., parent-child). This is a classical boolean problem with system responses being either kin and non-kin (i.e., related and unrelated, respectfully). Thus, this task tackles the one-to-one view of automatic kinship recognition.
Challenge portal is open to all individuals and teams! Register now at CodaLab, https://competitions.codalab.org/competitions/16745.
Challenge portal is open to all individuals and teams! Register now at CodaLab, https://competitions.codalab.org/competitions/16745.
Intended Use
Prior research efforts have considered mainly considered parent-child kinship types, i.e., father-daughter (F-D), father- son (F-S), mother-daughter (M-D), mother-son (M-S). While far less, but still some, attention has been given to sibling pairs, i.e., Sister-Sister (S-S), Brother-Brother (B-B), and siblings of opposite sex (SIBS). As research in both psychology and computer vision revealed, different kin relations render different familial features and, hence, the four kin relations are generally treated differently during the model training. Thus, additional kinship relations (i.e., pairwise types) would further both our understanding and capabilities of automatic kinship recognition. With the release of FIW, the number of facial pairs accessible for kinship verification has greatly increased; along with four additional relationship types (i.e., grandparent- grandchild) were made available (see the middle column of the following Figure).
Data Splits
FIW includes a total of 644,000 pairs, from which 538,519 from the 7 different types will be used for this data challenge. Thus, modern-day data-driven approaches (i.e., deep learning) can now be employed to the problem of kinship verification like never before possible.
The data for Track-1 (i.e., verification) is partitioned into 3 disjoint sets referred to as Train, Validation, and Test sets-- ground truth for the former will be provided during Phase 1 for self-evaluation, while runs on the Validation can be submitted for scoring. Ground truth for Validation will be made available during Phase 2. The "blind" Test set will be released during Phase 3. No labels will be provided for the Test set until the challenge is adjourned and results are reported. Teams will be asked to only process the Test set to generate submissions and, hence, any attempt of analyzing or understanding the Test set is prohibited. All sets will be made up of an equal number of positive and negative pairs. Lastly, note that there is no family or subject identity overlapping between any of the sets. |
Evaluation Settings and Metrics
Following conventional face verification protocols, we offer 3 modes (or settings) for evaluation, which are listed as follows:
Participants will be allowed to submit up to 6 sets of results for each mode (i.e., teams participating in all 3 settings will be allowed to submit up to 18 sets of results). Note that runs must be processed and, hence, submitted independent of one another.
For all modes, the metric used is accuracy. Also, true positives as a function of false positives (i.e., ROC curves) will also be used for comparing scores.
- Unsupervised: No labels provided, i.e., no prior knowledge about kinship or subject IDs will be provided.
- Image-restricted: Kin/ non-kin labels will be provided for Development set– note there is no family overlap between training and testing.
- Image-unrestricted: Kinship labels and subject IDs will be provided– allows parsing of additional negative pairs.
Participants will be allowed to submit up to 6 sets of results for each mode (i.e., teams participating in all 3 settings will be allowed to submit up to 18 sets of results). Note that runs must be processed and, hence, submitted independent of one another.
For all modes, the metric used is accuracy. Also, true positives as a function of false positives (i.e., ROC curves) will also be used for comparing scores.