How about utilizing ordinal information from the distribution of unlabeled data.

Mingjie Qian, Bo Chen, Hongzhi Xu, Hongwei Qi

Browse the full CIKM paper archive.