World Scientific
Skip main navigation
Close Drawer MenuOpen Drawer Menu

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

During this period, the E-commerce and registration of new users may not be available for up to 6 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.
No Access

A COMPREHENSIVE SURVEY OF THE REVIEWER ASSIGNMENT PROBLEM

    https://doi.org/10.1142/S0219622010003993Cited by:14
    Previous

    Abstract

    Reviewer Assignment Problem (RAP) is an important issue in peer-review of academic writing. This issue directly influences the quality of the publication and as such is the brickwork of scientific authentication. Due to the obvious limitations of manual assignment, automatic approaches for RAP is in demand. In this paper, we conduct a survey on those automatic approaches appeared in academic literatures. In this paper, regardless of the way reviewer assignment is structured, we formally divide the RAP into three phases: reviewer candidate search, matching degree computation, and assignment optimization. We find that current research mainly focus on one or two phases, but obviously, these three phases are correlative. For each phase, we describe and classify the main issues and methods for addressing them. Methodologies in these three phases have been developed in a variety of research disciplines, including information retrieval, artificial intelligence, operations research, etc. Naturally, we categorize different approaches by these disciplines and provide comments on their advantages and limitations. With an emphasis on identifying the gaps between current approaches and the practical needs, we point out the potential future research opportunities, including integrated optimization, online optimization, etc.

    References

    • P. Evans, Literati Newsline: Special Issue for Authors and Editors 2, (1995). Google Scholar
    • H. El-Munchid, Annals of Saudi Medicine 21, 5 (2001). ISI, Google Scholar
    • V. Bence and C. Oppenheim, Journal of Information Science 30, 347 (2004). Crossref, ISI, Google Scholar
    • S. Santini, IEEE Computer 38, 128 (2005). Crossref, Google Scholar
    • F. Casati,  Giunchiglia and F. M. Marchese, Ubiquity 8, 1 (2007). Google Scholar
    • JAIRJournal of Artificial Intelligence Research e-journal  (2008), www.jair.org. Google Scholar
    • R. Smith, British Medical Journal 318, 4 (1999). Crossref, Google Scholar
    • R. Weber, Communications of the Association for Information Systems 2, (1999). Google Scholar
    • S. Harnad, D-Lib Magazine 5, (1999). Google Scholar
    • J. C. Wei, D. Hartvigsen and R. Czuchlewski, Decision Science 30, 865 (1999). ISI, Google Scholar
    • M. A. Rodriguez and J. Bollen, An Algorithm to Determine Peer-Reviewers, 2006 , arXiv:cs.DL/0605112 . Google Scholar
    • S. Dumais and J. Nielsen, Automating the assignment of submitted manuscripts to reviewers, Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (1992) pp. 233–244. Google Scholar
    • A. Pesenhofer, R. Mayer and A. Rauber, Improving scientific conferences by enhancing conference management systems with information mining capabilities, Proceedings of IEEE International Conference on Digital Information Management (ICDIM 2006) pp. 359–366. Google Scholar
    • R. T.   Snodgrass and J.   Li , Summary of Conference Management Software ( 2009 ) ,   http://www.acm.org/sigs/sgb/summary.html . Google Scholar
    • J. Goldsmith and R. H. Solan, The AI conference paper assignment problem, Proc. AAAI 2007 Workshop on Preference Handling for Artificial Intelligence (2007) pp. 53–57. Google Scholar
    • S.   Hettich and M. J.   Pazzani , Mining for proposal reviewers: Lessons learned at the national science foundation , Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'06) ( 2006 ) . Google Scholar
    • P. Willett, Information Processing and Management 24, 577 (1998). Crossref, ISI, Google Scholar
    • B. Larsen and A. Chinatsu, Fast and effective text mining using linear-time document clustering, Proceedings of the fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (1999) pp. 16–22. Google Scholar
    • H. K.   Biswas and M. M.   Hasan , Using publications and domain knowledge to build research profiles: An application in automatic reviewer assignment , Proceedings of the 2007 International Conference on Information and Communication Technology ( 2007 ) . Google Scholar
    • C. Giles, K. Bollacker and S. Lawrence, CiteSeer: An automatic citation indexing system, The Third ACM Conference on Digital Libraries (1998) pp. 89–98. Google Scholar
    • G.   Fox et al. , Information Reuse and Integration (IEEE IRI-2007) ( Las Vegas, USA , 2007 ) . Google Scholar
    • Y. H.   Sun et al. , A hybrid knowledge and model approach for reviewer assignment , Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07) ( 2007 ) . Google Scholar
    • A. D. Henriksen and A. J. Traynor, IEEE Trans. Eng. Manage. 46, 158 (1999). Crossref, ISI, Google Scholar
    • Q. Tian, J. Ma and O. Liu, Expert Syst. Appl. 23, 265 (2002). Crossref, ISI, Google Scholar
    • Q. Tianet al., Decis. Support Syst. 39, 403 (2005). Crossref, ISI, Google Scholar
    • B. Aleman-Mezaet al., Semantic analytics on social networks: Experiences in addressing the problem of conflict of interest detection, Proceedings of the 15th International World Wide Web Conference (2006) pp. 407–416. Google Scholar
    • M.   Papagelis , D.   Plexousakis and P. N.   Nikolaou , CONFIOUS: Managing the electronic submission and reviewing process of scientific conferences , Proc. of the 6th International Conference on Web Information Systems Engineering ( 2005 ) . Google Scholar
    • J.   Li et al. , Expert finding for eCollaboration using FOAF with RuleML rules , Conference on eTechnologies ( 2006 ) . Google Scholar
    • U.   Bojars , Extending FOAF with resume information , First Workshop on FOAF, Social Networks and the Semantic Web ( 2004 ) . Google Scholar
    • S. R.   Kruk and S.   Decker , Semantic social collaborative filtering with FOAFRealm , First Semantic Desktop Workshop ( 2005 ) . Google Scholar
    • A. Schirrer, K. F. Doerner and R. F. Hartl, Operations Research/Computer Science Interfaces Series 39, 113 (2007). Crossref, Google Scholar
    • D. Hartvigsen, J. C. Wei and R. Czuchlewski, Decision Sciences 30, 865 (1999). Crossref, ISI, Google Scholar
    • R. K.   Ahuja , T. L.   Magnanti and J. B.   Orlin , Network Flows: Theory, Algorithms, and Applications ( Prentice Hall , 2003 ) . Google Scholar
    • G.   Salton and M.   McGill , An Introduction to Modern Information Retrieval ( McGraw-Hill , New York , 1983 ) . Google Scholar
    • P. R. Cohen and R. Kjeldsen, Information Processing & Management 23, 255 (1987). Crossref, ISI, Google Scholar
    • S. Deerwesteret al., Journal of the American Society for Information Science 41, 391 (1990). Crossref, ISI, Google Scholar
    • D.   Yarowsky and R.   Florian , Taking the load off the conference chairs: Towards a digital paper-routing assistant , Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in NLP and Very-Large Corpora ( 1999 ) . Google Scholar
    • K. Antoninaet al., Prototype-matching system for allocating conference papers, Proceedings of the 36th Annual Hawaii International Conference on System Sciences (2003) pp. 70–80. Google Scholar
    • J. Konstanet al., Communications of the ACM 40, 77 (1997). Crossref, ISI, Google Scholar
    • W. Hillet al., Recommending and evaluating choices in a virtual community of use, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2005) pp. 194–201. Google Scholar
    • C.   Basu , H.   Hirsh and W.   Cohen , Recommendation as classification: Using social and content-based information in recommendation , Proceedings of the Fifteenth National Conference on Artificial Intelligence ( 1998 ) . Google Scholar
    • U.   Shardanand and P.   Maes , Social information filtering: Algorithms for automating "Word of Mouth" , Proceedings of CHI-95 ( 1995 ) . Google Scholar
    • W. Cohen and W. Fan, Web-collaborative filtering: Recommending music by crawling the web, Proceedings of the 9th International World Wide Web Conference on Computer Networks: The International Journal of Computer and Telecommunications Networking Amsterdam (2000) pp. 685–698. Google Scholar
    • D.   Gupta et al. , Jester 2.0: A new linear time collaborative filtering algorithm applied to jokes , Workshop on Recommender Systems Algorithms and Evaluation, 22nd International Conference on Research and Development in Information Retrieval ( 1999 ) . Google Scholar
    • L. Yu, L. Liu and X. Li, Expert Systems with Applications 28, 67 (2005). ISI, Google Scholar
    • S. Watanabeet al., A paper recommendation mechanism for the research support system papits, Proceeding of International Workshop on Data Engineering Issues in E-Commerce (2005) pp. 71–80. Google Scholar
    • M. A. Rodriguez and J. Bollen, Simulating Network Influence Algorithms Using Particle-Swarms: Pagerank and Pagerank-priors, CoRR abs/cs/0602002 (2006) . Google Scholar
    • X. Liuet al., Information Processing and Management 41, 1462 (2005). Crossref, ISI, Google Scholar
    • E. Atlam, International Journal of Information Technology & Decision Making 7(1), 23 (2008). Link, ISI, Google Scholar
    • A.   Popescul et al. , Probabilistic models for unified collaborative and content-base recommendation in sparse-data environments , Proceeding of the 17th Conference on Uncertainty in Artificial Intelligence (UAI-2001) ( Morgan Kaufmann , 2001 ) . Google Scholar
    • T. Hofmann, Probabilistic latent semantic analysis, Proceeding of the 15th Conference on Uncertainty in Artificial Intelligence (1999) pp. 289–296. Google Scholar
    • C. Basuet al., Journal of Artificial Intelligence Research 14, 231 (2001). Crossref, ISI, Google Scholar
    • T. J. Siddiqui and U. S. Tiwary, International Journal of Information Technology & Decision Making 7, (2008). Google Scholar
    • Y. Penget al., International Journal of Information Technology & Decision Making 7(4), 639 (2008). Link, ISI, Google Scholar
    • K. Tatsumi and T. Tanino, International Journal of Information Technology & Decision Making 7(1), 53 (2008). Link, ISI, Google Scholar
    • S. L. Janaket al., Industrial & Engineering Chemistry Research 45, 258 (2006). Crossref, ISI, Google Scholar
    • A. Caprara, P. Toth and M. Fischetti, Annals of Operations Research 98, 353 (2000). Crossref, ISI, Google Scholar
    • M. Dell'Amico and S. Martello, Discrete Applied Mathematics 76, 103 (1997). Crossref, ISI, Google Scholar
    • L. J. LeBlanc, D. Randels and T. K. Swann, Interface 30, 95 (2000). Crossref, Google Scholar
    • M. W. Carter and C. A. Tovey, Operations Research 40, 28 (1992). Crossref, Google Scholar
    • P. Hansen and R. E. Wendell, Interfaces 11, 85 (1982). ISI, Google Scholar
    • G. Caron, P. Hansen and B. Jaumard, Operations Research 47, 449 (1999). Crossref, ISI, Google Scholar
    • D. Klingman and N. Phillips, Management Sciences 30, 1362 (1984). Crossref, ISI, Google Scholar
    • D. O. Bauschet al., Interfaces 21, 26 (1991). Crossref, Google Scholar
    • A. Veronika, L. S. Riantini and B. Trigunarsyah, Corrective action recommendation for project cost variance in construction material management, Proceedings of the 10th East Asia-Pacific Conference on Structural Engineering and Construcion (2006) pp. 23–28. Google Scholar
    • J. J. Merelo-Guervoset al., How evolutionary computation and Perl saved my conference, MAEB03, Segundo Congreso Espaool sobre Metaheurosticas, ed. L. Sonchez (Algoritmos Evolutivos y Bioinspirados, 2003) pp. 93–99. Google Scholar
    • S. Benferhat and J. Lang, International Journal of Intelligent Systems 16, 1183 (2001). Crossref, ISI, Google Scholar
    • D. G. Cattrysse and L. N. Van Wassenhove, European Journal of Operational Research 60, 260 (1992). Crossref, ISI, Google Scholar
    • M. Savesbergh, Operational Research 45, 831 (1997). Crossref, Google Scholar
    • J. J. Merelo-Guervos and P. Castillo-Valdivieso, Conference paper assignment using a combined Greedy/evolutionary algorithm, Lecture Notes in Computer Science (2004), pp. 602–611 . Google Scholar
    • P. Moscato and C. Cotta, Handbook of Metaheuristics, eds. Fred Glover and Gary A. Kochenberger (Kluwer Academic Publishers, 2003) pp. 105–144. Crossref, Google Scholar
    • W. D. Cooket al., Management Science 51, 655 (2005). Crossref, ISI, Google Scholar
    • M. M. Solomon, Operations Research 35, 254 (1987). Crossref, ISI, Google Scholar
    • Netflix, http://www.netflixprize.com/Last check date: November 4, 2008 . Google Scholar
    • R.   Jin et al. , Collaborative filtering with decoupled models for preferences and ratings , Proceedings of the 12th International Conference on Information and Knowledge Management ( 2003 ) . Google Scholar
    • C. Christakou, S. Veettos and A. Stafylopatis, International Journal of Information Technology & Decision Making 7(4), 639 (2008). ISI, Google Scholar
    • J.   Lehmann , J.   Schuppel and S.   Auer , Discovering unknown connections — The DBpedia relationship finder , Proceedings of the 1st SABRE Conference on Social Semantic Web (CSSW) ( 2007 ) . Google Scholar
    • G. Adomavicius and Y. Kwon, IEEE Intelligent Systems 22, 48 (2007). Crossref, ISI, Google Scholar
    • D. W.   Fellner and M.   Zens , Managing Conference Proceeding ( TUBSCG TU , Braunschweig , 2001 ) ,   cgv.tugraz.at . Google Scholar
    • J. Geller, How IJCAI 1999 can prove the value of AI by using AI, Proceeding of the 15-th International Joint Conference on Artificial Intelligence (1997) pp. 55–58. Google Scholar
    • J.   Geller and R.   Scherl , Challenge: Technology for Automated Reviewer Selection ( 1997 ) ,   http://njit.edu/geller/finalchall.ps . Google Scholar