Confused about the Publication Forum’s Ratings for Conference Publications?

The Publication Forum changed the definition criteria for conferences substantially in June 2016: since the year 2017, conferences are rated individually by their own established name only in cases in which their Publication Forum level substantially differs from the level of the series or publisher serving as the publication channel. With the change, the rating of over 3000 conferences was removed and replaced with ratings based on publication channels. Together with the conference name, the ISSN or ISBN number of the channel has been a mandatory data field in publication information collection already before the 2016 change.

As a part of the responsibility of validating the publication information of the University of Turku researchers, the Library has invested remarkably in identifying the publication channel information of conferences. New identification rules have also been created in the University’s data warehouse from which publication information is transferred to the national VIRTA publication data service on a nightly basis. Nevertheless, occasional errors in the Publication Forum ratings of the conferences in the data warehouse report may occur. In case you identify these errors, please contact the warehouse maintenance in varasto@utu.fi and the Library in researchportal@utu.fi. 

The national VIRTA publication data service transfers all publication information to be reported to the Ministry of Education. Rules for defining the correct publication channel to a conference irrespective of whether an organization reports a Publication Forum identifier or class in an individual publication entry have now been established in VIRTA. The core of the seemingly complex diagram is that the rating level of peer-reviewed conference articles (type A4) is defined by the name of the conference event whenever the conference is rated individually in the Publication Forum’s publication channel database. In case the conference has not been rated individually by its name, the rating level is as a rule defined by the publication series serving as the publication channel (the ISSN number). Finally, in case the publication series has not been rated by the Publication Forum either, the rating is defined by the book’s publisher (the ISBN prefix).

You can browse publication information transferred to the VIRTA service in real time in the national JUULI portal.

Using the Publication Forum classification for evaluation

From the viewpoint of responsible metrics, using publications’ Publication Forum (JUFO) classification to research evaluation must be examined critically and the limitations of the classification must be taken into account.

The user guide for the Publication Forum classification has been updated by the Publication Forum steering group in early summer 2019.  It has to be remembered that the classification system was originally meant for the evaluation of a large number of publications produced by universities.

The Publication Forum classification is not intended for 

• the evaluation of the quality of a small number of publications
• the evaluation of the quality of individual publications
• the evaluation or comparison of individual researchers
• comparison between scientific disciplines

Research groups or units should only be evaluated, if there is a sufficiently large number of publications. The Publication Forum classification is only suitable for the evaluation of profiles and internal development of large research units.

The Publication Forum level is only a rough indicator, which does not take into consideration the special characteristics of each field of science or the more specialised or new publication channels. An individual publication in a specific publication channel can represent higher or lower level of quality, impact or significance than the level based on the average quality and impact of the publication channel. 

Limitations / challenges of the Publication Forum classification

• Due to the level quotas, levels 2 and 3 cannot contain all publication channels that are appreciated by the scientific community.
• Level 1 is especially extensive, which means that there can be a significant difference in the quality of the publications.
• Level 0 publication channels may publish duly peer-reviewed articles and books (e.g. universities’ own publication channels or publication channels just starting their operation) or they may be so-called predatory journals that are considered poor in quality.
• Publication channels placed in level 0 need to be evaluated also through other means, for example by using the Cabells Blacklist service.

The JUFO levels 0 and 1 are updated annually whereas the levels 2 and 3 are updated once every four years. Due to this, the JUFO level may change during the publication process. 

In addition to the Publication Forum classifications, responsible research evaluation should also use other publication channel and publication-specific metrics and consider the differences and characteristics of various fields of science. For further information, please visit the Evaluation Based on Scientific Publications guide.