http://espace.library.uq.edu.au/ The University of Queensland en Fez http://blogs.law.harvard.edu/tech/rss http://espace.library.uq.edu.au/view/UQ:177904 The first multi-indicator ranking of world universities, “Academic Ranking of World Universities” (ARWU), was published by the Graduate School of Education (formerly Institute of Higher Education) of Shanghai Jiao Tong University in June 2003. Although the initial purpose of ARWU was to determine the global standing of top Chinese universities, it has been attracting worldwide attention from governments, universities, media, and the public. Since then, about 10 global university rankings have been published. The impact of global university rankings has also been increasing. In this presentation, the history, methodologies of ARWU will be introduced and its features and limitations will be discussed. The positions of Asia Pacific universities and their trends will be analysed according to ARWU. The academic performance of Asia Pacific universities in various indicators will be investigated. Recommendations will be provided on how to use global university rankings and how to improve the performance of Asia Pacific universities. 2009-05-26T00:00:00Z Liu, Nian Cai og Cheng, Ying http://espace.library.uq.edu.au/view/UQ:241924 2011-06-09T00:00:00Z Batterham, Robin J. http://espace.library.uq.edu.au/view/UQ:177915 Bibliometrics are increasingly being used in national research assessment exercises. This article reports on measures and applications of bibliometrics within a large Australian School of Population Health, the School of Population Health at The University of Queensland. As a result of the production of annual staff research assessment reports, the research culture of the School has greatly strengthened and consideration of more metrics to include in future assessments is underway for future staff performance appraisals. This presentation reports on measures and application of bibliometrics that were introduced in 2003 in an attempt to enhance the research culture of the School and to provide clear benchmarks for staff at different academic levels of appointment, with the information being used extensively in annual staff performance appraisals. 2009-05-26T00:00:00Z Lopez, Alan og Penn, Danielle http://espace.library.uq.edu.au/view/UQ:177918 There have been a large number of studies carried out in which scores in the United Kingdom's Research Assessment Exercise have been correlated against citation counts. Subjects that have received scrutiny include Civil Engineering, Psychology, Business and Management Studies, Library and Information Management, Genetics, Archaeology and Music. Somewhat surprisingly, the strongest correlations have been found in music. The paper will review these results and provide some explanations for the correlations. The paper will then go on to report on-going research which examines the use of h index and g index measures in the fields of Pharmacy, Library and Information Management and in Anthropology to see if they are correlated with RAE scores. This is believed to be the first time research has been carried out where the h index and its derivatives have been correlated against RAE scores. 2009-05-26T00:00:00Z Norris, Michael og Oppenheim, Charles http://espace.library.uq.edu.au/view/UQ:177913 One of the most common methods for assessing the performance of universities is bibliometric analysis based on fields of research. Most commonly for broad sectorwide studies, fields of research are defined by journal sets, usually utilising existing lists such as the Web of Science subject categories. However these journal sets do not always align well with Australia’s standard Fields of Research classification scheme – the scheme used for reporting most higher education data, and the scheme against which most policy analysts expect assessments to be made. In addition, many existing systems have considerable overlap between the journal sets for related fields, leaving analysts in a quandary about what it is exactly that is being assessing. Today’s presentation will provide an overview of some of the benefits for bibliometric analysis in Australia that have flowed from the construction of a completely new series of journal set. It will also present empirical evidence to demonstrate the effect of these changes on citation benchmarks. 2009-05-26T00:00:00Z Butler, Linda http://espace.library.uq.edu.au/view/UQ:241904 2011-06-09T00:00:00Z Pankhurst, Ned http://espace.library.uq.edu.au/view/UQ:177903 Research assessment exercises around the world use different units of assessment for assessing research outputs. These include department, individual, research group and field of research. It is recognised that each of these approaches have their own intrinsic strengths and weaknesses. In the build up to the Research Quality Framework (RQF) in Australia, many universities undertook advanced modelling of research outputs in order to determine the most competitive research groups for submission. Now, under the new Excellence in Research for Australia initiative (ERA), research performance will be evaluated by the field of research as defined by the Australian and New Zealand Standard Research Classification (ANZSRC). The new approach has raised some concern about the ability of universities to recompile research outputs from departments, centres and various multidisciplinary research groups. The presentation will review the various approaches internationally, with data presented to demonstrate the ERA methodology and the types of analyses universities will be able to undertake inhouse with the supplied data and benchmarks. 2009-05-26T00:00:00Z Calder, Andrew http://espace.library.uq.edu.au/view/UQ:241907 Steven Wooding will talk about the importance of evaluating the outcomes of medical research - the different roles evaluation can play: advocacy, accountability and analysis; alongside the challenges of understanding a process that may take 20 years to develop to fruition. He will discuss a number of recent studies he has been involved in including Project Retrosight - a three country, 29 case study evaluation of biomedical research in cardiovascular medicine over 15-20 years; the RAISS tool - a questionnaire that has an average completion time of under an hour yet collects information on 250 distinct impacts of biomedical research and 'What's it worth?' a study that came up with a number for the economic impact of medical research investment 2011-06-09T00:00:00Z Wooding, Steven http://espace.library.uq.edu.au/view/UQ:241906 2011-06-09T00:00:00Z Tijssen, Robert http://espace.library.uq.edu.au/view/UQ:177912 The globalisation of research means that universities are no longer content merely with national comparisons of research performance and standing. Universities that wish to compete for leading academic talent and access global research opportunities must develop an advanced understanding of their relative position within the global knowledge economy. This entails benchmarking performance at the level of the institution, academic elements and research centres, and of individual researchers - within broad disciplines and narrow subject areas. This presentation explores several methods used to benchmark performance, and discusses the issues associated with making global comparisons and the opportunities to employ more advanced techniques. 2009-05-26T00:00:00Z Sheil, Tony http://espace.library.uq.edu.au/view/UQ:241900 The developmental trajectory of building institutional research performance mirrors the trajectory of individual research careers. An increase in the volume of research activity precedes a more mature phase when work of high quality and impact is generated which attracts attention from international peers resulting in the opportunity to develop international collaborations, highly cited outputs and associated awards and public recognition. Research evaluation frameworks such as Excellence in Research for Australia (ERA) focus appropriately on measuring the ‘destination’ of a research trajectory- world class performance in disciplinary fields. As would be expected, in the most recent ERA exercise, there is a clear relationship between the age of Australian institutions and the number of disciplinary areas in those institutions assessed as being of world standard or above. In this context, it is important for institutions to determine whether they are ‘hitting above their weight’ and are on an upward trajectory in terms of research performance relative to their institutional age. Thus ERA outcomes, in combination with a dashboard of other research performance measures can inform institutions of the effectiveness of their research strategy. The major limitation of ERA, however, is the long shadow cast by the 6 year reference period – the length of the reference period necessarily limits the usefulness of the outcomes for those younger institutions which are rapidly growing their research capacity, activity and quality ‘in real time’. This is of particular relevance in Australia where the median age of universities is around 24 years and where the future of Australia’s research capacity will reside in the growth of high quality research in those institutions which are currently ‘market challengers'. 2011-06-09T00:00:00Z McMillen, Caroline http://espace.library.uq.edu.au/view/UQ:177923 The National Library has built a new service called People Australia that will allow users to discover the resources people and organisations create along with related contextual information. People Australia has the potential to benefit the Australian research community by creating profiles for researchers and research teams and persistently identifying them. This presentation will provide an overview of People Australia and detail how People Australia is being used to support the identification of researchers in the NicNames project. 2009-05-26T00:00:00Z Dewhurst, Basil http://espace.library.uq.edu.au/view/UQ:177906 High-quality scientific research, technological development and commercial innovations are the key to competitiveness and future prosperity of advanced knowledge-based economies. It is no surprise that many federal and local governments now expect their universities and research institutes to engage in “third mission” activities (alongside teaching and research) specifically devoted to transfer, utilisation and commercialisation (TUC) of their knowledge and skills base. Contrary to teaching and research, the rate of success and gains resulting from “third mission” activities are often difficult to identify and measure, let alone to reliably assess the performance of institutes. Significant positive effects and economic benefits may take many years to emerge. Comparative statistical data is often scarce. 2009-05-26T00:00:00Z Tijssen, Robert http://espace.library.uq.edu.au/view/UQ:241899 2011-06-09T00:00:00Z Lu, Max http://espace.library.uq.edu.au/view/UQ:177920 UNSW Library serves a research-intensive university. In 2006, the Library’s information services were restructured to enable innovation in research support services and strengthen the relationship between the Library and academic staff. This paper outlines the rationale for this change and describes the new services, including research impact, publication measurement and bibliometric programs. An account of the challenges and barriers to developing these services is provided, together with an evaluation of their success to date. 2009-05-26T00:00:00Z Wells, Andrew http://espace.library.uq.edu.au/view/UQ:241896 To assess impact of faculty or programs, there are several accepted metrics, including publication output and follow-on citations. While these measures are robust and can be normalized by field of research, other metrics are needed to capture important aspects of faculty scholarship, namely teaching, service, and instances of research or scholarship not published in journals. We will present an enterprise solution developed by Thomson Reuters to gather and manage service, teaching, and research information across the university. The success of faculty knowledge management systems depends on the strength of the data model, the ease of data entry, integration with both existing faculty workflows and university systems, and automation of profile maintenance. A demonstration of the user interface will be provided in the context of a discussion of the data model and institutional reporting use cases. 2011-06-09T00:00:00Z Haak, Laurel http://espace.library.uq.edu.au/view/UQ:177917 In late 2006 The University of Western Australia launched Socrates, an online application designed to draw data from key research information systems, in order for the University to prepare portfolios for the Research Quality Framework (RQF). Socrates also incorporates bibliographic and citation data from Thomson Reuters’ Web of Science (WoS) using the web services interface (API). This presentation focuses upon the utilisation of Thomson Reuters’ data within Socrates. The benefits of importing Thomson Reuters’ data, including reducing the workload associated with the annual HERDC publications collection, and using imported research tags to trace the level of publication within specific disciplines, are explored. The presentation also outlines the technical problems faced by Socrates with regards to matching citation data imported from the WOS to data from the UWA Publications Database. Overall, it is argued that by drawing data from the WOS, Socrates is able to provide a detailed analysis of UWA’s indexed publications at a university, organisational unit and individual level, thereby shedding significant light on the University’s research output performance. 2009-05-26T00:00:00Z Owens, R. og Mast, N. G. og Glance, D. G. og McEachern, D. http://espace.library.uq.edu.au/view/UQ:241902 For approximately five years, The University of Western Australia has used an online application called Socrates to draw data from key research information systems, including bibliographic and citation data from Thomson Reuters’ Web of Science (WOS) and Scopus, using the web services interface (API). Initially designed to respond to the Australian research evaluation exercise, the ERA, the evolution of Socrates has seen it being used for a variety of staff development and staff performance tasks. In this presentation I will overview Socrates and outline how its evolution has responded to the way it which it has been used by research staff and university leaders in developing staff performance. 2011-06-09T00:00:00Z Owens, Robyn http://espace.library.uq.edu.au/view/UQ:241895 What is the underpinning theory of research performance and how can one apply logic models to evaluate performance? What are the appropriate system inputs, outputs, impacts, and metrics? These questions will be addressed in reference to a research portfolio management system developed by the US National Institutes of Health (NIH). The NIH supports basic and applied biomedical research by funding grants and contracts. To measure the outcomes and impact of their programs, NIH staff conduct program evaluations and undertake targeted investigations of research portfolios. Recently, the NIH launched the electronic Scientific Portfolio Assistant (eSPA), a Web-based analytics system based on linked scientific databases. After describing the logic model upon which the eSPA system is based, we will describe practical applications of the system that show the relationship between policy and the inputs and outcomes of research. 2011-06-09T00:00:00Z Haak, Laurel http://espace.library.uq.edu.au/view/UQ:236733 2011-03-16T00:00:00Z Greenhill, Andrew og Blaney, Barry og Shipton, Warren og Pue, Aisak og Fletcher, Mary http://espace.library.uq.edu.au/view/UQ:241903 2011-06-09T00:00:00Z Drummond, Robyn og Wood, David og Sheil, Tony og Mast , Natalie http://espace.library.uq.edu.au/view/UQ:241898 2011-06-09T00:00:00Z Henderson, Tracey http://espace.library.uq.edu.au/view/UQ:177960 2009-05-28T00:00:00Z http://espace.library.uq.edu.au/view/UQ:177925 Historically, the collection and management of UQ’s publication data has involved manual data entry, with citation metrics retrieved from Thomson Reuters Web of Knowledge on an “as-needed” basis: a time-consuming and costly workflow. A partnership between Thomson Reuters and the UQ Library has demonstrated that the integration of Thomson Reuters ResearcherId with UQ’s publication data provides significant benefits, including increased efficiency and accuracy. Furthermore, the ability to integrate citation metrics with UQ publications data provides a rich source of information with many potential applications, including the measurement of research performance and assisting in the preparation of grant applications. 2009-05-26T00:00:00Z Thomas, Amberyn og Bennett, Andrew og Martlew, Andrew og Kortekaas, Christiaan og Guida, Reynold og Rotenberg, Ellen http://espace.library.uq.edu.au/view/UQ:177921 University of South Australia (UniSA) management wanted to know who is publishing, where and how often. Researchers wanted a list of their publications with citation counts. The Library wanted to provide an ongoing service that assisted the University to measure and assess its research output. Citation searching can be difficult, time consuming and tedious, complicated by incomplete or inaccurate citations. Busy researchers, while keen to know their current citation counts, are much less keen to do it themselves. Faced with these challenges the Library established a project to collect the citation counts for all our researchers’ publications. Searching for each publication's citation count is resource intensive but we discovered the added benefit of verifying each reference’s details. We then further built on this work by developing scripted solutions that automatically increment citation counts from the harvested alerts. This paper outlines how the Library is meeting the challenge of managing bibliometric and bibliographic data. In providing a service that helps the University answer some of the hard questions asked about measuring and assessing the University's research outputs, the Library is not only filling its repository with clean metadata, but also reinforcing the Library's reputation as metadata and bibliometric experts. 2009-05-26T00:00:00Z Sergeant, Kate http://espace.library.uq.edu.au/view/UQ:241908 Steven Wooding will describe the RAISS* tool - a low burden method of mapping the impacts of biomedical research across the research portfolio. The main body of RAISS is around 250 Yes/No questions that explore what impacts a piece of research has contributed to. By using high level questions to determine which detail questions are asked and by asking exclusively Yes/No questions the average completion time of the questionnaire is just over half an hour, with 90% of questionnaires completed in less than an hour. Despite this quick completion time the questionnaire still collects information across a huge range of impacts including on career development; collaboration within and outside academia; research capacity building, research tool production, dissemination in academic and non-academic contexts, impacts on health policy through a variety of routes; impacts on education and training of others and a wide variety of intervention, product development and public health advice outcomes. RAISS provides an overview of research impact and a basis for more detailed examination of particular research evaluation questions focusing on the 'why and how' of translation. We have also developed "Impact Arrays" as a way to represent the impacts of research and give an instant overview of the research portfolio and the ease of completion can allow tracking of impacts over time. 2011-06-09T00:00:00Z Wooding, Steven http://espace.library.uq.edu.au/view/UQ:241901 2011-06-09T00:00:00Z Noonan, Phillip http://espace.library.uq.edu.au/view/UQ:177480 In the past, the UK's Research Assessment Exercises have relied heavily on assessment by a panel of experts in the particular Unit of Assessment that the researchers being evaluated happened to work in. It was therefore quite a surprise when the then Chancellor of the Exchequer, Gordon Brown, announced that in future, research evaluations would be bibliometrics based. In this talk, the history of the RAE will be sketched out and the motives for changing to a bibliometrics approach will be explained. The current REF Pilot experiment will be explained and the bibliometric methods being used will be critically examined. The role of Evidence Ltd. and the fact that it is now owned by Thomson Reuters will be explored. The Keynote will end with speculation about how the future REF might be applied and with commentary on the validity of citation counting for analysing research quality. 2009-05-07T00:00:00Z Oppenheim, Charles http://espace.library.uq.edu.au/view/UQ:177924 The University Library holds a long-established place as a key part of academic research infrastructure. For centuries, the library has been responsible for acquiring and providing access to records of the world’s scholarship. The advent of electronic information resources has brought profound change to the idea of the university library. The curatorial and mediation functions of library staff with respect to formally published knowledge are increasingly marginalised. Librarians, however, have been educated in a body of professional knowledge that is central to many aspects of research and scholarship in an electronic world. Understanding the research process, mapping the published outputs of research and modelling citation patterns are all fundamental aspects of a research librarian’s armoury. In this paper the opportunity for librarians to be central players in the research valuation process will be set out, and a new vision for the future of the research library will be described. 2009-05-26T00:00:00Z Webster, Keith http://espace.library.uq.edu.au/view/UQ:177905 This presentation discusses the role of bibliometrics in monitoring the performance of the New Zealand tertiary education system. The Ministry of Education uses aggregated data from Thomson Reuters to monitor the research performance of the tertiary sector and individual providers. The bibliometric data is used to benchmark the performance of New Zealand universities and is also used to monitor the impact of the Performance-Based Research Fund (PBRF) – the research funding allocation system for New Zealand’s tertiary education organisations. The bibliometric data shows that the median relative academic impact of New Zealand indexed research sits between Australian Go8 universities and non-Go8 universities. It also suggests there has been a shift in the research performance of New Zealand universities since the PBRF was introduced in 2004. Since then, the share of world indexed publications by New Zealand university authors has increased along with their share of world citations. The operation of the PBRF is constantly being reviewed and the Ministry of Education will watch the increased use of bibliometrics in the United Kingdom and Australia with interest. 2009-05-26T00:00:00Z Smart, Warren http://espace.library.uq.edu.au/view/UQ:241897 2011-06-09T00:00:00Z Hattie, John http://espace.library.uq.edu.au/view/UQ:241905 Liz will offer an overview of the approach to research performance data capture, analysis, interpretation and exploitation at the University of Melbourne, touching on analysis at multiple levels: individual, institutional, and in-between. 2011-06-09T00:00:00Z Sonenberg, Liz