Original Research

Online Research Output Submission System as a mechanism to influence publication citations: A pilot study

Reetha Nundulall

Received: 21 Sept. 2016; Accepted: 26 Apr. 2017; Published: 31 July 2017

Copyright: © 2017. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: Higher Education Institutions (HEIs) need to ensure that the education provided meets the student’s and employer’s requirements, for today and the future. However, in addition to the challenges of teaching and learning, internationalisation, globalisation and world university rankings are rearing their heads thus increasing the demands made on many HEIs.

Objective: One of the ways in which HEIs can make their mark is through world university rankings. This may be achieved by exposing more information on new and innovative research knowledge to the broader community in the global market via research publications that attract citations on open access platforms, hence influencing the university’s ranking. For this purpose and intent, a ‘simple’ and ‘easy-to-use’ online web tool was developed at a HEI. The aim was to have research publications submitted via the Online Research Output Submission System (OROSS) tool, screened and deposited in the institution’s open access database.

Method: Training was provided to the relevant participants and a survey was conducted to ascertain the participants’ perceptions about the utilisation of the OROSS tool and the training provided.

Conclusion: This article reflects on the pilot phase of a longitudinal study. Results of an evaluation conducted by the researcher of the OROSS application from a user perspective (process) are highlighted. In general, users rated OROSS favourably in terms of it being a useful, simple and easy-to-use web-based tool. The findings of this study may assist University of Johannesburg’s executive management in deciding the fate of the OROSS tool for future use.

Introduction

‘Higher education is rapidly growing and becoming a veritable global sector in its own right. That means challenges for educators, students and policy makers’ (Yelland 2011:1).

Quality teaching in higher education matters for students learning outcomes. However, this presents higher education institutions (HEIs) with a range of challenges at a time when the higher education sector is coming under pressure from many different directions. HEIs need to ensure that the education provided meets the student’s and employer’s requirements, for today and the future. It is further elaborated that HEIs are complex organisations where the vision and strategy need to be well-aligned with bottom-up practices and innovations in teaching and learning (Henard & Roseveare 2012:3). Likewise, Eid (2014:1) is also of the view that higher education plays an essential role in society by creating new knowledge, transmitting it to students and fostering innovation.

However, in addition to the challenges of teaching and learning, internationalisation, globalisation and world university rankings are rearing their heads thus increasing the demands made on many HEIs.

According to Christopherson, Garretsen and Martin (2008:343), ‘globalisation’ has become increasingly prominent since the 1990s. It has become a feature of economic, social and political discourse, not just within the academic community but also in the popular press and in the world of policy-making. However, views tend to differ as to what this means and whether it is a trend for good or ill. In response to the issue of good and evil of globalisation, De Wit (2012:1) is of the view that there is a strong inclination to identify globalisation in higher education as ‘evil’ and internationalisation as ‘good’ although the reality is more complex. Similarly, the Centre for Educational Research and Innovation (2009:23) is of the opinion that ‘globalisation and internationalisation in higher education are potentially conflicting, while at the same time interactive and mutually generative’.

The differences and similarities between globalisation and internationalisation of higher education have been intensely debated over the past years with questions such as

Is globalisation more an expression of the commercialisation of international higher education (education as a tradeable commodity) and internationalisation more the traditional concept of cooperation and exchange (higher education as a public good)? (De Wit 2012:1).

Blessinger (2015:1) is of the opinion that more internationalisation leads to more globalisation and that it can also be viewed as the integration of globalisation into the tripartite mission, that is, teaching, research and service of the university.

The view of Van Vught and Magnificus (2004:3) is that in higher education, the term ‘internationalisation’ is used to identify certain internal changes in HEIs, especially the integration of an international dimension into the functions of teaching, research and societal service. Internationalisation can also be interpreted as a process that can be shaped and influenced by HEIs themselves. ‘Globalisation’, on the other hand, is mostly seen as an external macro socio-economic process that cannot be influenced at the level of HEIs.

One of the ways in which HEIs can make their mark is through world university rankings. This may be achieved by exposing more information on new and innovative research knowledge to the broader community in the global market via research publications that attract citations on open access (OA) platforms, hence influencing the university’s ranking. According to Jobbins (2014:1), the European University Association published a significant piece of research that highlighted one of the unintended consequences of university rankings. It validated the activities of national and international ranking agencies by recognising the role they played in the process of the continual improvement of higher education. However, Collyer (2013:257) argued that current debates about university ranking systems focussed on the need for improvement to ensure these measure ‘output’ and ‘performance’ rather than simply prestige and status. Although efforts to expand sources of data for these systems to eliminate bias towards science and Anglo-American university sector are welcomed, there is still a need to challenge and problematise the system of ranking itself. Consideration should also be given to the broad variety of work undertaken in diverse disciplines across the university sector, including efforts towards scholarship, when universities are being evaluated.

Public HEIs are also accountable in the marketisation of its institutions, and one of the elements is to profile themselves as leading institutions. One of the ways that this can be done is to set up quality mechanisms. A most significant and recent mechanism that most institutions are subjecting themselves to is the world ranking systems. Although there is a range of world ranking systems, not all of them use the same criteria for judging all the institutions. Nevertheless, the idea of subjecting themselves on the world ranking system requires institutions to introspect, plan, promote, strategise in meeting those objectives and go beyond the criteria for evaluation. Research production is increasingly becoming a focal point in higher education transformation to influence ranking (African Union 2007:8).

However, this research production must be made visible and accessible. One of the ways this may be achieved is through OA of research publications which in turn may influence citations, thus a rippling effect on university rankings. According to Gearing (2011:3), ‘open access = accessibility = higher citations and research inquiries’. Similarly, Molecular Diversity Preservation International (MDPI) (1996–2016:1) concurred that the advantage of OA for authors is the high availability and visibility of articles. MDPI further elaborated that the higher citation impact of OA results from their high publicity and availability.

Citation can be defined as ‘a written reference to a specific work or portion of a work by a particular author that identifies the document in which the work can be located’ (Adriaanse & Rensleigh 2011:170).

In addition, De Groote (2015:2) stated that ‘citation analysis is the process whereby the impact or “quality” of an article is assessed by counting the number of times other authors mention it in their work’. Citation analysis is used to establish the impact that a particular author or article had by showing which other authors cited the work within their own papers.

This article reflects on the pilot phase of a longitudinal study on the utilisation of the Online Research Output Submission System (OROSS) tool as a mechanism to influence citations of published research outputs on the OA database.

According to Graziano and Raulin (2013:148), longitudinal study follows the same people over time to observe development changes, thus controlling for cohort effects. However, the drawback is that this design has the disadvantage of taking a long time to complete. Arising out of the pilot study, the researcher will explore the implementation and utilisation of the OROSS tool by the larger community of University of Johannesburg (UJ). This will need to be monitored and evaluated over a period of time.

Literature review

Open access

It is more than a decade since the Budapest Open Access Initiative coined the term OA and united a movement to free scholarly literature from barriers. Incredible progress has been made in this time, and the momentum only seems to be increasing in recent year. OA is now considered to be inevitable, with one prediction estimating that it will be the dominant model for scholarly literature in the next decade (Dawson 2013:1).

According to Drahos and Braithwaite (2002:15), bulk of the intellectual property rights are owned by corporations instead of the initial creators. The corporations acquire intellectual property portfolios which end up in patent portfolios where citizens have to pay for the same knowledge again. Drahos and Braithwaite further added that the recycling of public knowledge for private reward is also occurring in the educational sector. Copyright owners uplift university-generated, publicly funded research into journals or databases and then charge universities and students for the use of them, incurring exorbitant costs. Hence, countries behind the development of the intellectual property and copyright systems are the major beneficiaries, with developing countries being the net importers of knowledge (Drahos & Braithwaite 2002:11).

However, the global shift towards making research findings available free of charge for readers, so-called OA, was confirmed in a study funded by the European Commission. The new research suggests that OA is reaching the tipping point, with around 50% of scientific papers published in 2011 is now available for free (European Commission 2013a:1).

The European Union Commission also announced new policies both for OA to publications and for access to data arising from research funded under Horizon 2020, the successor to Framework Programme 7, which will have come into effect in 2014 (Finch Report 2013:2). Similarly, in South Africa, in a statement released by the National Research Foundation (NRF) that as from 01 March 2015, authors of research papers generated from research either fully or partially funded by NRF are required to deposit their final peer-reviewed manuscripts that have been accepted by the journals to the administering institutional repository with an embargo of no more than 12 months. The NRF further requires its stakeholder community to actively seek collaboration with the international scientific community to facilitate the OA of publications generated from publicly funded research across the world (Sinha 2015:1). For Africa, OA allows those who have been largely silent and invisible contributors to global research production to express themselves freely (Botman 2012:3).

Open access in higher education

According to Botman (2012:2), ‘knowledge production is important because it drives development, and open access accelerates that drive’.

Open access may be defined as the practice of providing online access to scientific information that is free of charge to the end user and that is reusable. In the context of research and innovation, ‘scientific information’ can refer to (I) peer-reviewed scientific research articles (published in scholarly journals) or (II) research data (data underlying publications, curated data or raw data). There are two main routes towards OA to publications, one of which is ‘self-archiving’. This means that the published article or the final peer-reviewed manuscript is archived by the author or a representative in an online repository before, alongside or after its publication. Repository software usually allow authors to delay access to the article (embargo period) (European Commission 2013b:2).

The OA philosophy is transforming higher education. From the use of social media to engage students to tools designed to facilitate record keeping in higher education, it would seem the academic revolution will be digitised. Arguably no other aspects of digital hold the promise of the OA philosophy and open educational resources (OER). It seems that the benefits of OA in higher education could go beyond teaching and research. A report published by JISC (Joint Information Systems Committee, a United Kingdom not-for-profit company), showed that the private sector also benefitted from OA in higher education (The Guardian 2014:1).

A similar view is expressed by Hall (2014:1) who stated that universities are ‘digital machines’ and for the future research the need for openness is far more than a convenience. The rise in volume and rate of production of online publications and digital data sets has now outgrown the limits of conventional research methods, and it is changing the ways in which new knowledge is created. Without openness across global digital networks, it is doubtful that large and complex problems can be solved.

There are many arguments in favour of new OA publishing models providing cost-efficient methods for disseminating research findings, eradicating excess profits by publishers and massively widening the readership of scholarly works (Gatti 2014:1). There is even the boycotting of some academic publishers by Professor Randy Schekman who collected a Nobel Prize for physiology and medicine in December 2013. Professor Schekman boycotted three ‘luxury’ scientific journals stating that their decisions to publish work, or not, are made according to how fashionable it is, rather than its scientific merit, and pursuing their own agenda to publish work that will be cited (Livermore 2014:1). Likewise, Boswell (2014:2) stated that South African universities pay thrice for the privilege of publication – once for the cost of research, a second time for the page fees and a third time for institutional access to journal. Boswell further stated that it may be time for South African universities to pursue an international agreement that secures the publications’ income of universities and academics. If they do not, then publishing houses will continue to tell stories about printing, distribution, marketing and administration costs.

However, Brown (2014:1–2) argued to the contrary. On pondering over many accusations made of publishers raking in profits while adding little value, he is of the opinion that publishers are doing a great deal to help move science forward. He stated that although the online space has afforded benefits to all, publishers have been presented with new challenges by this medium. Many assume that publishers’ costs have declined because of non-printing and distribution of physical copies but there are important new functions requiring significant expenditures. Some of these are the need for strong, skilled editors to ensure that research can be universally understood, to recognise emerging fields and create new journals, and to build and maintain the brands and reputation of journals; recruitment and management of editorial review board; coordination of peer review to ensure integrity of scholarly record; and developing new ways for students, researchers and librarians to find and use content via metadata, XML generation, tagging and a host of other tools. This is concurred by Osborne (2014:2) who stated that there is no such thing as free access to academic research and for those who wish to have access, there is an admission cost. He is of the view that UK scholars who are obliged to publish in Gold OA journals will end up publishing in journals that are less international and, although all that access to them is cost-free, are in fact less accessed. UK research published via this medium will end up being ignored.

The argument by Brown regarding the important role that publishing houses play to ensure quality and integrity in research may be considered as valid in light of a new challenge that is now impacting on OA publishing, that is, the threat of predatory publishers and fake or hijacked journals.

The publishing of scholarly journals has undergone radical transformation because of the emergence of Internet.

According to Dadkhan and Maliszewski (2015:281), hijacked journals are launched by fraudulent cyber entities for financial gain by using names and ISSNs of reputable journals and cheat researchers. They publish papers by receiving publication charges similar to those of OA journals, but they are not authentic. Gunaydin and Dogan (2015:94) had similarly stated that recently scientists have been the targets for cybercrime in different ways. Hijacked or fake journals and predatory journals have emerged and many scientists have been victimised by these journals. Authors and readers are deceived by being charged for services that are not provided, like peer review or editorial review. The journals also do not follow traditional standards for the acceptance of articles published.

Likewise, Beall (2012:179) highlighted the signs of predatory publishers where they:

• set up websites that closely resemble those of legitimate online publishers
• publish questionable journals of low quality
• are dishonest and lack transparency
• solicit manuscripts from researchers but fail to mention the required author fee, which is revealed after the paper is accepted and published. The author is obliged to pay as they are often asked to sign over their copyright as part of the submission process.

Beall was of the opinion that the publishers are not totally responsible for this. The scientists who are taking the unethical shortcuts and paying for the publication of plagiarised or self-plagiarised work are also to be blamed. These unethical scientists gaming the system are earning tenure and promotion at the expense of the honest. This is concurred by Gunaydin and Dogan (2015:96) who stated that scientists are also to blame as by sending their work to predatory journals are in fact supporting the system and should resist the temptation of publishing their low quality work fast and with ease. However, Gunaydin and Dogan (2015:94) argued that because many scientists are under pressure to publish hence, they are ‘forced’ to publish to gain promotion, reputation, pay rise and so on.

The pressure on academics to publish in scientific journals is also highlighted by Singh and Remenyi (2016:54) who stated that the term publish or perish which originated in the United States of America in the 1930s has progressively spread throughout academe around the world in subsequent decades. Traditionally, to publish in a scientific journal is still a substantial challenge for most academics, whereby the research has to be relevant, rigorous and written in academic language. Added to this reviewers’ feedback could result in considerable amount of additional work taking a long time to complete. There is also the issue of some leading journals having a backlog in their publications and that it may take a year or longer after the paper has been accepted before it appears in print.

The researcher is of the opinion that perhaps these may be some of the contributing factors tempting some academics to look for a ‘quick solution’ by publishing in these journals. The threats in publishing in these journals could result in poor quality work being published, and in large volumes, researchers’ reputation may be comprised. This could also impact negatively on their affiliated institutions, especially if these HEIs are subjecting themselves to the world university ranking system.

Jalalian and Mahboobi (2014:394) are of the opinion that fake publishers and impact factors have created an urgent need to evaluate the methods that are currently used to assess academic research. As a long-term strategy, it is suggested to move from quantitative methods to qualitative approaches to assessing the quality of academic research to protect academia from all the obvious misconduct of fake publishers, hijacked journals, and the predatory, non-reviewed, low quality publications that are great threats to the validity and integrity to science.

Whatever the challenges and arguments for and against OA that may continue, ultimately the researchers must enjoy the academic freedom to choose their preferred channels of publications (Rice 2014:1).

Research problem

Challenges are experienced by researchers to make timeous submission of accredited research publications for the Department of Higher Education and Training (DHET) subsidy and accredited and non-accredited publications for depositing into the institutional repository for citation purpose.

In addition to their teaching and learning and community engagement roles, the academics are expected to do research and publish their work. Research publications are critical. Visibility of high quantity and quality of publications add to the stature of an institution, especially for those HEIs who are subjecting themselves to world university rankings. Saddled with these tasks, researchers have the administrative onus to submit these publications to the institution. Because it is the technology era, mechanisms should be in place to address these challenges.

The OROSS tool was developed to address the optimal way to manage and streamline the submission process. It is also to assist in ensuring that research publication information is managed effectively and efficiently, resulting in no financial and citation loss to the institution.

Development of the Online Research Output Submission System tool

University of Johannesburg, a comprehensive HEI in Gauteng, has a goal to be recognised globally and win a position in university rankings; planning, promoting and strategising is required to increase its visibility. The option of the ‘self-archiving’ route to deposit publications in OA in order to make research data visible and accessible, receive citations and improve the university’s ranking was explored. This resulted in the development of a new and simple tool for this purpose, hence the birth of the OROSS tool. This tool was developed by Professor Alan Amory of UJ’s Centre for Academic Technology (CAT), and the software is available under the General Public License (GPL). The purpose of this tool was to encourage researchers to submit all their research publications timeously without burdening them with too many administrative processes. The aim was to receive and screen as many publications as possible for depositing in the institutional repository without infringing any copyright. Hence, research information is being readily visible and accessible for referencing, influencing citations and thus a rippling effect on university rankings.

With the NRF’s mandate (2015) that:

from 1 March 2015, authors of research papers generated from research either partially or fully funded by NRF, when submitting and publishing in academic journals, should deposit their final peer-reviewed manuscripts that have been accepted, to the administering institution repository with an embargo period of no more than 12 months. (p. 1)

further supported the need for the OROSS tool.

According to Towert (2015),

OROSS is not intended to be a database to store research output submission; it merely routes what has been captured within the faculties to the library for archiving in the institutional repository (UJDSpace) for web visibility/exposure. Research outputs are also routed to the Faculty Coordinators (FCs) responsible for managing research output in the faculties to screen and capture on the Research Information Management System (RIMS) for the Department of Higher Education and Training (DHET) subsidy purpose. (p. 1)

University of Johannesburg’s institutional repository (UJDSpace) was started in the late 1990s. UJ was one of the first universities in Africa to implement the new software called DSpace. Many breakthroughs were achieved, one of which was the loading of theses and dissertations electronically. At the beginning of 2016, migration of the institutional repository to a new software started in order to improve the structure. This was successfully achieved by the end of April 2016. The institutional repository is no longer called UJDSpace but UJ Institutional Repository (UJIR) (University of Johannesburg 2016:11).

The need to drive online visibility is concurred by Louw (2014:1–2) who cited that researchers at cyber metrics use web content, web visibility and web impact to determine ranking. The number of links and pages on the site domain; files in online formats such as pdf and doc; articles, papers and citations in the academic domain; as well as articles published in high impact international journals all contribute to ranking. after University of Cape Town (UCT) recently launched the OpenUCT institutional repository, providing a platform for staff to share their research, teaching and learning content with the world.

In another briefing paper written by Swan (2017) for Open Access Scholarly Information Sourcebook (OASIS) is that at the beginning of 2009, there was over 1300 repositories around the world, and it has been growing at an average rate of one per day over the last 3 years. Swan further highlighted some of the advantages of an institutional repository as follows:

• opens up the university’s outputs to the world hence maximising visibility and impact of these outputs
• showcases the university to prospective staff, students and other stakeholders
• enables and encourages interdisciplinary approaches to research
• supports student endeavours, providing access to theses and dissertations and a location for the development of e-portfolios.

One of the aims of developing OROSS is to allow the researcher to capture his or her outputs in a quick and simple way. It is also to avoid undue administrative burden on the researcher’s teaching, learning and research work load. Once captured, the publication will be immediately available to the Library department for screening and uploading on OA. At the same time, the faculties will receive it for uploading it on RIMS for DHET subsidy purpose, as illustrated in the ‘workflow’ (Figure 1).

FIGURE 1: Online Research Output Submission System work flow.

Implementation of Online Research Output Submission System: A pilot study

The pilot phase is part of a longitudinal study, as illustrated in Figure 2.

FIGURE 2: Online Research Output Submission System as a longitudinal study.

As a pilot study, OROSS was formally introduced in July 2014 in one of the nine faculties as requested by the Dean. Five out of fifteen departments volunteered to participate.

According to Vogt and Johnson (2016:326), a pilot study is ‘a preliminary test or study to try out procedures and discover problems before the main study begins’. This enables researchers to make important corrections and adjustments. It is a research project’s ‘dress rehearsal’. In a pilot, the entire study with all its instruments and procedures is conducted in miniature (e.g. on a small sample). ‘By contrast, a *pretest, definition (b), is used to assess some part of an instrument or procedure’ (Vogt & Johnson 2016:326).

Utilisation of OROSS involved the completion of a few online steps and providing the relevant documentation as required. Several hands-on training sessions were provided by the Library and Information Services Department in conjunction with the Research and Innovation Division to the volunteers (researchers and administrative support staff). Submissions were then viewed, screened for copyrights and deposited in the UJ’s repository – that is, the OA database while at the same time these outputs were channelled to the faculty for DHET subsidy consideration.

The aim to pilot the utilisation of OROSS was to ascertain its influence on the timeframe submission on researchers to ‘self-archive’ their publications. An additional aim was to gauge the perception of the end users on OROSS and the value of the tool for future use.

Research methodology

According to Clough and Nutbrown (2012:21), the task of methodology ‘is to explain the particularity of the methods made for a given study’. The purpose of a methodology is to show not how such and such appeared to be the best method available for the given purposes of the study, but how and why this way of doing it was unavoidable – was required by – in the context and purpose of this particular enquiry. Methodology requires researchers to justify their particular research decisions from the outset to the conclusion of their enquiry.

Study type

For the purpose of this study, the quantitative research method using the inductive strategy was chosen, as illustrated in Figure 3.

FIGURE 3: Research methodology.

According to Neuman (2014):

in quantitative studies, measurement is a distinct step in the research process that occurs prior to data collection. Quantitative measurement has a special terminology and set of techniques because the goal is to precisely capture details of the empirical social world and express what we find in numbers. (p. 203)

Similarly, Zappia (2015:1) described quantitative research as where the emphasis is on objectivity and the use of statistics or data gathered through polls, questionnaires or surveys. The data gathered are then generalised across groups of people to explain a trend or phenomena. Zappia further elaborated that this method can avoid personal bias in studying the research problem in the social sciences.

Inductive reasoning or strategy is moving from specific observations to broader generalisations and theories. It starts with specific observations and measures, then detecting patterns and regularities, formulate some tentative hypotheses that can be explored, and finally end up developing some general conclusions or theories as indicated in Figure 4 (Trochim & Donnelly 2008:17).

FIGURE 4: A schematic representation of inductive reasoning.

A survey via a questionnaire was chosen to collect data. Key (1997:1) defined a questionnaire as a means of eliciting the feelings, beliefs, experiences, perceptions or attitudes of some sample of individuals. As a data collecting instrument, it could be structured or unstructured.

A semi-structured questionnaire was designed to obtain the perceptions, attitudes and preferences of the tool from the volunteers trained on OROSS, as well as the impact of the training provided. This method was a quick and easy means to reach the trained group in a short space of time.

Pre-testing of the instrument

Prior to administering the survey, a trial run was done to test the instrument. The purpose of this was to test the questionnaire for clarity, ascertain if there were any problems, and whether it was simple and easy to understand. Suggestions were received, reviewed and incorporated.

The final questionnaire was then administered as an online survey with the assistance from the UJ’s Statkon Department. The URL link was emailed to the 41 staff members that made up the total sample size. With a slow response to the questionnaire by the deadline date, the researcher followed up with telephone calls to those staff members who were contactable. It was established that although the names of some of the staff members on the list did belong to the specific departments, they did not receive training. In two cases, the use of OROSS was not applicable to them, whereas in another two cases one person was no longer at the university while the other was overseas. The final target audience was 34 out of the 41 who did receive the training. Seventeen (50%) of the target audience responded to the questionnaire.

Ethical considerations

To conduct the online survey, an email was sent to the Dean of the pilot faculty who agreed via email dated 17 February 2015 and was also referred to the Vice-Dean: Research of the faculty for further liaison. Suggestion on data analysis and approval was provided by the Vice-Dean (Research) via email dated 22 July 2016.

Evaluation of the Online Research Output Submission System tool

Findings of the study

Data analysis

Quantitative data: For the purpose of this study, a survey via a questionnaire was chosen as one of the instruments to collect data. The questionnaire was designed to obtain feedback from the respondents on the training provided on the utilisation of OROSS and its user-friendliness.

The analysis of the data gathered via the questionnaires are revealed in Parts 1–5.

Part 1: Biographic data: A 50% return was received from respondents in the five departments. The only purpose of obtaining this data was to communicate with those respondents who may have indicated that they had experienced challenges when utilising OROSS. Data on race and gender were not relevant for this study.

Part 2: Evaluation of the training to use Online Research Output Submission System: This section was to ascertain feedback on the training provided. Figure 5 indicates that 50% or more respondents from four out of five departments, that is, Departments A, B, C and D agreed that their training requirements were met while Departments C and D (34%) strongly agreed. With respondents in departments choosing to remain neutral, in particular, Department E (100%) will provide the researcher an opportunity to do a follow-up to address the feedback provided.

FIGURE 5: Online Research Output Submission System training met my requirements.

Part 3: Evaluation of the Online Research Output Submission System tool: The purpose of this section was to obtain the respondents’ perceptions about the OROSS tool. According to Singh (2013:5), ‘website usability plays a vital role in the success of a website. Good usability helps to provide a seamless experience for users and improve the chance of successes’.

Figure 6 shows that more than 50% of the respondents agreed or strongly agreed that the OROSS app is user-friendly with Department E having a 100% agreement. The researcher is of the opinion that there is no cause for concern regarding those departments that remained neutral as the reasons provided did not make a significant impact on the overall data analysis, for example, ‘haven’t used it yet’ and ‘no outputs [to capture]’. Attempts will be made to address the issues raised by those respondents who have disagreed or strongly disagreed, that is, 34% in Department C and 20% in Department D.

FIGURE 6: Online Research Output Submission System is user-friendly.

Figure 7 indicates that more than 50% of the respondents agreed or strongly agreed that OROSS is a useful tool with Departments A and E having a 100% consensus. Reasons provided by those respondents who remained neutral did not make a significant impact on the overall data analysis, for example, ‘haven’t used it yet’ and ‘no output [to capture]’.

FIGURE 7: Online Research Output Submission System is a useful tool.

Figure 8 shows that more than 50% of the respondents agreed or strongly agreed that the OROSS has achieved the Vice Chancellor’s directive for a simple, easy-to-use web-based tool for researchers to submit their research online for subsidy and archival with Department E having a 100% of strongly agreeing. The researcher will attempt to address the issues raised by respondents who opted to be neutral.

FIGURE 8: Online Research Output Submission System is a simple, easy-to-use web-based tool.

Part 4: Conceptual outcome: This section was to ascertain the theoretic outcome of the OROSS training. Figure 9 indicates that respondents from all five departments agreed at varying percentage that the OROSS training provided them with a better insight regarding manuscripts and the publisher version PDF copy policies and institutional archiving in UJ’s OA database. An overwhelming response of 100% was received from Departments A and E, whereas 83% from Department C. A fair response of 50% and 40% was received from Departments B and D, respectively. Feedback from respondents who remained neutral did not make a significant impact on the overall data analysis.

FIGURE 9: Better insight.

Part 5: Practical outcomes: This section was to gauge the impact of the training on the respondents and whether the tool is recommended for future use. According to Feghali, Zbib and Hallal (2011:84), the user attitude is important as it contributes to the user’s intention to use the system.

Figure 10 indicates that there was an overwhelming response from all five departments where it was agreed or strongly agreed that they were confident to ‘self-load’ their outputs after the training. With the tool being user-friendly (Figure 6) and simple to use (Figure 8), a confident researcher can upload publications with minimal effort.

FIGURE 10: Confidence to ‘self-upload’ output.

According to Feghali et al. (2011:84), the satisfied users of a website may recommend it to others.

Figure 11 shows that a fair percentage of respondents favoured OROSS be implemented on a permanent basis. However, there was a greater lean towards it being recommended for implementation on an optional basis.

FIGURE 11: Utilisation of Online Research Output Submission System.

Overall summary of the analysis

Overall analysis of the data leans towards a positive attitude on the utilisation of the OROSS tool, in particular, respondents found that OROSS was a ‘useful tool’ (Figure 7) and that it was also ‘a simple and easy web-based tool’ (Figure 8). A very significant finding is that an overwhelming percentage of respondents from all five departments agreed that the OROSS training provided them with a better insight regarding manuscripts and the publisher version PDF copy policies and institutional archiving in UJ’s OA database (Figure 9). It is important for researchers to understand what is required, hence impacting on the ‘buy-in’ of the utilisation of the tool.

Feedback received from those respondents who have ‘disagreed’, ‘totally disagreed’ or opted to remain ‘neutral’ did not make a significant impact on the overall data analysis. However, this has provided the basis on which the tool may be reviewed for future enhancement and utilisation.

Review of the Online Research Output Submission System tool in its entirety

Feedback via the survey conducted indicated a positive uptake on the utilisation of OROSS. Data extracted in January 2016 (Macanda 2016) indicated that there had been a marked increase in publication volumes deposited in the UJ repository (Figure 12).

FIGURE 12: Volume of publications uploaded in the open access database.

Figure 12 indicates that as on December 2015, there had been a marked increase in the volume of publications uploaded onto UJ’s OA database in comparison with 2014, that is, from 499 to 698 publications (39.9% increase).

Development of the tool commenced in 2013. Once operational, it was left at the faculties’ discretion to utilise this tool. Prior to the pilot study in July 2014, the submission of publications in UJ’s OA database from January to June 2014 was 207, while for the same period in 2015 the volume of submission increased to 314, that is, by 51.7%.

The overall percentage increase of 39.9% of publications in UJ’s OA database as on December 2015 is encouraging, especially for citation purposes. However, a shortfall in this process was noted. Because of there being no measuring mechanism in the OROSS tool, it cannot be confirmed that all the outputs deposited in the repository were received via the OROSS submission.

The researcher will explore possible solutions to address this shortcoming which will form another part of this longitudinal study.

Conclusion

The research arena has been steadily evolving and HEIs are continuously subjected to new challenges and demands nationally and globally.

One of the major challenges is that HEIs must increase their income apart from the dwindling funds received from government. It has now become crucial for HEIs to drive the process to increase research output publications in order to increase their subsidies from the DHET. To add to the funding challenges, focus on globalisation, internationalisation and world university ranking and recognition has gained momentum. Hence, universities may focus on ranking and recognition to get the attention of their national and global stakeholders.

One of the ways to address some of these challenges is to get research publications in OA. This will assist universities and academics to get their research knowledge to a wider audience to make their work more relevant and have greater impact. It will also attract citations that will have a ripple effect on the university’s ranking. The OA philosophy is transforming higher education as universities are now considered ‘digital machines’. The rise in volume and rate of production of online publications has now outgrown the limits of conventional researcher methods. OA publishing models are considered as a cost-efficient method of disseminating research findings, eradicating excess profits by publishers and massively widening the readership of scholarly works.

University of Johannesburg is not immune to these challenges. Research is one of the critical key strategic goals of the institution and huge investments in various ways are made to promote this agenda. Hence, in addition to an institution achieving its goals to increase publications for subsidy purposes and promote research productivity, citations of publications play a key role in contributing to the institution’s position in the world university rankings, globalisation and internationalisation.

One of the ways in which this may be achieved is the vigorous promotion of the utilisation of the OROSS tool. With a very positive feedback indicated in Figure 10 where an overwhelming response from all five departments agreed or strongly agreed that they were confident to ‘self-load’ their outputs after the training, the tool being user-friendly (Figure 6) and simple to use (Figure 8), a confident researcher can upload publications with minimal effort. This is advantageous as publications can be submitted speedily and timeously to the Library department for screening and depositing in the institutional repository, hence immediate accessibility to the wider market for citation.

The impact of citations influencing a university’s ranking is highlighted in the latest 2015 QS World University of Rankings. Stellenbosch University (SUN) outranked the University of Witwatersrand (Wits) by becoming South Africa’s second best university after UCT. SUN moved from 390th in 2014 to 302nd in 2015, and the reason attributed to this big climb has been credited to research output and impact measured – not only in terms of the number of papers but also citations (Staff Writer 2015:1–2).

Prior to the development of the OROSS tool, the depositing of research publications in the institutional repository was done via manual process. The documents loaded in a particular year were for the period n-1 (n=current year, 1=previous year), hence a delay in having documents readily available for public viewing, thus impacting on citations. With the implementation of the OROSS tool, current publications are made available to the Library department to deposit in the institutional repository for citation.

Figure 12 shows a marked increase in the volume of publications uploaded in the OA database. The assumption is that this has resulted from the utilisation of the OROSS tool. However, unfortunately because of a minor flaw in the development of the tool, there was no solid evidence to confirm that majority of the publications that were deposited in the repository were submitted via the OROSS tool.

An alternate way that researchers may capture research outputs is via the Research Information Management System (RIMS). This is a system purchased by the Department of Science and Technology with other consortium members being the DHET, NRF and some other South African HEIs, including UJ. This system is utilised by the UJ’s Research and Innovation Division to make the final submission of accredited research publications to the DHET for subsidy purposes. There is also a provision in the system to upload the manuscript and the final officially published outputs. The Library department may then access the documentation via RIMS for screening and depositing in the UJ repository. However, the disadvantage of submitting via this system is that currently non-accredited outputs (i.e. publications not submitted for DHET subsidy) are not captured. Hence, the Library department will not be aware of these publications.

RIMS is a complex system requiring much training in capturing outputs as detailed information is required. This will be a time-consuming process that may lead to a possibility of researcher frustration and time delay in getting the research outputs in the UJ repository.

Hence, the development of OROSS with one of the aims being to allow the researcher to capture his or her outputs in a quick and simple way.

With the positive feedback received regarding the OROSS tool, the way forward is that the researcher will develop strategies for the promotion on the utilisation of the tool first to the entire academic community in the pilot faculty and then to the entire UJ community with the optimism of assisting UJ achieving a similar outcome as SUN’s success rate in rankings.

After the post-pilot study, it can be assumed that the OROSS tool has partially solved the issues of some of the academics regarding the submission process. The interest in the utilisation of OROSS has spread to other faculties as the researcher has been providing training on an ad hoc basis.

This study has made a contribution to new research through the development of a new tool for the management research publication information, especially for the institutional repository for citation purposes, in a quick and simple way.

For further studies, the researcher will explore possible solutions to address all the shortcomings of the tool. Once the tool has been revised, tested and re-implemented, it will be re-assessed. The possibility of capturing dissertations via OROSS will also then be given attention.

The findings may assist UJ’s executive management in deciding the fate of the OROSS tool for future use. The utilisation of this tool could either be made compulsory for all publishing researchers or defunct or left for use on an optional basis. It may also be of use to other HEIs on whether to adopt the strategy of utilising such a tool.

Acknowledgements

Thanks to Dr Richard Devey, UJ Statkon, for assisting in the questionnaire design and guidance on data analysis.

The author declares that he has no financial or personal relationships that may have inappropriately influenced him in writing this article.

References