key: cord-0057901-j4px1q86 authors: Antipova, Tatiana title: Public Universities’ Performance Evaluation date: 2021-02-17 journal: Advances in Digital Science DOI: 10.1007/978-3-030-71782-7_12 sha: 91541e30c55453c9fe94e55f5f7ae2c048a5c84e doc_id: 57901 cord_uid: j4px1q86 Public University has state ownership and/or receives public funds through a federal/national or subnational government. As the budgetary entity, Public University spends budget money and every citizen demands greater understanding of where their tax money goes and spends. The purpose of this paper is to study performance evaluation of Public Universities complying with Public Sector laws/rules. This paper suggests that the construction process of performance measurement should be aligned with outputs and outcomes of budget policy. Performance evaluation of Public University as a whole organization is considered in the three aspects: design/creating, evaluating, and challenging/monitoring. Subsequently, author aims to investigate the latest achievement in evaluation, monitoring, and challenges for public universities’ KPI system. Texts and contents from different sources such as financial annual reports and budgetary entities’ websites provided insightful and findings. In addition, the research result lightens the most important drawbacks and challenges facing during the public universities performance measurement. Public University is a university that is in state ownership or receives public funds through a federal/national or subnational government, as opposed to a private university. So Public University is subject to the laws of the public sector and is a budgetary entity. As the budgetary entity Public University expenses budget money and every citizen demands greater understanding of where their tax money goes and spends [1] . An increasing social interest in effectiveness of resources management and in efficiency of operations in public management has triggered a change in perception of the roles and tasks of analysis in this group of institutions [2] . Such public organizations as public universities should be good citizens, fulfil social roles and tackle social problems, thus obtaining social recognition by engaging actively in dialogue with stakeholders. In this vein, public universities are responsible for the significant influence over a large population of future leaders. This influence is not limited to teaching and research, but it extends to include the need and ability to maintain its long-term performance better than its competitors. Public universities started gearing their key performance indicators (KPIs) towards social responsibility by voluntarily integrating social activities into their core policy and social performance measurements [3] . The public universities' performance can be measured by the quality, effectiveness, productivity, efficiency, innovation and quality of work life. Improving the performance of an educational institution is expected to generate good output, in which many preparations should be taken into account including enhancing the quality of facilities, infrastructure, education/teaching, research and social community service [4] . Performance evaluation in academia has traditionally been somewhat biased towards research indicators and it seems that this orientation is becoming even more profound. A growing number of reports document that universities striving for government funding, research grants and high rankings have adopted strategies including recruitment and reward systems that favour academics with top publications or the potential to secure such publications [5] . Indicators of CSR studies have been embedded with the existing CSR studies, for example, "CSR disclosure and organizational attraction", "motivation of CSR adoption", "internal CSR practice", "commitment of CSR education", "integration of CSR into teaching and research", and "standalone or embedded CSR subjects", were mainly implicated in practice, and have not been fully explained in the literature. In addition, to minimize the ongoing uncertainty of COVID-19, Public Universities should be collectively engaged with stakeholders for long-term value creation, and decision-making by governing bodies should focus on sustainable development as an integral part of society [3] . Achieving the quality of education is achieved through the existence of a mechanism that clarifies the policy that the institution should follow from the efficiency of administrative organization and the provision of high-level training systems for the educational and administrative staff. The quality of education is the totality of the attributes and characteristics that relate to the educational service and can meet the needs of students. [6] . Beyond recognizing the importance of openness, transparency, and quality, institutions and funders should work together to enable the establishment of local resources that assist and support researchers in fostering these values [7] . Performance evaluation of Public University as a whole organization is considered in three aspects: design/creating of KPI system, evaluating, and changing as shown in following parts of this work. There is a very large body of literature on key performance indicators system. Our search for "key performance indicators in public universities" in Taylor & Francis databases gave us 371,088 results on 06.01.2021. Despite the extensive literature on this topic, there are relatively few evidences that the contributions of governance dimensions to public sector performance actually determine the quality with which particular governance system function requires careful, systematic and replicate measurement. In implementing an analysis process, this study was aimed to provide sufficient transparency and reproducibility to allow for both replication, critique and alternative approaches to this analysis similarly to this analysis in works [6, 8-10, 12-15, 18, 22, 29, 30, 32] . In the case of agonistic, for example, this type of behavior can occur because the teacher tries to show anger on his face to control the class or because he cannot clearly express his ideas to make himself understood by the students. In the case of submitting, the teacher tries to keep their attention (pressure from the class) under control by means of humility and resignation, which at the time can be similar to sadness. The important thing is that it is possible to determine the emotional behavior of the teacher by the emotional vectors assigned to a time window, and to know what recommendations to provide to deal with a specific topic during the teaching-learning process and thereby feed a recommender system [10] . To evaluate the results of Public University activity there is needed to design Key Performance Indicators system. The current performance management system takes account of multi-dimensional and multi-layered performance objectives and pursues realization of various values including not only efficiency and effectiveness, which are important factors in management activities, but also equity and fairness of the public services. Thus, performance evaluation is conducted by using both qualitative and quantitative measures at the individual level as well as the organizational level in the aspect of both short and long periods of time. It is emphasized to construct a comprehensive system that encompasses strategic objectives and crisis management to provide the necessary public services to the people at all times [11] [12] [13] . Many researches consider KPI for evaluation of quality education/graduation, e.g. [28, 29] . In addition, there is a removal from the types of boredom and general frustration, as well as unjustified fatigue resulting from the constant pressure of quality requests in a disturbing manner, which is called emotional stress. The faculty member faces several requirements between the basic tasks of the teaching function, the quality requirements, and professional development on the one hand, as well as the general depletion of mental, muscular, and psychological capabilities. 88% of the study sample confirmed that, despite this, they are cooperating with colleagues in order to develop academic curricula and develop university work, despite bearing the intense pressure burdened with quality requirements. Many faculty members suffer from the lack of clarity in the university's mechanism to take care of the opinions and observations of employees, as well as 76% suffer from severe pressure because of direct interaction [6] . But this study aims to evaluate Public university as a whole organization as a part of budget/government system in the other words as public sector subject. In this aspect some of scientists offer four groups of indicators for public universities: 1) personnel, 2) material-technical, 3) economic, and 4) pedagogical content to comprehensively inspect the process from the perspective of an organization activity [8] . As the result of our previous researches [1, 2, 4, 5] , we stated that the most important KPI of Universities' activity as a whole might be divided on three groups of indicators: 1) indicators that determine the financial position of the universities, 2) indicators related to the publications activity of teaching staff/faculties, 3) quantitative indicators counted staff, student, area of buildings and amount of tax. Each group of indicators consists of several units as shows in Table 1 . [2] show that used factor analysis for ranking is behind the constellations of performance measures. Our ranking was compared with the data available on Internet: the leaders were the same. The significance of the factor analysis should be enhanced with the increase in the number of the objects and the period of reporting. For a more comprehensive evaluation of universities, we need a comprehensive system, which is discussed in the next paragraph. Performance measurement is only one of the management tools. The basis of this position is simple: if the organization is complex and consists of many different components, evaluation of this activity should be complex and multifaceted. It has been said that the public sector cannot improve what it does not measure, especially given the significant reduction in government funding with increased community demand for quality services. Public sector key performance indicators (PSKPIs) need to be based on a comprehensive system of indicators [12] . Logic model of the construction and implementation of a rational and optimal PSKPI system must consist of following elements: (1) gather historical data from the organization, (2) organize and prepare the final database, (3) ascertain and define the numbers of strategic perspectives and performance indicators connected to those, (4) assemble the cause-and-effect link between all strategic perspectives and, lastly, (5) employ and operate this management tool for long-term vision. This logic sequence for Public University's performance evaluation is shown on Fig. 1. Fig. 1 . Logic sequence of performance evaluating (adopted from [11] ). Most budgetary entities need a clear, precise statement of purpose and a description of the work assignment, in conjunction with readily identifiable responsibility for the quality of their work. In this case, one of the main effectiveness criteria should be an assessment of how well they manage to achieve outcomes and/or outputs according to New Public Management (NPM). In internal performance evaluation it is reasonable to use auditors' methodology to evaluate Public University activity to avoid revealing the misuse, fraud, violent, etc. by Supreme Audit Institutions (SAI) auditors during budgetary control [15, 21, 22] . To manage Public University evaluating results we need to use of modern digital technologies but digital evaluating is not just a technology-based effort. It involves changing the expectations of what is included within an evaluation, and adjusting managers' knowledge, skills and abilities. This is especially true in terms of implementation of results-based budgeting, as well as openness and transparency in the sphere of Public Sector evaluating [1, 12, 15, 17, 20, 22] . The formation of an information database in the field of Public Sector evaluating results will allow for the recording, monitoring and analysis of inefficiencies and take measures to optimize budget expenditures. This database will provide the opportunity to hold videoconferences, implement electronic document management, work with documents, create and use an electronic library, conduct training, store information of the and other control and accounting agencies and implement other services. It would be great if the results of Public Sector evaluating were taken into account for budgeting of the next period. In the future, it is possible to link the definition of the size of the budget subsidy depends on KPI achievement [12, 22] . To achieve it needs to be ensured that KPI system reflects Public University activities fair, precise, and accuracy. Block-scheme for evaluating of Public University's KPI system is shown on Fig. 2 . The use of Block-scheme on Fig. 2 would help to analyze the result of budgetary funds using of Public University, find ways to solve the problem of rational use of budget resources and optimize fiscal relations in Public Sector unit. Digitalization. The digital revolution is changing the reality surrounding universities and changing the universities themselves. The connection of university changes with the total digitalization of the economy and society has become a generally accepted idea. The modern realities of the increasingly rapid introduction of digital learning have led to the creation of various digital environments at universities: a virtual educational environment, a mobile learning environment, an adaptive and intuitive environment [11, 14, 16-19, 24, 28, 29, 31] . The key for the future is learning to communicate and work virtually with Public University. More details about impact of Digitization on Public University activities are considered in works [9-12, 14, 16-19, 24, 28-30] . Preliminary. Public University managers must ensure reasonable confidence in preventing or detecting suspicious acquisitions in a timely manner at the expense of budgetary funds. To enable predictive modeling, data sets must comprise transactions considered suspicious and the outcomes of investigations. There can be used simple model with Excel function as in [29] . Combining observations and their outcomes allows managers to begin to build the link needed to predict future occurrences of purchase misuse. It should not be forgotten that Public Universities are audited by Supreme Audit Institutions which reveal frauds, violent, misuse, etc. [21] . While rule violations are not always indicative of misuse, they are an effective and simple way to alert program administrators to aberrant behavior especially using blockchain technology [22] . Whether a public sector transaction supervision system is automated or manual, supervisors should explore ways to integrate business rules and rule violations into the Public Sector evaluating process depends on time factor [29] . Wherein an effective notification system operates over the Treasury central server, delivers event messaging to predefined employees in "real time", bearing in mind time factor [29] , as the event occurs, and is sent directly to the employees and their smart devices. This level of event notification ensures that the people who need to know about an incident are made aware in a timely manner and fosters immediate and unified response as required [12] . Continuity. Continuous evaluating involves frequent monitoring throughout the year to ensure that transactions are captured properly and are flowing correctly to the income statement. During Digital Budgetary Transactions Surfing flows of transactions are continuously monitored, identifying transactions that match certain pre-determined integrity constraints and, in the event of a constrain violation, alert the Public University manager and copy the transaction data to a file. A natural first step in implementing datadriven techniques is to determine to appropriate transactions in the context of the Public University's day-to-day life [20] . Digital evaluation techniques can be combined with this. Analysis. Data analytics should be the foundation of Public University evaluating. Exploratory analysis and trending allow managers to identify patterns and detect anomalous behavior. Basic statistics like mean, standard deviation and skew, along with commonly accepted "tests", help identify transactions that are unusual. Such outliers and anomalies should raise red flags with supervisors and indicate the need for further investigation. Using existing technology, managers examine Public University transactions to spot trends. For example, a customer who's previous on time payment suddenly slows to 10 days late may signal a risk of default. Historically, researchers have used many quantitative and qualitative variables to evaluate the performance of KPI system implementations, ranging from return on investment to quality of user decision-making. Academic institutions, such as public universities, often have knowledge creation as a final goal, adding to the complexity of the KPI performance measurement challenge [23] . However, the application of quality in a manner that leads to a set of challenges facing the educational institutions, such as waste of human resources and misdirection leads to what is known as institutional burnout, and to meet these challenges had to be the proper and comprehensive application of the concept of total quality management to improve quality levels and enable the organization to excellence [6] . One problem cited is the cost in terms of data collection and analysis: many public universities lack this specific expertise and must employ consultants to assist. A further challenge is the difficulty with attributing performance to such specific budgetary entity as Public University. What is more, any performance evaluation system must take into account the fact that performance is the result of effective activity. So, the one of the practical drawbacks in the evaluation models lies in the problem of linking performance indicators that objectively must be reflecting dimensions of budget funding. Consequently, budgetary entities are often afraid to report bad news in case it affects their future funding. It must be born in mind that lack of differentiation in performance evaluation is also a very drawback factor [1] . In addition, unless experimental methods are used to isolate a research group and a control group, attributing any outcome in a beneficiary's life to a specific budgetary entity intervention, will also be challenged. However, the need remains for budgetary entities to show the difference they make in their communities, to be clear about the outcomes they are working towards, and to use performance frameworks to utilise scarce resources effectively. For this reason, it is important to understand how social welfare fit into the conventional set of performance indicator distinctions. Well-known that in recent (2019/2020, 2020/2021 learning years) reality most meaningful unprecedented phenomenon on the sphere of education is COVID-19 pandemic [25, 26, 31] . The outbreak of the pandemic is a global concern for universities, professors, students and public policy makers around the world. Reducing the funding of public universities will have medium and long-term effects and also puts us at a disadvantage for the new norm [27] . Because of the Covid-19, and the pandemic restrictions, we all have had to face situations that we have never imagined before. Every day we live in special circumstances that we have to understand and cope with, as best as possible. Teachers suddenly have to be not only educators but most of all, humans, that must take care especially of the needs, feelings and response to stress of their students. The educational system worldwide has not been prepared for such a sudden change, but both teachers and students have managed to cope, often with unexpected success, with this radical transformation. We start by emphasizing some obvious benefits of distance education brought beside and in opposition to the classic/face-to-face education [24] . At the present time, learning processes are being developed in online models, thereby also allowing flexibility of the process [10] . The impact of COVID-19 on the quality and evaluation of higher education has already been covered quite extensively in the recent literature [22] [23] [24] [25] [26] [27] [28] . Teachers and students are the main factors affecting the quality of online education. Due to the lack of information technology and insufficient information technology literacy of many teachers, online education implemented during the epidemic period copied the traditional education mode to the online environment, resulting in low educational effect [31] . As a survey result, the most meaningful barrier factor in teaching-learning process were mentioned: Social interactions (78% of respondents); Learner motivation (70% of respondents); Technical problems (50% of respondents); Personal perception (30% of respondents); Language skills (25% of respondents) [28] . This study was considered Performance evaluation of Public University as a whole organization in the three aspects: design/creating, evaluating, and challenging/monitoring. Subsequently, there were state the latest achievement in evaluation, monitoring, and challenges for public universities' KPI system. Distance education is a difficile task both for teachers and students. As long the necessity to keep education online as long the danger of generations with many gaps in education and certain subjects and topics. On the other hand, there are so many challenges in front of a qualitative distance education and the effects of the lack of social interactions, beside Distance Teaching-Learning-Evaluation Triad drawbacks, are about to be known and revealed during the following years [24] . It is obvious that digitalization of education involves the use of mobile and Internet technologies by students, expanding the horizons of their knowledge, making them limitless. The productive use of digital technologies, the inclusion of students into an independent search, the selection of information, and participation in project activities form the competences of the 21st century. It is clear that the benefits outweigh the cons. Technology can be a very effective tool, but it is just a tool. Technologies are not intended to replace the teacher, rather, the idea is to create a learning environment that will allow you to switch the organization of the educational process from one-actor theater to cooperation and interaction. Digitization has no doubt changed our education system, but we cannot say that it has diminished the value of traditional classroom learning. The best part about the digitization of education in the 21st century is that it is combined with the aspects of both; classroom learning and online learning methods. Walking hand in hand both act as a support system to each other, which gives a stronghold to our modern students. The implementation process can upset someone, annoy, take a lot of time and effort, but ultimately technology can "open doors" to new experiences, discoveries, ways of learning and cooperation of students and teachers [17] . The issues of pedagogical design and development of courses were not put on the agenda for the transition to distance learning. This was a consequence of force majeure circumstances that forced universities and teachers to mobilize all available resources and make a breakthrough in the massive emergency introduction of distance learning technologies. However, during the process professors improved the quality of online learning tools, which certainly takes much longer. In general, we can conclude that Russian education system coped with challenges and continued an educational process without failures in accordance with the schedule. Its format has changed, but not its quality. Moreover, improvements become a part of day-to day professors' work [24] . Introducing new approaches and techniques in Public Sector evaluating has been a challenging yet immensely rewarding process. By improving the impact of evaluation, Public Universities can steadily move closer to better public governance. Embracing digital Public Universities evaluating is a necessary investment to move higher education to new and evolving techniques by making full use of current and emerging technologies. We are in an information age and the exponential growth of data brings both challenges and opportunities to overhaul traditional sampling-based educational approaches and fully leverage technology. Public University evaluation provides a window to view trends, issues, and relationships across a wider expanse of data, and provide more meaningful and insightful observations to Public Sector leaders and stakeholders for improving Public Sector performance [20] . To do this, we should reform the education system and mechanism, introduce rules and regulations on socialized coordinated supply of educational resources and services, standardize and guide the participation of diverse subjects in online education [32] . The COVID-19 pandemic has a dramatic Impact that presents an unprecedented challenge to public educational system. Now is the time for global solidarity and support, especially with the most vulnerable in our societies, particularly in the emerging and developing world. Only together can we overcome the intertwined social and economic impacts of the pandemic and prevent its escalation into a protracted humanitarian catastrophe, with the potential loss of already achieved development gains [31] . Performance measurement in the Russian public sector: drawbacks and challenges Computer-assisted factor analysis of university performance indicators CSR implication and disclosure in higher education: uncovered points. results from a systematic literature review and agenda for future research Strategic management accounting and university performance: a critical review Research, teaching and performance evaluation in academia: the salience of quality The relationship between ineffective methods of applying quality management and functional combustion in Arab universities Advancing science or advancing careers? Researchers' success opinions on indicators Value for money in organizations providing public education services and how to measure it Evaluating institutional open access performance: methodology, challenges and assessment Hierarchical clustering to identify emotional human behavior in online classes: the teacher's point of view Indonesian and Russian public sector accounting education: difficulties, challenges, and effectiveness Human-computer interaction in the public sector performance evaluation analysis Performance evaluation in public institutions in Korea. Historical evolution and challenges for the future Informational technologies for the efficiency of public debt management in Russia Public sector performance auditing Digital universities in Russia: prospects and problems Integration of digital technologies into education An educational model of graduation project for students at automation and computer engineering Digital universities in Russia: digitization with extra speed Digital public sector auditing: a look into the future Fraud prevention by government auditors Using blockchain technology for government auditing A performance evaluation framework for a public university knowledge management system Is distance teaching-learning-evaluation triad a form of digital art? Situation Reports Coronavirus pandemic as black swan event Importance of university communication in times of pandemic COVID-19 and E-learning: challenges for Russian professors Organizational model for public sector auditing bearingin mind time factor Management of financial bubbles as control technology of digital economy Comprehensible Science. ICCS 2020 Online education action for defeating COVID-19 in China: an analysis of the system, mechanism and mode