OVER THREE DECADES OF DATA ENVELOPMENT ANALYSIS APPLIED TO THE MEASUREMENT OF EFFICIENCY IN HIGHER EDUCATION: A BIBLIOMETRIC ANALYSIS

The higher education efficiency evaluation model using the data envelopment analysis method has interested many researchers. This paper uses bibliometric analysis on publications extracted from the Scopus database to provide a comprehensive overview of research publications on the measurement of higher education efficiency based on data envelopment analysis: its growth rate, major collaboration networks, the most important and popular research topic. A total of 169 related publications were collected and analyzed from 1988 to 2021. The analysis results show that: Publications published every year have increased sharply in the last six years; The quality of publications is relatively high as publications tend to be published in journals with high-ranking indexes; Countries with the most influence in studies on this topic are: Italy, China, Spain, the USA, and the United Kingdom; Authors with the most influence in this research direction are Agasisti T., Abbott M., Doucouliagos C


INTRODUCTION
The higher education (HE) sectors of many countries derive part of their income from public funds.Therefore, for the sake of accountability, measuring the efficiency of institutions including this sector is essential (Johnes, 2006).Unlike economic efficiency, which is measured through the combination of several inputs with one output, the higher education sector has characteristics that are difficult to measure its performance: it is a non-profit operation; absence of input and output prices; and higher education institutions generate many outputs from many inputs (Daghbashyan, 2009;Johnes, 2006).Efficiency concepts are frequently found in national education planning documents but without clarification as to whether efficiency is a final goal or a channel to achieve a certain educational objective.When used by economists the term efficiency is mostly contextspecific, whereas practitioners affix diverse, conceptually different, uses to the term.In this section the conceptual and definitional issues regarding efficiency in education is presented.The commonly examined types of efficiency in both the public sector and education studies cover both technical and allocative efficiency (Kosor, 2013).McMahon (1983) expands these two concepts to four efficiency concepts in the provision of education: technology, price, exchange, and allocative efficiency.Technical efficiency examines the time and resources used in the production of a given output (these resources include teaching methods, instructional materials, student's learning activities, over some time period).Price efficiency is an extension of technical efficiency since it takes into consideration the relative costs of resources.Exchange efficiency represents education's ability to meet the needs of other institutions (business, civic and religious organisations), and concerns questions like whether the credentials are valued in the labour market or whether workers are overqualified.Allocative efficiency is attained when there is technical, (factor) price and exchange efficiency, i.e., it represents the maximisation of satisfaction given scarce resources with competing uses (and it allows for a comparison of educational costs with its expected benefits.Efficiency measures can be divided into four aspects, technical efficiency, allocative efficiency, scale of efficiency and dynamic efficiency.Two methods are mainly used to measure efficiency: Stochastic Frontier Analysis (SFA), and Data Envelopment Analysis (DEA) (Cavaignac and Petiot, 2017).SFA was first introduced by Aigner et al. (1977) and Meeusen and Van den Broeck (1977).It consists of estimating a parametric marginal econometric model.DEA, proposed by Charnes, Cooper and Rhodes in 1978, is a non-parametric method for measuring the production efficiency of Decision-Making Units (Charnes et al., 1978).Its main advantage over SFA is that it does not require any parametric assumptions regarding production frontiers.The contour of the observed input and output levels of DMUs are calculated by the linear programming and can be considered as the best practice frontier.By measuring the gap between a company and the efficient frontier, it is possible to calculate the efficiency of Decision-Making Units (Cavaignac and Petiot, 2017).Various DEA models (two-stage DEA, input/output-oriented DEA, etc.) have been used in materials, and additional statistical inference methods may consolidate the validity of the results (Simar and Wilson, 1998).DEA also has many weaknesses, for example, it gives efficient frontiers that can be quite large (Cooper et al., 2011).Currently, DEA has been widely used in the field of higher education efficiency evaluation, achieving many significant findings.DEA-based higher education efficiency studies have appeared in various academic journals worldwide.Bibliometrics is an essential branch of information and library science, and it is based on various literature.It provides a quantitative analysis of academic literature (Merigó et al., 2015).Scientific research in bibliometrics has developed recently (Liu, 2019).It can also be used to evaluate the growth of scientific research in a country and understand its current position (Ha et al., 2020;Pham-Duc, Tran, et al., 2020;Pham-Duc, Nguyen, et al., 2020).Many authors have also used bibliometrics as an approach to evaluate higher education efficiency (Abramo et al., 2011;Abramo et al., 2008;Abramo and D'Angelo, 2009;Andersson et al., 2017;Andersson and Sund, 2021;Ferro and D'Elia, 2020;Ibrahim and Fadhli, 2021;Johnes and Johnes, 1992;Mikušová, 2017).
Bibliometric analysis of studies using the DEA method of efficiency evaluation has been applied in various fields such as: energy efficiency (Trianni et al., 2018;Yu and He, 2020), trajectories of efficiency measurement (Lampe and Hilgers, 2015), Islamic banking (Rusydiana et al., 2021), transport sector (Cavaignac and Petiot, 2017).Furthermore, in evaluating "efficiency in higher education", there has also been a bibliometric analysis by Ramírez-Gutiérrez et al. (2019) about university rankings disclosure and efficiency in higher education.In this paper, the author's objective is to do a bibliographic analysis of all scientific publications on the measurement of efficiency in higher education using the DEA method, which has been indexed in the Scopus database recently.We consider scientific articles, conference papers, book chapters, and reviews for analysis.The three objectives are: (a) summarize the general characteristics and trends of scientific publications, the most important source journals, the most productive institutions, and the most productive scholars; (b) analyze the international cooperation between countries in this sector; and (c) to extract the most popular research topics and trends based on word analysis of titles, abstracts, and keywords.After presenting the methodology, in the next section, we will present our main findings and discuss them before concluding the study in the final section.Five central bibliographic databases can be used to conduct a bibliometric analysis, including Web of Science (WoS), Scopus, Google Scholar, Microsoft Academic, and Dimensions (Moral-Muñoz et al., 2020).Among them, WoS and Scopus are most used for bibliometric analyses.We decided to use the Scopus database as the search engine because it covers a broader range of documents than other databases (Ha et al., 2020;Mongeon and Paul-Hus, 2016;Pham-Duc, Tran, et al., 2020).

METHODOLOGY
This study used a general scientific mapping process consisting of five stages: 1) Study design; 2) Data collection; 3) Data analysis; 4) Data visualization; and 5) Interpretation (Börner et al., 2005;Zupic and Čater, 2015).In the study design stage, the main research question was: What is the bibliography of research publications indexed in the Scopus database on measuring efficiency in higher education using the Data Envelopment Analysis method?The data collection stage is divided into three sub-stages: data collection, data filtering and data cleaning.
Step 1: Collect data.The authors performed the search from the Scopus database (http://www.scopus.com),with advanced search options for entering search terms and matching operators according to this search engine's syntax.The identified search keyword consists of 3 components: 1) Related to efficiency and efficiency evaluation: efficiency; "measurement efficiency"; performance; "measurement performance".2) Related to Higher Education: university; "higher education"; "higher public education" and 3) Related to data envelopment analysis: "data envelopment analysis"; DEA.The author uses the OR and AND operators to combine the keywords accordingly.Such keywords are searched in the document's abstract, keywords, and title.
The data limited to the Document type is article, conference paper, book chapter, and review.The field of study is identified as social sciences, written in English.TITLE-ABS-KEY ((efficiency OR "measurement efficiency" OR performance OR "measurement performance") AND (university OR "higher education" OR "higher public education") AND ("data envelopment analysis" OR dea)) AND (LIMIT-TO (DOCTYPE, "ar") OR LIMIT-TO (DOCTYPE, "cp") OR LIMIT-TO (DOCTYPE, "ch") OR LIMIT-TO (DOCTYPE, "re")) AND (LIMIT-TO (SUBJAREA, "SOCI")) AND (LIMIT-TO (LANGUAGE, "English")) Box 1: Query string in the Scopus database Data query was performed from the Scopus database on July 05, 2021.As a result, 226 documents were obtained.
Step 2: Filter the data.The author conducted data filtering by censoring titles, abstracts, and keywords to remove documents that are not directly related to the research issue.The number of documents remaining was 169.With these lists, we conducted some initial analysis on the tools provided by Scopus to collect additional information regarding authors, affiliations, and journals.Metadata of the final publication collection were exported to CSV format for post-processing in two bibliometrics analysis tools, the Biblioshiny and the VOSviewer.Additional information on the Scopus website (https://www.scopus.com/),and from the Scimago Journal & Country Rank (https://www.scimagojr.com/)was also used to support our analyses.
Step 3: Clean the data.The downloaded data should be cleaned because the quality of the analysis is highly dependent on the quality of input data (Ha et al., 2020).Several data errors were fixed in this sub-step.For example, "Lancaster University Management School" and "University of Lancaster" and "The Management School, Lancaster University" were corrected as one affiliation.
In the data analysis stage, the author used several analytical techniques to extract information from a collection of publications.
In order to understand the growing trend of the research field, general information on publication collection was summarized, and the number of publications per year was analyzed.Contribution by countries, institutions, journals, and authors based on the number of papers and citations was analyzed to identify the most productive ones.The top 10 most cited papers were extracted based on the number of citations, along with their citations and the authors' information.
A co-occurrence network of 40 most popular keywords based on their frequency was generated using the VOSviewer tool.In addition, keywords often appearing in published papers were coded in the same colors and grouped in the same clusters.

General information and growth trend
The primary information of the paper's dataset is shown in Table 1.The total number of publications in the collection is 169 documents, published in 92 different Sources (journals, books, etc.) from 1988 to 2021.Most are articles with 156 publications (92.3%), followed by conference papers: 6 publications, book chapters: 4, and review papers: 3 publications.The total number of citations is 3695, or 21.86 citations per document.The h-index of this collection is 30, which means that out of 169 publications under study, 30 have been cited at least 30 times).367 authors have participated in research and publication in this field (equivalent to 2.17 Authors per Document).In particular, the percentage of Single-authored documents in the collection is quite large, with 37 documents (21.9% of publications) of 34 authors.The information on annual publication output and cumulative citations is shown in Figure 1.The annual growth rate of studies on measuring efficiency in HE using DEA is 8.94%.The growth trend of scientific output can be divided into three sub-stages: Stage 1: 1988-2003: The first stage saw a light output, and the first publications in this field began to appear.However, studies on this issue were not continuous over the years.A total of 11 publications were published during this 15-year stage.Stage 2: 2004-2015: Number of publications published: 58.Annual Growth Rate: 6.5%.During this stage, the number of publications published annually was always less than 10.Therefore, the growth of studies in this field was not evident.Stage 3: 2016-2021: This stage saw a significant increase in the number of published publications.There were 100 publications from 2016 to the time that the data for this study were collected (July 5, 2021) that have been published.Excluding 2021 data (because the time of the study is not over yet), the Annual Growth Rate: is 17.76%The number of citations increased steadily every year during stages 2 and 3, increasing rapidly in recent years, corresponding to the increase in the number of studies on this topic.The statistics for the number of citations of publicationsrelated studies on the measurement of efficiency in HE using DEA are shown in Table 2.The number of uncited articles from any document is 31 (18.3%), and the number of publications with citations more minor than the average number of citations of the whole collection (21.86) is 130 (accounting for 76.9%).The number of articles with more than 50 citations is 20 (11.8%), of which seven articles have more than 100 citations.

Contribution by countries
Information on the top 12 countries with the highest number of publications related to the measurement of efficiency in HE using DEA is shown in Authors of single-authored documents 33 Authors of multi-authored documents 334 Single-authored documents 37 Authors per Document 2.17    The international cooperation network (at least two papers) in the measurement of efficiency in HE using DEA is shown in Figure 2. The size of the nodes indicates the number of publications, while the thickness of the lines between nodes shows the strength of collaboration.Authors from 46 countries participated in the study and had publications in the field.After removing the countries that do not have authors cooperating and linking with other countries, the number of countries remaining is 19.China is the country whose number of publications results from the most extensive international cooperation with other countries such as the USA, Spain, Malaysia, Japan, Germany, Sweden, the Philippines, Korea, and Australia, followed by Italy, Spain, and the USA.The most international cooperation is between Spain and Colombia, followed by Spain and Italy.

Contribution by Institutions
According to our retrieved Scopus database, the authors participating in the study on measuring efficiency in HE using DEA came from 160 affiliations.The top 10 most productive institutions are based on the number of publications shown in Table 4.The institutions in this Top 10 list are from Italy (3), China (2), The United Kingdom (1), Spain (1), Poland (1), Australia (1), and Russia (1).The first on this list is Politecnico di Milano with nine published publications that have been cited 382 times.Although ranked second in the number of publications (6 papers), Lancaster University has an astonishing total number of citations with 591.This number of citations mainly comes from two articles with a high number of citations: "Data envelopment analysis and its application to the measurement of efficiency in higher education" (Johnes, 2006) with 381 citations and "Research funding and performance in U.K. University Departments of Economics: A frontier analysis" (Johnes and Johnes, 1995)

Contribution by journals
As mentioned above, the number of sources publishing research publications on measuring efficiency in HE using DEA is 92 different.The top 11 most active journal publishing is shown in Table 5.The number of publications published by these sources is 44 (accounting for 39.1% of the whole collection), but the total number of citations recorded is 2772 (accounting for 75.0% of the whole collection).SEPS is the journal publishing the most publications on this topic, with 12 papers and 617 citations.Followed by Scientometrics with ten papers, 253 citations, and Education Economics with eight papers, 486 citations.The remaining journals publish from 4 to 6 papers on this topic.
Regarding citations, it is noteworthy that the Economics of Education Review (EER) with a total citation of 5 papers is 895.This journal has contributed three publications with the highest number of citations, as shown in Table 7. ((Abbott and Doucouliagos, 2003) -381 citations, (Johnes, 2006) -329 citations, (Johnes and Johnes, 1995) -146 citations ).These journals rank highly in Scopus's journal rankings: Q1 (7) and Q2 (2).However, only one journal is rated Q3 (ERIES), and one journal is rated Q4 (IJEED).Ranked according to CiteScore, the Journal of Informetrics has the highest index (8.6),followed by Higher Education (6.3), and two journals with a CiteScore index of 4.9 are SEPS and Research Evaluation.

Contribution by authors
The top 10 most productive authors based on the number of publications and their citations are shown in Table 6.Half of these top authors are from Italy, followed by the United Kingdom (2 authors), the USA (1), Australia (1), and Germany (1).The author with the most contribution in this field is Agasisti T. from Politecnico di Milano with ten papers and 434 citations.He is the main author of 9/10 papers in this field.This was followed by The annual publications and citations of the top 10 authors can be shown in Figure 3. Agasisti T. is the author with the most and most frequently published publications.His first paper on this topic was in 2007.Johnes J. and Johnes G. are two authors with a long research history on this topic.They have cooperated to publish articles since 1992 (Johnes and Johnes, 1992) and 1995 (Johnes and Johnes, 1995), but the two authors' research history on this topic is not continuous.The remaining authors have several publications that are not large enough to represent research trends, mainly published in recent years.The co-authorship network of scholars is shown in Figure 4. Nodes represent authors.The size of the nodes indicates the number of publications, while the thickness of the lines between nodes shows the strength of collaboration (meaning the number of publications they were co-authors).This network consists of authors with at least two published papers in the measurement of efficiency in HE using DEA.There are 34 authors, all meeting this condition.The most significant collaboration is between the group of authors headed by Agasisti T. and the authors: Wolszczak-derlacz J., Landoni P., Dal Bianco A., De La Torre E.M., Johnes G., and Johnes J.There are two groups with three authors: Zhu Q., Wu J. and Zhang G. from China; Guccio C., Martorana M.F., Monaco L. from Italy.In addition, there are six groups of two authors.

The most influential articles
A list of the top 10 most cited papers in the published collection of 169 papers is shown in Table 7, along with their average citations per year.The total number of citations of the top 10 papers is 1785, corresponding to 48.3% of the total citations of the collection at the time of this study (3695 citations).The paper on efficiency of Australian universities, which Abbott M. authors from Victoria University, and Doucouliagos C. from Deakin University ( 2003) published in Economics of Education Review has the most significant number of citations at 381.The average annual citation number of this paper is 20.05.The second position in terms of the number of citations is the publication of Avkiran N.K. from The University of Queensland ( 2001), published in Socio-Economic Planning Sciences journal on technical and scale efficiencies of Australian universities, and this paper has been cited 329 times, equivalent to an average annual citation number of 15.67.The third position on this list is a paper on data envelopment analysis and its application to the measurement of efficiency in higher education (Johnes, 2006), authored by Johnes J. from Lancaster University published in 2006 on Economics of Education Review, 312 is the number of citations as of the time of this study, equivalent to 19.50 citations per year.These two papers are publications in the form of Single-authored documents.The remaining papers on the list have citations from 88 to 106, published between 1988 and 2011 (first and second substage).The three publications on this list have outstanding citations compared to the remaining studies.These are highly representative studies for the research direction of measuring efficiency in HE using DEA.Table 7 also shows the number of citations of publications by citation position in the scientific article.Cohan et al. (2019) categorize citation intents into three types: background information, methods used, and comparing results.Information on the number of citations by citation position was collected from Semantic Scholar (https://www.semanticscholar.org).Note that the total number of citations for these three categories may not be the same as the number

Electronic ISSN
1803-1617 of citations listed in the TC column because this number of citations is limited to articles for which Semantic Scholar has access to the full text (Semantic Scholar, 2022).Accordingly, the total number of citations of the top 10 most cited articles by content is: Background Citations: 920 (64.2%),Methods Citations: 436 (30.4%), Results Citations: 77 (5.4%).The historical direct citation network of the 17 most local cited articles in the publication collection is illustrated in Figure 5.Each node represents a paper, and the lines between nodes indicate a direct citation between two papers.The historical direct citation network is beneficial in identifying the core papers and all related papers to a specific research direction.Several authors in Table 6 also have papers shown in Figure 5, such as Agasisti ( 2011), (Guccio et al., 2017;Guccio et al., 2016a;Guccio et al., 2016b), etc.

Keywords and terms analysis
The co-occurrence network of keywords over the period is shown in Figures 6.In the co-occurrence network of keywords, each keyword is represented by a node, and the thickness of lines between two nodes is proportional to the strength of the relationship between them.This relationship was determined by the number of times they appeared together in published papers of the published collection.Close and related keywords (or research topics) were coded in the same colors and grouped in the same clusters (Pham-Duc, Tran, et al., 2020).To build this network, the author has removed phrases that cannot show research trends such as: article, review… combined synonymous keywords: dea, "data envelopment analysis (dea)" into "data envelopment analysis"; "Malmquist productivity index (mpi)", mpi into "Malmquist productivity index", etc.There are 169 publications and 563 keywords for the period 1988-2021.The co-occurrence network of the 40 most popular keywords, which appeared at least three times, is presented in Figure 6.The keywords in the network are grouped into three main groups with three different colors: the Green group with main keyword phrases: data envelopment analysis, Higher education, efficiency, university, productivity, higher education institutions, bibliometrics.The Red group with the keyword phrases: university sector, technical efficiency, performance assessment, efficiency measurement.And the Blue group with the main keywords: China, Malmquist index, sustainability.Research trends in this field by year through keywords are shown in Figure 7.In this figure, the line represents the timeline of the keyword, a bubble at a given year means the keyword appeared the most in the publications of the respective year, the bubble size is proportional to the number of publications containing this keyword.When building the figure, we only selected keywords that appeared in at least 3 publications.The author also removed the keyword "data envelopment analysis" because according to the search method, this keyword appeared in all publications.The keywords in Figure 7 can all be found in Figure 6.The topics that have received a lot of attention recently are: efficiency measurement, research efficiency, resource allocation, performance evaluation, benchmarking, stochastic frontier analysis, efficiency analysis.Some topics have received much attention in the past, but have received little attention recently, such as: technical efficiency, university performance, university libraries, scale efficiency.These are good suggestions for research on the topic of measurement of efficiency in higher education using Data Envelopment Analysis.

DISCUSSION
In this study, bibliographic data in the Scopus database were used to plot the overall picture of all research publications and discussed the measurement of efficiency in higher education using DEA from 1988 to 2021.In the period from 1988 to 2003, although the number of published publications is not much, these are all highly influential papers with a high number of citations (directive for further studies).From 2004 to 2015, the number of publications published annually has increased but not much.In the recent period, there has been a remarkable growth in studies on this topic.This growth model is quite similar to the bibliometric studies in several other fields in recent years, such as STEM Education (Ha et al., 2020;Özkaya, 2019); Internet of things (Ruiz-Rosero et al., 2017); DEA on Islamic Banking (Rusydiana et al., 2021); DEA applied to energy efficiency (Yu and He, 2020).The number of studies on the topic of DEA applied to the measurement of efficiency in higher education is significantly less than the research on the application of DEA in other fields.The bibliometric study of Yu and He (2020) shows that there are 1206 research papers on DEA applied to energy efficiency in the period 1992-2018; meanwhile, Cavaignac and Petiot (2017) found 461 research articles on the use of DEA applied to the transport sector  in their research.This shows that scholars have not paid enough attention to the use of DEA to measure efficiency in higher education.The countries with the most influence in studies on this topic are Italy, China, Spain, the USA, and the United Kingdom, when they have the most significant number of publications and the greatest number of citations (see Table 3).Most affiliations with the most significant number of published papers belong to these countries (see Table 4).Research collaboration in this area is not strong (see Figure 2), both in collaboration among countries and among affiliations.Authors who research this topic tend to publish in journals with a high ranking in Scopus's journal rankings (See Table 5).Most of the journals in the Top 10 most active journals list have Q1 and Q2 rankings.This demonstrates the quality of studies in this field.Economics of Education Review has only five published papers but has an outstanding number of citations of 895, accounting for 24.2% of total citations.The overall quality of these studies is also reflected in the whole collection's average number of citations, with 21.86 citations/document.This is slightly larger than the average number of citations for publications using DEA in energy efficiency evaluations of 18.33 citations/document (Yu and He, 2020).The three authors with the most influence in the research direction of measurement of efficiency in higher education using DEA are: Agasisti T., Johnes J., and Johnes G.These are the authors with the most publications and such publications have the most citations.Especially, Agasisti T. is the author of 10 publications in this field and has a research history from 2007 to present.He is also a key member of collaborations with other authors and groups of authors.This research group is also the most significant collaboration among research authors on this topic.The papers with the most influence on studies in this field are from the following authors: Abbott and Doucouliagos (2003), Avkiran (2001), andJohnes (2006).In addition, the papers in the list of 10 most influential papers also have rapidly increased citations in the past five years.This is consistent with the rapid increase in the number of studies published during this period.The most cited content is used in the background information, with nearly two-thirds of the total citations of the top 10 publications with the highest citations.This shows that these publications have a significant role in guiding the research on this topic.The keyword analysis showed that the studies focused on using the DEA method to evaluate efficiency in higher education.In addition, the studies also go into specific directions such as: efficiency measurement, resource allocation, Malmquist index, performance evaluation, benchmarking, stochastic frontier analysis, efficiency analysis, bibliometrics.These research directions have been shaped mainly in recent years.Although this study only deals with studies related to higher education performance assessment using the data envelopment analysis method, through keyword analysis, the results show that research trends in this area include both parametric and non-parametric techniques such as DEA, the Malmquist index, and stochastic frontier analysis (SFA).This can be explained by the keywords that appear in the reviews, or there may be studies using multiple analytical methods, as in the study by Ferro and D'Elia (2020).Ferro and D'Elia analyzed 89 studies published between 1997 and 2019 on higher education efficiency frontier analysis.Most of the papers used the non-parametric DEA model to estimate the efficiency (54%), followed by the SFA model (40%), and both methods (6%).Considering higher education as a production process with corresponding inputs and outputs for performance evaluation is a common approach across many fields today (see also (Viet Nguyen et al., 2019)).Higher education efficiency is about maximizing output and minimizing input in the production process.Tools for estimating this degree of optimization have, to date, been developed very rapidly; Accordingly, the DEA method is very commonly used in statistical research in the world today.Therefore, implementing scientific quantification to find the main exploitation directions and quality documents on DEA is necessary, helping researchers and managers have an overview to select and use.Use appropriate documents to build a higher education performance analysis tool.This study has several limitations, which were already reported in previous papers (Ha et al., 2020;Pham-Duc, Tran, et al., 2020).First, this study analyzed only publications related to measuring of efficacy in HE by DEA from the Scopus database, written in English.Indeed, the data will not include all articles on this topic.Publications not analyzed may include publications written in languages other than English; publications from sources not yet indexed in the Scopus database.Second, the authors have attempted to filter out all irrelevant articles from Scopus data manually, but this filtering may not be ideal and may contain omissions.Third, some types of information, such as author names and author institutions, are not standardized in the Scopus database.Since manual corrections are not possible, the results may be affected as the analysis depends entirely on the quality of the input information retrieved from the Scopus database.

CONCLUSION
This study uses bibliometric analysis to analyze research trends and develop publications using DEA to evaluate the efficiency of higher education indexed in the Scopus database.The authors used a 5-step process recommended by Börner et al. (2003) and Zupic and Čater (2015) to conduct the study.The main findings of this study are: 1) Although the first publications on this topic appeared in 1988, the studies on this topic have received much attention in the last five years.Therefore, the influence of publications in the first period (1988)(1989)(1990)(1991)(1992)(1993)(1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002)(2003) is quite considerable, oriented for further studies; 2) The overall quality of publications is relatively high when the average number of citations is significant, and publications are mostly published on journals with high ranking indexes; 3) The countries with the most influence in studies on this topic are: Italy, China, Spain, the USA, and the United Kingdom but the international cooperation in these studies is not strong; 4) While Agasisti T. is the author with the most publications, the authors with the most influence on research trends are: Abbott M., Doucouliagos C., Avkiran N.K., and Johnes J. when publishing the publications with the highest number of citations.
The research direction on applying the DEA method in evaluating the efficiency of higher education is a research direction with high application and efficiency.However, the number of studies on this topic is still modest.Scholars interested in the field can refer to the high-influence publications mentioned above.Scholars also should strengthen international cooperation further to improve the quality of studies in the coming time.

Table 1 :
Main information of the publication collection related to the measurement of efficiency in HE using DEA (source: own calculation).

Figure 1 :
Figure 1: Bar chart illustrates the annual number of publications and their cumulative citations

Figure 2 :
Figure 2: International cooperation network in the measurement of efficiency in HE using DEA.Artwork generated with Biblioshiny.

Figure 3 :
Figure 3: The top 10 most productive authors over the time

Figure 4 :
Figure 4: Co-authorship network of 34 scholars with at least two publications related to measuring efficiency in HE using DEA.

Figure 5 :
Figure 5: Historical direct citation network of the 17 most local cited papers in the published collection.The artwork was generated with Biblioshiny.

Figure 6 :
Figure 6: Co-occurrence network of the most popular keywords period 1988 -2021.

Figure 7 :
Figure 7: Topic trend of the publication collection related to the measurement of efficiency in HE using DEA.The artwork was generated with Biblioshiny.

Table 3 .
The total number of published publications in the countries in this list is 117, accounting for 69.2% of the collection.These publications have been

Table 2 : Number of citations of the publication collection (source: own calculation)
2% of the total citations for the whole collection.It is noted that the total number of publications in this list is more significant than 117 because an article can have co-authors from different countries.Italy contributed the most publications in this field, with 20 papers (accounting for 11.8% of publications) and 629 citations (accounting for 17% of total citations).China ranks second on this list with publications less than the leading country, but these publications are cited only 72 times.The following two positions are Spain and the USA, with the same 17 papers and 321 citations.The number of papers is 11, but the United Kingdom has the highest citations among all other countries (834, accounting for 22.6% of total citations), and Taiwan has 96 citations.Australia has the second number of citations, with seven papers (4.1% of publications) and 747 citations (accounting for 20.2%).The remaining five countries on the list have 6, namely India, Colombia, Turkey, Germany, and Brazil, with citations from 111 down to 21.
TP: Total publications; TC: Total citations

Table 4 : Top 10 most productive institutions publishing based on the total number of publications (source: own calculation)
with 146 citations.The remaining institutions have published three publications.

Table 5 : Top 10 most active journals publishing research related to the field of measurement of efficiency in HE using DEA based on the total number of publications (source: own calculation)
TP: Total publications; TC: Total citations *According to data from Scimago Journal & Country Rank (https://www.scimagojr.com)dated July 10, 2021

Table 6 : Top 10 most productive authors (source: own calculation)
Johnes J. from the University of Huddersfield with four papers (492 citations) and Johnes G. from Lancaster University with four papers in this field.The remaining authors have published three publications.Among these authors, Abramo G. (Consiglio Nazionale delle Ricerche) and D'Angelo C.A. (Università degli Studi di Roma Tor Vergata) are in their names together in all three papers.The same goes for two authors from the Università degli Studi di Catania, who are Guccio C. and Martorana M.F.
TP: Total publications; TC: Total citations