META-ANALYSIS STUDY: THE RELATIONSHIP BETWEEN REFLECTIVE THINKING AND LEARNING ACHIEVEMENT

Reflective thinking is a must-have skill to connect the knowledge obtained with previous knowledge and can be seen from learning achievement. This study aims to prove and determine the relationship between reflective thinking and learning achievement and its effect size. This study used a quantitative meta-analysis method. Reflective thinking is the independent variable and learning achievement is the dependent variable. The data sources were obtained from online database searches on Google Scholar and international journal platforms from 2012 to 2021. Based on the search, 22 research publications met the predetermined criteria through a strict screening. Quantitative meta-analysis with correlation meta-analysis type was used to analyze the data. The software used was JASP 0.8 4.0. The results showed that this research Ho is rejected. It can be concluded that there was a significant relationship between reflective thinking and student achievement ( z = 8.139; p < 0.001; 95%CI [0.400; 0.654]). The effect of reflective thinking on student achievement was in the medium category ( r RE = 0.527). The findings are consistent with those of previous research on reflective thinking skills and learning achievement.


INTRODUCTION
Developing students' higher-order thinking skills (HOTS) is a complex multidimensional educational challenge.This thinking skill is part of the general skills that must be trained to students in all subjects to improve their performance and reduce their weaknesses (Arif, 2019).According to Qasrawi and Beni Abdelrahman (2020), cognitive processes of analysis, evaluation, and creation in Bloom's taxonomy are grouped into HOTS while knowledge, understanding, and application are grouped into LOTS.Yen and Halili (2015) state that thinking skills, especially HOTS, are the main benchmark in achieving learning objectives.Included in HOTS are critical thinking, logical thinking, reflective thinking, metacognition, and creative thinking.One of HOTS is reflective thinking (Setiawan et al., 2021;Dwyer, Hogan and Stewart, 2014).
Reflective thinking is a thinking activity that can make students try to connect the knowledge they have acquired to solve new problems related to their old knowledge (Choy and Oo, 2011).Khalid et al. (2020) state that reflective thinking is the ability to manage information or data to respond internally and explain what has been done.Some one who thinks reflectively will also realize their own mistakes and correct them and communicate ideas with symbols or images (abstract), instead of direct objects (concrete) (Chamdani, Salimi and Fajari, 2022).Reflective thinking is part of the critical thinking process, which refers to the process of analyzing and making judgments about what has happened.Reflective thinking is the most important skill in encouraging learning during complex problem-solving situations because it allows students to step

ERIES Journal volume 15 issue 3
Electronic ISSN 1803-1617 back and think about how to solve the problem and how a set of problem-solving strategies is accomplished to achieve their goals (Shavit and Moshe, 2019;Orakcı, 2021).According to Dewey (Ozudogru, 2021), reflective thinking means being active, continuous, persistent, and carefully considering everything that is believed to be true or the format of knowledge with supporting reasons leading to a conclusion.Boody, Hamilton, and Schon (Ozudogru, 2021) explain the characteristics of reflective thinking as follows: (1) reflection as retrospective analysis or recall (ability to judge oneself), (2) reflection as a problem-solving process (awareness of how one learns), (3) self-critical reflection (developing self-improvement continuously), and (4) reflection on selfconfidence and success.Students who have a reflective style tend to spend more time responding and reflecting on the accuracy of answers.Reflective individuals are very slow and careful in responding but tend to give correct answers (Kholid et al., 2020).Reflective students are more likely to perform tasks such as remembering structured information, reading by understanding and interpreting texts, solving problems, and making decisions.They may also determine their own learning goals and concentrate on relevant information.They usually have high work standards (Choy and Oo, 2011;Kablan and Gunen, 2021).Empowering reflective thinking skills is the task of all levels of education.Belief in reflective thinking plays an important role because it is closely related to how students can evaluate themselves.Reflective thinking can also be used to encourage thinking processes during problem-solving.With reflective thinking, students can predict the correct answer immediately so that they can explore problems by identifying the concepts involved, using various strategies, building ideas, drawing conclusions, re-examining solutions, and developing alternative strategies (Mirzaei, Phang and Kashefi, 2014a;Kablan and Gunen, 2021).In addition, Qasrawi and Beni Abdelrahman (2020) state that HOTS is closely related to thinking skills following the cognitive, affective, and psychomotor domains as an integral part of the teaching and learning process.Therefore, reflective thinking skills, one of HOTS, influence learning achievement in terms of cognitive, affective, and psychomotor aspects.Several studies mention that reflective thinking contributes to learning achievement, including the research of Farahian, Avarzamani, and Rajabi (2020), Akpur (2020), Pham et al. (2020), Hsia and Hwang (2020), Aslam et al. (2021), Safari, Davaribina, and Khoshnevis (2020), etc.Studies related to reflective thinking and its relationship to learning achievement tend to be carried out partially or only part in certain situations, so that it is difficult to see as a whole.For instance: Pham et al. (2020) examine reflective thinking skills and learning achievement only for ELF students in the context of academic writing.Then, Aslam et al. (2021) examines reflective thinking skills and specific learning achievement only for the students of teacher education.Furthermore, Hsia and Hwang (2020) examine reflective thinking skills, dance learning achievement, self-efficacy and task load, especially in flipped learning conditions.Studies that are not comprehensive and explicitly focused on the effect of reflective thinking skills and learning achievement have several shortcomings, such as: the research results are often biased and only applied to certain situations so that these are difficult to apply to other situations.One alternative to thoroughly analyze the relationship between reflective thinking and learning achievement is by using a meta-analysis study.Meta-analysis is systematic and quantitative research using the existing studies used by other researchers to obtain accurate conclusions (Briggs, 2005;Basu, 2017).Malički et al. (2021: 3) state, 'Meta-analysis is a form of research using data from other existing studies (secondary data)'.Therefore, it is a quantitative research method by analyzing quantitative data from the results of previous studies to reject or fail to reject their hypotheses.This type of research method is increasingly popular to summarize research results (Yusuf and Fajari, 2022).Meta-analysis is widely used in the study of research theory.In addition, it can be a source of foundation in policymaking (Borenstein et al., 2009;King and He, 2006).In the meta-analysis, the data processed are used to make statistical conclusions.The data can be expressed by various measures that are calculated or searched in advance by formulas in various mathematical equations, which are closely related to the research objectives of the meta-analysis carried out (Pereira et al., 2019;Turner, Bird and Higgins, 2013).This size is known as the effect size.Meta-analysis includes content analysis that encodes the characteristics of a study, such as age, research location, or other domains in a scientific field.Effect sizes with the same characteristics are grouped and compared (Mueller et al., 2018).Meta-analysis has some advantages, including (1) metaanalysis procedures apply useful disciplines in summarizing research findings; (2) meta-analysis is conducted in a more sophisticated manner than conventional review procedures, which tend to rely on qualitative summaries or "vote-counting"; (3) meta-analysis can find influences or relationships that are obscured in other approaches to summarizing research; (4) meta-analysis provides an organized way of dealing with information from a large number of research findings under review (Briggs, 2005;Borenstein et al., 2009;Basu, 2017).Based on the explanation above, reflective thinking skills are very important for every student at all levels of education.
To make an overview of the relationship between reflective thinking and learning achievement around the world, a metaanalysis study is needed.This is the first meta-analysis study that tests the universality of this relationship among participants from different countries.Therefore, this study aims to prove and determine the effect size of the relationship between reflective thinking and learning achievement through a quantitative meta-analysis approach.

Research Design
The quantitative meta-analysis method was used in this study.Quantitative meta-analysis is a statistical technique that combines two or more similar studies to obtain a quantitative mix of data (Mueller et al., 2018).Viewed from the process, meta-analysis is a retrospective observational study where

Electronic ISSN
1803-1617 the researcher recapitulates the data without performing experimental manipulation.The recapitulated data were obtained from research publications related to the relationship between reflective thinking skills and learning achievement at the tertiary level.

Eligibility Criteria
The research publications were selected by several criteria with the aim that the results of this extensive analysis can be more centralized.The studies to be included depend on the purpose of the meta-analysis (Tawfik et al., 2019).Therefore, the metaanalysis study hypothesis is very helpful in determining the inclusion and exclusion criteria that should be used from the outset to identify relevant studies (Higgins et al., 2019)

Data Analysis
The data analysis in this study was carried out through the following steps: (1) analysis of the characteristics of the research sample; (2) data coding; (3) conversion of the values of t and F to the value of r correlation with the formula below; 2

F t =
(1) (4) heterogeneity test of effect size; (5) calculation of the summary effect or mean effect size; (6) creating forest plots and funnel plots; (7) hypothesis testing; (8) checking publication bias.The data were analyzed using correlation meta-analysis.At the hypothesis testing stage, the p-value obtained was used to test the following hypothesis.

ERIES Journal volume 15 issue 3
Electronic ISSN

1803-1617
H o : There is no significant relationship between the reflective thinking and learning achievement Ha: There is a significant relationship between the reflective thinking and learning achievement Effect size can be categorized into the values of 0-1 based on Cohen's effect size criteria (Cohen et al., 2020).The software used in this research was JASP 0.8.4 because it can be installed on various computer operating systems, has Cohen's criteria options, provides assumption testing, and has many helpful features for those who want to learn the analysis and interpretation of statistical results.The Cohen's effect size criteria are presented in Table 2.

Value
Criteria < 0 + / -.1 Weak effect < 0 + / -.3 Modest effect < 0 + / -.5 Moderate effect < 0 + / -.8 Strong effect ≥ + / -.8 Very strong effect  The results of the heterogeneity test above showed that Q = 411.734with p < 0.001; τ² or τ > 0; I² (%) is close to 100%; it means that the 22 effect sizes of the analyzed studies were heterogeneous.Furthermore, an analysis of the estimation of the summary effect or mean effect size was carried out, and a publication bias test was performed using a random effect approach.The results of the analysis of the summary effect or mean effect size is presented in Table 5.The results of the analysis using the random effect model showed a significant positive correlation between reflective thinking and student achievement (z = 8.139; 95%CI [0.400; 0.654]).The p-value which shows < 0.001 proves that this research H o is rejected.It can be concluded that there is a significant relationship between the reflective thinking and learning achievement The relationship between reflective thinking and student achievement was included in the moderate category (r RE = 0.527).Furthermore, the analysis results are presented using a visually attractive graphical method, referred to as forest plots.Forest plots allow us to know the estimated combined effect depicted by plots (dots) at certain intervals at the same time to make comparisons between studies clearer.A chart of the forest plots of the 22 analyzed studies is presented in Figure 1.

95% Confidence Interval
Based on the forest plot chart, the effect sizes of the analyzed studies vary from -0.06 to 1.30.Furthermore, the funnel plot was made.Begg's funnel plot is a scatter diagram used in meta-analysis to visually detect the possibility of publication bias (symmetrical or asymmetrical research sample).A funnel plot chart for the 22 studies analyzed is presented in Figure 2.
The results of the funnel plot chart had no clear indication of publication bias because the model formed was symmetrical or asymmetrical, so further analysis using Egger's test was necessary.Egger's test results are shown in Table 6.

185
Electronic ISSN 1803-1617 Table 6 shows Z = 0.591 with p > 0.05.This confirms that the funnel plot is symmetrical.Thus, there is no publication bias problem in this meta-analysis study.

DISCUSSION
Based on the 22 research results which were analyzed through this meta-analysis, there was a significant positive relationship between reflective thinking and learning achievement (p-value < 0.05).The more skilled students reflect the relationship between concepts, causal relationships, analogous relationships, or differences, the more skilled they are in making decisions, conclusions, and working on questions quickly and precisely for better learning achievements (Turan, Fidan and Yildiran, 2019;Isler, Yilmaz and Dogruyol, 2020).Furthermore, based on the results of the effect size analysis, the 22 studies showed that the relationship between reflective thinking and student achievement was in the moderate category based on Cohen's effect criteria (r RE = 0.527).Students' reflective thinking processes affect learning achievement by responding quickly to a problem and linking what is known and asked in the problem with their previous knowledge to reflect on and determine the right strategy to solve the problem with reasoning (Tsingos-Lucas et al., 2016;Aldahmash, Alshalhoub, and Naji, 2021).Students' reflective thinking processes can be seen from the confusion and doubt in solving a problem and obstacles that make students quickly investigate it with their knowledge (Turan, Fidan and Yildiran, 2019;Spears et al., 2021).Reflective thinking allows students to learn to think about the best strategies in achieving learning objectives (Mirzaei, Phang and Kashefi, 2014b).In addition, reflective thinking can help them integrate their thinking skills by conducting assessments (Maksimović and Osmanovic, 2019).Reflective thinking is important for students to solve problems optimally (Spears et al., 2021).Therefore, it affects the way students decide on everything including cognitive, affective, and psychomotor activities in the components of learning achievement.This is supported by several studies stating the same theory, including Farahian, Avarzamani and Rajabi (2020), Akpur (2020), Pham et al. (2020), andChen, Hwang, andChang (2019).
Furthermore, Kholid et al. (2020) state that students should have reflective thinking skills in the learning process to solve problems of everyday life.With reflective thinking, someone can understand, criticize, assess, find alternative solutions, and evaluate the issues being studied.To improve students' reflective thinking skills, teachers can support them to hone their skills by using problembased learning models, varied approaches, and open-ended essay questions (Killingsworth and Xue, 2015;Toman, 2017;Mirzaei, Phang and Kashefi, 2014b;Yilmaz, 2020).In this study, no publication bias was found.Publication bias can be detected through analysis of the symmetrical shape of the funnel plot and Egger's test.They have the same conclusion.Analysis of publication bias is needed to determine the level of significance of the sources used, the quality of relevant research methods, accurate study conclusions, and different sample sizes which will affect minimally biased research conclusions (Nair, 2019;Joober et al. al., 2012).Therefore, the studies that were not included had the same results as those included as a sample in this meta-analysis.

CONCLUSION
From the results and discussion above, it is confirmed that reflective thinking skills affect learning achievement, which is indicated by the effect size of 22 publications which are proven to be heterogeneous and have a positive correlation value in the moderate effect category.Furthermore, publication bias does not exist, which means that the publications under review truly reflect the actual situation.The characteristics of the publications studied show the same sample, namely students, even though they are from various scientific fields.It is recommended that future researchers use similar themes by focusing on the sample of the research publications, such as elementary school, junior high school, high school, or nonformal education students.It is intended that there will be more theories on the relationship between reflective thinking skills and student achievement so that teachers will improve their teaching and consider this topic.The limitation of this research is that some publications are not reputable by Scopus, Web of Science, or SINTA.In fact, the better the reputation of the journal being studied is, the higher the quality of the data presented.

Table 2 : Cohen's Effect Size Criteria (Source: Cohen et al., 2020) RESULTS
Based on the analysis of 22 publications with specific criteria, various values of r, t, and F were obtained for each study.Before performing the heterogeneity test, the researchers converted the t-or F-values of all research publications that have no r-value to r-value.The results of the heterogeneity test are presented in Table3and residual heterogeneity estimates are presented in Table4.The model was estimated using the Restricted ML method.