Development of Chemistry Tests for Students’ on Reaction Rate Subject Matter Based on Critical Thinking Skills using Framework DOT Test

Received: 31 Mei 2021 Revised: 07 Sept 2021 Accepted: 19 Oct 2021 Published online: 24 Oct 2021 This study aims to develop a valid and reliable test to measure the critical thinking skills of the high school students of class XI on reaction rate subject matter using the Danczak – Overton – Thompson test framework (DOT test). The research method used in this research is development and validation. In this article, there are three stages of test development, namely the analysis of KI and KD in Chemistry of 2013 curriculum, analysis of the DOT test framework, and the preparation of the test predictions. The development stages produce a draft instrument that is ready to be validated. The developed instrument consists of 30 test questions. The validation stage includes the content validity test by 5 expert judgments. In the validity test, the calculated CVI value is 0.97 so that the instrument is valid. Based on the analysis of research data, it is concluded that the chemistry test based on critical thinking skills that was developed had a good validity value to measure students' critical thinking skills on the reaction rate subject matter.



The rapid demands and development of science have resulted in a shift of the educational institutions' focus from developing theoretical knowledge to developing thinking skills, one of which is critical thinking skills (Sadhu & Laksono 2018). The 2013 curriculum requires students to have critical thinking skills in science learning in order to face the world demand in the 21st century. Critical thinking skills are inherent in science education that becomes a vehicle for improving the quality of human resources in the midst of the change of world demands. Improving human resources who have critical thinking skills need to be prepared in connection with changes in the 21st century era which is full of demands for human resource capabilities (Lutfianto & Sari, 2017). In fact, one of the life skills required in this globalization era is critical thinking skills (Khasanah et al, 2017;and Anazifa & Djukri, 2017). The critical thinking skills is an essential skill in life and the working world. Also, it has an effective function in all other aspects of life to improve human beings quality. According to Wartono et al. (2018), critical thinking skills are a superior ability which plays an important role in all aspects of life. Furthermore, Liliasari (1999) states that the formation of critical thinking skills is very decisive in building personality and patterns of action in the life of every Indonesian. Therefore, to achieve this goal, it is necessary to empower science learning.
The 2013 curriculum syllabus explains that critical thinking skills in chemistry learning will equip students to live in a society or further study related to the characteristics of chemistry as the foundation for various basic and applied sciences. In this regard, Danczak et al (2019) state that critical thinking skills are rarely assessed explicitly, even though these skills have an important role as an attribute needed in higher education or in the work environments. Developing critical thinking skills for students means to prepare them in making decisions, solving problems and reflecting on their performance. According to Suwandi (2011), the achievements of high-level thinking skills cannot be separated from the assessment which must be implemented as an integrated part of the learning process to determine the development and learning outcomes of students and also to improve the learning process. Therefore, education providers have the responsibility to develop students' critical thinking skills, one of which is through assessment in the form of tests.
A literature review by Danczak et al (2019) shows several assessments of critical thinking skills with commercially available context approaches including the California Critical Thinking Skills Test (CCTST), Watson-Glaser Critical Thinking Appraisal (WGCTA), Watson-Glaser Critical Thinking Appraisal Short Form (WGCTA-S), Cornell Critical Thinking Test Level Z (CCTT-Z), Ennis-Weir Critical Thinking Essay Test (EWCTET), and Halpern Critical Thinking Assessment (HCTA). However, this commercial assessment only measures broad critical thinking from general disciplines, so it requires a critical thinking assessment to measure critical thinking specifically from the results of chemistry learning. In addition, the research conducted by Danczak et al (2019) developed and evaluated critical thinking tests (Danczak-Overton-Thompson's chemistry critical thinking test or DOT test) which were arranged and designed in a chemistry context in the form of multiple choice test in inorganic chemistry topics which included: (1) making assumptions: 7 questions (2) analyzing arguments: 7 questions (3) developing hypotheses: 6 questions (4) testing hypotheses: 5 questions (5) drawing conclusions: 5 questions. The results showed that the developed test instrument was classified as a good measuring instrument in terms of validity and reliability so that it was able to measure the critical thinking skills of chemistry students in inorganic chemistry lectures.
Some similar studies have also been conducted by Kartini & Liliasari (2012) who developed a multiple choice critical thinking skill test on thermochemistry material, Amalia & Susilaningsih (2014) who developed a critical thinking skill test in the form of a description of acid-base material, Nurpratiwi (2017) who developed a multiple choice critical thinking skills test on the buffer solution material. However, according to Danczak et al. (2019), the studies have not referred to the test framework which has a chemistry context, so it is considered inaccurate to reflect students' critical thinking skills specifically in chemistry learning.
Based on the literature study, there has not been found a critical thinking skill test instrument with a chemistry context for high school students with a test framework that refers to Danczak et al (2019). Therefore, a study is needed to develop a test instrument to measure the critical thinking skills of high school students which refers to the test framework according to Danczak et al (2019). One of the selected SMA (High School) chemistry subject matter is the reaction rate. This material is listed in Permendikbud No. 37 in 2018 concerning KI and KD for SMA/MA chemistry subjects and SMA/MA chemistry syllabus for class XI on basic competency 3.6. The reaction rate material was chosen because of the many events around which unconsciously involve the simple concept of reaction rate. One of them was conveyed by Kurniawan & Fadhilah (2018) that the reaction time of a bomb exploding which is faster than the rusting of iron is a phenomenon involving the reaction rate material. In addition, there are also events that cannot be seen with the naked eye, such as molecules colliding with each other which also involves the rate of reaction. These events when analyzed using a critical thinking process will bring up new skills in decision making and problem solving in everyday life using the concept of reaction rate as a whole. To determine the critical thinking skills that will be assessed, it is necessary to study the material on the reaction rate. Besides, this material contains a thinking process in analyzing arguments and provides knowledge for each meaning and interpretation of the influence of concentration factors, surface area, temperature and catalysts on reaction rate. In this case, characteristics of the material include aspects of critical thinking skills, so that it has a great opportunity to explore aspects of students' critical thinking skills. For that reason, students' critical thinking skills are expected to be measured properly using the critical thinking skills test instrument that will be developed.
Based on the description above, a study of "The development of chemistry tests for high school students on reaction rate material based on critical thinking skills using the DOT test framework" will be conducted. The study aims to develop a test to measure the critical thinking skills of high school students in class XI on a valid and reliable reaction rate material using the DOT test framework.

Methodology
This study uses the Development & validation research method by Adams & Wieman (2010). This method explains how an educational researcher can create valid and reliable assessment tools. An adjustment was also made to the stages of developing a test instrument carried out by Danczak et al (2019).
The data collection technique in this research is the validation sheet. The validity test used is the content validity. A total of 5 expert judgments were asked to see the suitability of the indicators with the test items. Data analysis on the validity test used CVR calculations. The calculated CVR results are then compared with the critical CVR in the following  (Wilson, et.al 2012) In general, the stages of the study consist of the development and validation stages as shown in figure 1. The development stage includes analysis of KI and KD in chemistry of 2013 curriculum, analysis of the DOT test framework, and preparation of test questions prediction. The validation stage is carried out to test the validity of the instrument. The instrument used in this study is the instrument validation sheet. The stages of the developed study are as shown in follows figure 1: Figure.1 Stages of developing and validating the instrument

Results and Discussion
This section will explain the development process and quality of chemistry tests using the DOT test framework to measure students' critical thinking skills. The development process includes the analysis of KI and KD chemistry of the 2013 curriculum, analysis of the DOT test framework, and preparation of test predictions. The instrument used in this study is the instrument validation sheet.
The first stage of the development process is the analysis of KI and KD chemistry in the 2013 curriculum. The purpose of this analysis is to determine the chemistry material used in chemistry tests using the DOT test framework to measure students' critical thinking skills. Danczak et al (2019) proposes that the critical thinking skills test in a chemistry context must be able to measure students' skills in forming assumption patterns, analyzing arguments, developing hypotheses which then testing them against a new phenomenon, and being able to draw reasonable conclusions. Therefore, in designing chemistry tests based on critical thinking skills, it is necessary to select subject matter that are in accordance with these indicators. The material chosen is the reaction rate in basic competencies 3.6 and 3.7. The basic competence 3.6 explains the factors that affect the reaction rate using collision theory. The basic competence 3.7 determines the reaction order and the constant of reaction rate based on experimental data. The reaction rate material was chosen because there are many phenomena that we don't realize involving a simple reaction rate concept. When the phenomena are analyzed using critical thinking processes, they will generate new skills in decision making and problem solving in everyday life using the concept of a whole reaction rate.
After analyzing KI and KD in the chemistry of 2013 curriculum, the next step is to analyze the DOT test framework. As previously explained, the critical thinking test version of the DOT test is specifically designed for the chemistry context so that it is used as a reference in compiling the test questions in this study. Based on the results of the analysis, the DOT test framework consists of a general structure and a special structure. The following is the explanation of both:

The general structure of the DOT test
Based on the analysis result, the DOT test has a general structure as described in the following explanation:

The short section
This section contains various types of discourse according to the characteristics of each CTS indicator. The information in this section is always assumed to be correct by the readers. This section is a stimulus in test questions.

The follower Section
This section contains citations of statements that follow previously read discourses, in accordance with the characteristics of each CTS indicator. This section is regarded as a test item.

Answer Choices
Each question is answered individually based on the reader's decision on the previously read short section and follower section. The editorial of the sentence on the answer choices will be the same for each CTS indicator.

DOT test specific structure
There are five indicators of critical thinking skills (CTS) on DOT test questions, namely: (1) making assumptions (2) developing hypotheses (3) testing hypotheses (4) drawing conclusions (5) analyzing arguments. The following table 2 contains the results of analysis for each indicator: Table 2. DOT test specific structure

DOT CTS Indicator Making Assumptions
Structure -There is a discourse that contains information based on facts or test results. (short section) -There is a short statement containing assumptions related to the discourse that has been presented. (follower section) Answer choices -Option A. Valid assumptions: selected if the assumptions can be accepted easily, referring to the discourse presented. -Option B. Invalid assumptions: selected if the assumption cannot be accepted easily, referring to the discourse presented.

Number of Questions
-7 questions -Short section contains 2 discourses -First discourse followed by 4 assumptions (items of question) -Second discourse followed by 3 assumptions (items of question)

Developing hypotheses
Structure -There is a discourse that contains temporary / uncertain conclusions, yet based on the available information, there is a certainty in the hypothesis that being developed. (short section) -There is a short statement containing inferences related to the discourse that has been presented. (follower section) Answer Choices -Option A. Accurate inference -Option B. Inference that is not necessarily accurate (due to insufficient information) -Option C. Inaccurate inference Number of questions -6 questions - The short section consists of 2 discourses. -The first discourse is followed by 3 inferences (items of question) -The second discourse is followed by 3 inferences (items of question)

Testing Hypotheses
Structure -There is a discourse that contains hypotheses that are believed to be correct, and systematically seek for information to confirm or refute these hypotheses. This results in a premise that is believed to be accurate or true. (short section) -There is a short statement containing a hypothesis relating to the discourse that has been presented. (follower section) Answer Choices -Option A. Reasonable deduction: the evidence presented is supported by a hypothesis. -Option B. Unreasonable deduction: the deduction is not related to the hypothesis and there is not enough evidence to support the claim.

Number of questions
-5 questions -2 Discourses of short section. -The first discourse is followed by 3 hypotheses (item of questions) -The second discourse is followed by 2 hypotheses (item of questions)

Drawing conclusion
Structure -There is a discourse that contains deductions, preliminary conclusions / inferences or premises to draw a conclusion.
(short section) -There is a short statement containing conclusions regarding the discourse that has been presented. (follower section) Answer choices -Option A. Reasonable conclusion: deduction based on relevant information and undoubted (though uncertain) logic. -Option B. Unreasonable conclusion: conclusion not directly supported by relevant information.

Number of questions
-5 questions -2 discourses of short section. -First discourse followed by 2 conclusions (item of question) -Second discourse followed by 3 conclusions (item of question).

Analysing argument
Structure -There is a discourse that contains assumptions, inferences, deductions & premises, conclusions (short section) -There is a short statement contains arguments related to the discourse that has been presented. (follower section) Answer choice -Option A. Valid argument -Option B. Invalid argument Number of questions -7 questions.
-2 discourses of short section. -First discourse followed by 4 arguments (item of question). -Second discourse followed by 3 arguments (item question) Based on the explanation regarding the analysis of the Danzcak-Overton-Thompson test framework, it can be concluded that the framework presents specifications in accordance with the characteristics of critical thinking skills tests in a chemistry context. Thus, for the study on the development and validation of critical thinking skills tests, the reaction rate material refers to the framework. The DOT version of the CTS test is used to measure CTS for chemistry students on inorganic chemistry, so the adjustments are conducted to measure CTS for high school students. In addition, the adjustment process made to the DOT version of the CTS test includes: (1) change of some of the terms in the answer choices that are less familiar to students, (2) addition to illustrations / pictures to stimulate students' skills, (3) adjustment of the scope of the material for high school students' reaction rates of class XI.
The third stage is the preparation of a test prediction to produce a draft of test items that are ready to be validated. The test prediction consists of basic competencies, test indicators, and the number of test questions. The designed test indicators are the result of the integration of the learning indicators and the critical thinking skills indicator from the DOT test version. There are 20 test indicators that have been designed and contain 5 indicators of critical thinking skills in the DOT test version with the distribution of the number of questions referring to the original DOT test.
After developing the test prediction, the next step is to arrange the test items based on the indicators that have been designed. The number of questions designed for the test is 30 multiple choices, the number is to adjust the original DOT test.
At the validation stage, the instrument validity is tested. The validity test used is the content validity. This validity will be carried out to see the suitability of the test indicators with the items. There are 5 expert judgments consisting of 2 chemistry education assessment lecturers, 1 physics chemistry lecturer, and 2 basic chemistry lecturers. The CVR value is calculated for each item. The result of the calculation shows that 29 test questions have a value of CVR = 1, while 1 item of the test has a value of CVR = 0.2. Thus, the CVI value is obtained of 0.97. This shown in figure 2 that the instrument is categorized as a valid instrument.  This phenomenon is equipped with illustrations / pictures so that critical thinking skills are more explored.
3.6.8 Estimating the number of particles produced at low and high temperatures.
Changing the editorial of sentences in the phenomenon section. 3.6.17 Drawing conclusion of relationship between the size of the substance and the touch area.
Adding a wooden cube illustration to explain the six-sided meaning.
The revision is based on suggestions and input from expert judgment. The revision includes content focusing on item questions, improving the words editorial and sentences on phenomena and statement items, adding illustrations or images to complete the phenomenon section. After the validation, 29 valid questions obtained with several revisions on each item and one question was declared invalid so that the question was discarded.

Conclusion
Based on the results of data analysis and discussion, it can be concluded that the chemistry test using the DOT test framework to measure students' critical thinking skills has a good quality. The quality can be seen from the validity of the contents of the instrument, the instrument is categorized as a valid instrument. The development of chemistry tests using the DOT test framework to measure students' critical thinking skills is conducted in 3 stages. The stages include analysis of KI and KD chemistry of 2013 curriculum, analysis of the Danzcak-Overton-Thompson test framework, and preparation of test predictions. The three stages will produce a draft instrument which is then validated by expert judgment. At the development stage, the obtained 30 test items consist of 20 test indicators and the DOT version of the CTS indicator including 7 questions to make assumptions, 6 questions to develop hypotheses, 5 questions to test hypotheses, 7 questions to analyze arguments, and 5 questions to draw conclusions. At the validation stage, 29 test questions are declared as valid with several revisions. The revision includes content focusing on item questions, improving the editorial of words and sentences on phenomena and statement items, adding illustrations or images to complete the phenomenon section.