Recently, Artificial Intelligence (AI) systems focused on natural language processing have
generated a climate of motivation and concerns, especially in the educational context. Chats
based on Large Language Models (GML), such as ChatGPT, have the capacity to produce texts
in natural language on a wide range of subjects and diverse tasks, such as preparing an item
(question) on a specific mathematics subject. In practice, the user can provide conditions,
parameters and instructions to the chat, obtaining answers or even new questions on the
proposed topic. Considering the potential of these tools, this study investigated the application
of free tools that use generative AI in the construction and review of multiple choice items.
Methodologically, the research is characterized as applied, of a qualitative and quantitative
nature, and is constituted as bibliographic, documentary and experimental. The chats used were
ChatGPT, based on the GPT 3.5 Model, and Bing Chat. The study presented natural language
guidelines and protocols that can be provided to chats for creating or reviewing math items
related to specific skills. Some of the items created from these protocols and tools with a test
composed of 20 items that include skills recommended by the Saeb Reference Matrix. This test
was answered by 61 students in the 9th year of elementary school at a public school located in
the municipality of Terra Santa, Pará. The results indicated that the chats presented problems
in all parts of an item: statement, support, command and alternatives. However, chats also
demonstrated the ability to understand the skills requested and formulate situations involving
them. Some items produced with the help of chats and reviewed by the author showed good
discriminatory power, indicating that the tools were useful for the process of preparing
mathematics items, as long as they were improved by the teacher/developer.
|
