The new artificial intelligence system capable of answering any question with natural language poses challenges for teaching methodology and evaluation. But the key to its good use is critical thinking, a core competence in university education.

ChatGPT has put first-year students on the URV’s bachelor’s degree in Mathematical Engineering and Physics to the test. In fact, it was the professor of applied mathematics Manel Sanromà who tested them by using the artificial intelligence (AI) tool that is dominating the news and conversations, especially those involving teachers. It is based on a language model that uses deep learning to produce texts similar to those written by humans. Sanromà says that the emergence of this system generated the same feeling of being faced with an element of disruption as the emergence of Internet, and considers that educational organizations “cannot stand on the side-lines”.

This is why he has already begun to test it by using it as part of the examination he gives to his students of Linear Algebra, the mathematical basis of machine learning, the technology at the heart of the explosion of Artificial Intelligence (AI), such as the famous Google algorithm. The professor asked ChatGPT to generate four statements about the linear independence of vectors, two of which were true and two of which were false. Then, the students had to decide which of the statements were which.

Asked how it will affect teaching at universities, ChatGPT lists three more advantages besides the automatic generation of questions and answers: the creation of educational materials; the automatic correction of exams, and real-time assistance for students. However, can this tool change the day-to-day lives of students, teachers and researchers?

First of all, Sanromà predicts that “in a matter of weeks Bing (Microsoft’s search engine) will incorporate the answer that has been formulated into searches and Google may move from being the number-one search engine to number two or three. The new system will also be added to the applications in the Microsoft 365 software package and we will be able to ask for content in natural language. For example, PowerPoint will be able to generate a presentation on any topic we ask for in the number of slides we specify.”

But are the answers given by ChatGPT reliable enough to be used in educational and research contexts? Antonio Moreno, a professor of computer languages and systems and an expert in logic for AI, points out that ChatGPT “is not a knowledge base with associated complex inference mechanisms, nor a tool that asks a question and goes to the web to look for the possible answer, but simply a language model. However, it is a sophisticated tool.”

This is why the machine only responded correctly to one of the four statements that it generated for the exam question that Sanromà requested because “the algorithm has no idea of logical validity. It is a generator of correct sentences, based on the text we enter and the learning it has had”. ChatGPT agrees with him and points out that “careful implementation and continuous evaluation are required to ensure that this technology is used effectively and ethically”.

In this respect, Marta Sales, a professor of physics and a specialist in data analysis and prediction, considers that “the major danger is that this tool will become the main source of information. It is useful, but you have to know how to apply critical thinking and check the information it has given you.” Sanromà agrees that it is “vital to teach critical spirit.” In fact, “the role of the teacher will be to teach students to check what the machine says and provoke them. What is true? What is false? Classes will be more fun,” he says.

The professor of network security and the URV’s Commissioner for Open Government and ICT, Jordi Castellà, goes a little further and reflects on the possible misuse of the tool to pass exams and what to do in case of conflict. It seems that it will be “difficult” to prove that it has been used, so it is more feasible “to do the tests in controlled environments where there are guarantees that the answer has come from the student.” However, he concludes that while these systems “can facilitate much more plagiarism or unacceptable behaviours by students, using them for this purpose goes against their own interests because their goal would no longer be to learn, but to pass the exam.”

According to Sanromà, the tool will also affect research: “Scientific articles will be written by the machine, and you will have to modify them”. Sales considers, however, that for writing reviews and the context of research projects “it can be a source of inspiration” but “it will be of no use if you are engaged in innovative research or the investigation of issues that no one has ever studied before.”

Moreno is of a similar opinion: “I would advise any researcher from any area to be attentive to the progress made in this field, but if you use ChatGPT right now, you must be aware of the mechanisms behind it and, therefore, of what you can or cannot expect it to do for you.” His research group, ITAKA, designs new mechanisms for intelligent data analysis and decision support, and then applies them to such fields as medicine, tourism, communication or linguistics. And while he is aware that systems such as ChatGPT “are currently very popular because they are very apparent, free and accessible to the general public”, he is clear that “better and more advanced systems will emerge.”

ChatGPT said exactly the same when we asked it: “Researchers are constantly working on improving the capabilities of AI systems, such as machine learning, natural language processing, and understanding context.” But it wasn’t able to go into any greater detail. “It is difficult to accurately predict when systems that are better and more advanced than ChatGPT will be developed. The advancement of artificial intelligence technology will depend on investment in research and development, as well as the speed at which the scientific community progresses.