Retrieval Augmented Generation Based System to Analyze Turkish News Texts According to Sustainable Development Goals Using Large Language Models (RAGaze)

Abstract

Due to its geographical location and social structure, Türkiye is a country where events that deeply affect many people happen every day. For example, climate changes are now clearly observed in our country where four seasons are experienced; this situation affects agriculture and the economy, while at the same time, since Türkiye has borders with many countries due to its geographic location, it makes our country a bridge country that contributes to peace. However, the recent increase in the number of women murders shows how essential it is for the press to cover important issues facing the society and thus raise public awareness about these issues. For this reason, a system that aims to identify the extent to which the written press in Türkiye pays attention to social sensitivities has been made. The developed system is a chatbot application that works on a user interface. It analyzes columns published in various newspapers based on user questions within the framework of Sustainable Development Goals (SDG) such as equality, justice, economy, climate change and gender equality and responds accordingly. It uses a large language model (LLM) infrastructure for this purpose. The model classifies the extent to which these issues are covered in Turkish newspapers using Retrieval Augmented Generation (RAG) and vector embeddings. In addition, two different embedding models, text-embedding-004 and gemini-embedding-exp-03-07, were used and it was determined which one was better for Turkish texts.As a result of this project, a successful chatbot application was developed to analyze press content related to SDG. The chatbot application RAGaze provides accurate and meaningful answers to user queries. In addition, the answers given to the questions always remain within the context of SDG and RAGaze does not act outside this context. © 2025 IEEE.