Abstract:
This paper explores the effectiveness of various retrieval and re-ranking strategies within a Retrieval-Augmented Generation (RAG) framework applied to the Azerbaijani Tax Code. We evaluated two sparse retrievers (BM25 and SPLADE) and three dense embedding models (BGE-m3, OpenAI’s text-embedding-3-large, and MiniLM’s all-MiniLM-L6-v2), comparing their performance across standard information retrieval metrics. Among individual retrievers, BGE-m3 achieved the highest recall of 0.49 at the top 100 retrieved documents but still missed over half of the relevant documents. To address this limitation, we implemented a hybrid retrieval strategy combining BM25 and BGE-m3, which improved recall to 0.60—a relative gain of 11%. Further, we applied cross-encoder re-ranking with the bge-reranker-base model, increasing NDCG from 0.39 to 0.44. These results highlight the importance of a layered architecture that integrates both hybrid retrieval and re-ranking to enhance relevance, especially in regulation-heavy domains. Our findings offer practical insights into building robust and interpretable RAG systems for legal and structured text retrieval.

Published in: 2025 IEEE 19th International Conference on Application of Information and Communication Technologies (AICT)
Date of Conference: 29–31 October 2025
Date Added to IEEE Xplore: 08 December 2025
Publisher: IEEE
Conference Location: Al Ain, United Arab Emirates

Authors:
Zaid Rustamov — MegaSec LLC, Baku, Azerbaijan
Mehdi Gasimzade — MegaSec LLC, Baku, Azerbaijan
Samir Rustamov — ADA University, Baku, Azerbaijan

DOI: https://doi.org/10.1109/AICT67988.2025.11268677

Insights & Blog