Dense Retrieval for Code-Mixed Queries: A study of Hindi–English and Yoruba–English search
Oluwaseun Adebayo, Elias Wagner, Ravi Menon
Users frequently mix languages within a single query, but retrieval benchmarks rarely reflect this. We construct code-mixed query sets for two language pairs and evaluate dense retrievers under different tokenisation and fine-tuning strategies. Adapting the query encoder with synthetic code-mixed data improves recall@20 by 9 points over a multilingual baseline, while transliteration normalisation helps for romanised queries but hurts for native-script queries.