Deewan: Semantic Search Engine for Arabic Poetry

Deewan is a semantic search engine for Arabic poetry. Instead of matching letters, it understands meaning — search for an idea or feeling like "longing for home" and it surfaces verses that express it, even when those exact words never appear.
The Corpus:
- 6.57 million verses indexed in total
- 6.44M classical (fusha) verses from the open ashaar dataset
- 125K Nabati verses from the tarab dataset
- Spanning twelve literary eras, from pre-Islamic to modern
How It Works:
- Every verse is encoded into a 384-dimensional vector using the Arabic-Triplet-Matryoshka-V2 embedding model
- Verses close in meaning end up close in vector space
- An HNSW (Hierarchical Navigable Small World) index searches all 6.57M verses in a few seconds on a single server
- Classical and Nabati live in two separate indexes so the traditions never cross-contaminate
- Era-aware re-ranking gives classical poets a fair share alongside modern ones
Academic Benchmark:
Deewan was evaluated on Fann-or-Flop (EMNLP 2025), a peer-reviewed benchmark of 53,047 scholar-written verse explanations. Given only an explanation, it must retrieve the correct poet out of thousands. Deewan places the correct poet in the top-10 for 3.13% of queries — about 22 times better than random (~0.14%). To our knowledge, this is the first time an Arabic poetry search engine has been measured against a published academic benchmark.
The Stack:
- FastAPI backend running as a systemd service
- Next.js 15 (App Router) frontend, Arabic-first, with next-intl
- hnswlib for approximate-nearest-neighbor search
- Self-hosted on a single dedicated server with no external API dependency
- Spans about 10,600 lines of code across frontend, backend, and data pipelines