What are code embeddings?
Vector representation of source code is needed for many practical tasks: from searching for similar fragments to automatically grouping functions by meaning. A program is transformed into a set of numbers that machine learning algorithms can work with. The quality of the vector depends on how well the model understands the semantics of the code, not just its syntax.
What's the challenge
Classical methods usually require separate training for each task, or at least fine-tuning on domain data. This is expensive and not always justified. Approaches based on large language models (LLMs) look more universal, but they have their own drawback: model responses often contain extraneous or erroneous information, and this "noise" ends up in the vectors.
The idea behind LSem2Vec
The authors of the paper proposed a two-stage approach with the telling name LSem2Vec — LLM-extracted code Semantics to Vector embedding. Instead of trying to train yet another heavy model, they combine ready-made components: an LLM extracts the meaning from code, and a sentence embedding model converts the description into a vector. No fine-tuning for a specific task is required.
Let's break down the process step by step. At the first stage, the LLM receives code and must articulate what it actually does. This is not a mechanical retelling of the source code, but an extraction of the essence: the model discards secondary details so that only what matters for meaning ends up in the description. Along the way, this approach also cuts off possible "hallucinations" of large models — erroneous information that most often appears when trying to encode code directly.
The second stage is trivial by machine learning standards: the description obtained from the LLM is fed into a sentence embedding model. Such a model is already trained to turn natural language into meaningful vectors, so there is no need to adapt it to the syntax of programming languages.
Why it works
The strong point of LSem2Vec is the clear division of responsibilities. One model is responsible for understanding logic, the other for translating text into a vector. Each solves its task in the best possible way, and together they produce a vector based on semantics rather than formal code features. The absence of a training stage makes the approach cheap and easy to reproduce.
How it was tested and what the results were
Experiments were conducted on three datasets compiled in different programming languages. The authors deliberately varied both the LLMs and the embedding models to make sure the result does not depend on a specific pair of tools.
In all configurations, LSem2Vec outperformed five modern unsupervised methods, which are usually tailored to specific conditions and require tuning.
Paper status
The paper was posted on arXiv in September 2024, with the latest available version being v5 from August 2026. According to the authors, the work has been accepted by the journal Frontiers of Computer Science. This means the approach now has not only a preprint page but also a peer-reviewed publication. The original can be found via DOI: 10.1007/s11704-026-61288-0.
Conclusion
The main takeaway of LSem2Vec is that complex problems do not always require complex solutions. It is enough to sequentially apply an LLM to extract meaning and a sentence embedding model to create a vector. It is fast, requires no special fine-tuning, and delivers competitive results.
