AI & Data7 November 20267 min read

Codebase Intelligence for AI-Assisted Engineering — Part 3 of 5

RAG finds similar text. A code graph finds relationships

Embedding search over a repository returns code that sounds like the question, not code that is connected to it. Part 3 of Codebase Intelligence for AI-Assisted Engineering compares lexical search, semantic search, symbol navigation and code graphs, traces one order-total rule across API, domain, persistence and tests, and sets out a hybrid pattern for large repositories.

#RAG#AI Coding#Coding Agents#AI Architecture#Knowledge Management
A dense block of small translucent tiles representing an old software system, with a luminous graph map floating above it; one amber node sends lines down to the few tiles a change would touch.

Komati Commerce, the fictional South African commerce platform in this series, had already indexed its repositories for retrieval. Each file was split into chunks, embedded and stored, so that a coding agent could ask a question in plain language and get back the most similar code. It worked well for "how do we format currency?" It worked badly for the question that mattered before the delivery-fee change: "where is the order total worked out, and what depends on it?"

The top five results were a README paragraph about order totals, a test fixture full of totals, a deprecated helper called totalHelper, the cart component that formats the total for display, and, in fifth place, the pricing engine. None of the results mentioned the invoice builder that recomputes the total, or the reconciliation view that reads it. Every result was similar to the question, but most of them were not connected to the answer.

RAG retrieves similar text. Enterprises need trusted meaning made this argument for documents. This article makes it for code, where the gap is wider, because code has explicit structure that similarity ignores.

Four ways to find code

Each method answers a different kind of question, and each fails in a predictable way.

MethodHow it worksGood atFails when
Lexical searchExact or regular-expression match on text, sometimes with a trigram indexExact names, column names, error strings, configuration keysThe name differs, or the concept has no fixed word
Semantic searchEmbeddings of chunks, nearest neighbours to the question"Where do we do something like X?", unfamiliar codebasesSimilar wording beats real connection; chunks cut functions in half
Symbol navigationLanguage server or index: definitions, references, implementations"Where is this defined and who uses it?" with compiler precisionYou do not yet know which symbol to start from
Code graph traversalCalls, imports, inheritance and data edges, followed hop by hop"What depends on this?", "how do these two connect?"Links exist only in strings, configuration or other repositories

The pattern is that semantic search is best at finding a starting point, while symbol navigation and graph traversal are best at following connections from it. Lexical search fills the gaps where connections are made through text.

Similar text versus related codeWhere is the order total worked out, and what depends on it?SEMANTIC SEARCHREADME: order totalsFixture: totals.jsonDeprecated totalHelperCart UI total formatPricingEngine.totalCODE GRAPHPricingEngine.totalcalled by: OrderServicewrites: Order.totalimported by: InvoiceBuildertested by: pricing testsLEXICAL SWEEPtotal_cents in recon SQLdeliveryFee copy in invoicefee threshold in configorder.totalled eventno other matchesHybrid answer: the union, each item with the reason it was included
Komati Commerce asked one question three ways. Semantic search returned text that sounds like order totals, mostly the wrong files. The code graph returned the function and its real relationships, including the invoice builder through its import of the order model. A lexical sweep for the column and function names found the reconciliation SQL view, which the graph could not see because it refers to the total only by a column name, and confirmed the invoice builder's copy of the fee rule. The useful answer is the union, with the reason each item is in it. Results are illustrative.

Why similarity misleads on code

Four properties of code make embedding retrieval weaker than it is for prose.

Names are reused. A ten-year-old codebase has several functions called total, several classes called Order, and test fixtures that use every domain word in the system. Similarity rewards the files that use the words most, and test fixtures and documentation use them most.

Meaning is spread across files. The behaviour "an order's total" is not in one chunk. It is the composition of a controller, a service, an engine, two helper functions and a repository. A chunk containing any one of them is only partly similar to the question.

Chunking breaks structure. Splitting files by size cuts functions and loses the import list at the top. Chunking is a business decision covers this for documents. For code, splitting along syntax boundaries, by function or class, helps, but it does not restore the relationships between chunks.

Not all code is equal. Generated clients, vendored libraries, migrations and old copies kept "just in case" are all indexed alongside the code that runs. Similarity has no way to prefer the live implementation over a deprecated one with a better name, and a repository that is ten years old has plenty of both. A graph can at least show that nothing calls the deprecated helper, which is often the quickest way to rule it out.

None of this makes semantic search useless. It makes it the wrong tool for the dependency question.

Tracing the order-total rule

Here is the trace Komati Commerce's agent built once it combined the methods. It started with semantic search to find a likely entry point, resolved that to a symbol, and followed the graph outwards.

Tracing the order-total rule across layersAPIDOMAINPERSISTENCEDOWNSTREAMTESTSPOST /ordersOrdersControllerOrderService.placePricingEngine.totaldeliveryFee, vatRoundOrderRepository.saveorders.total_centsRecon SQL viewInvoiceBuilder recomputeown copy of the ruleCOVERS ENGINEPricing unit testsCOVERS APIOrder API contractMISSINGInvoice parity test
The trace Komati Commerce's agent built before changing the delivery fee. The rule lives in the pricing engine, the total is stored in one column, and two downstream consumers read it. One of them, the invoice builder in the fulfilment service, recomputes the total from line items with its own copy of the fee rule, and no test checks that the two totals agree. Names are illustrative.
  • API. POST /orders is handled by OrdersController, which calls OrderService.place.
  • Domain. OrderService.place calls PricingEngine.total, which applies promotions and then calls deliveryFee and vatRound. This is where the delivery-fee rule lives.
  • Persistence. OrderRepository.save writes the result to the orders.total_cents column.
  • Downstream. Two consumers read that column. The reconciliation view does so in SQL, and the graph could not see it, because the only link is a column name inside a SQL file. The invoice builder in the fulfilment service reads the order, then recomputes the total from line items with its own copy of the fee rule. The graph found the invoice builder through its import of the order model. A lexical sweep for total_cents and deliveryFee found the reconciliation view and confirmed the copy.
  • Tests. Pricing unit tests cover the engine. An API contract test covers the order endpoint. No test checks that the invoice total matches the order total, which is why the earlier defect reached finance before anyone noticed.

Each step used a different method. No single method would have produced the whole trace.

A hybrid pattern for large repositories

The general pattern is a pipeline, similar in shape to the retrieval pipeline for documents but with structure in the middle.

  1. Find entry points. Use semantic and lexical search together to find candidate symbols for the concept in the question. Hybrid search in plain language explains how to merge the two lists.
  2. Resolve to symbols. Turn file hits into named definitions using a language server, a SCIP index or the graph. Discard hits that are only test data or documentation, unless the question is about them.
  3. Traverse, with a budget. Follow callers, callees, imports and data edges for a fixed number of hops. Two or three hops are usually enough, while unlimited traversal returns half the repository.
  4. Sweep for string links. Take the names that emerged (columns, events, configuration keys, routes) and search for them literally across the repository and, where you can, the other repositories that share the database or the message bus.
  5. Assemble with reasons. Return the union, and attach to each item the reason it is there: "called by", "reads column", "matched string". An item without a reason is a candidate, not a finding.
  6. Read, then decide. The agent reads the files in that set before changing anything.

For most teams, the cost sits in steps 2 and 4. Language indexes need a working build for each language, and the string sweep needs knowledge of which names are worth searching for. Start with the languages and the seams where misses have hurt you.

When you need which

A relationship question does not always need a graph database. When you need a knowledge graph sets out a ladder for enterprise knowledge, and the same proportionality applies here.

  • A small repository that one team knows well. Lexical search and the editor's own language server are usually enough. Add structure when misses start to appear.
  • A large or old repository. A generated graph of calls and imports, rebuilt on merge, starts to pay for itself, especially for agents, which do not carry knowledge between sessions.
  • Many repositories sharing data. Code graphs stop at the repository boundary. You need a catalogue of who reads and writes which tables, topics and APIs, maintained alongside the code.
  • Regulated or safety-relevant paths. Prefer compiler-precise references over inferred edges, and keep the trace as evidence.

Evaluating code retrieval

A retrieval method is only as good as its results on your questions. Evaluate what was retrieved before what was written applies directly. Komati Commerce built its test set from version history.

StepWhat to do
Pick past changes30 merged changes that had a known downstream effect, including defects found later
Write the questionThe question an agent would have asked before each change
Record the truthThe files that eventually changed, plus any that should have changed
Run each methodSemantic only, lexical only, graph only, hybrid
MeasureRecall of the true files within the first 10 and 20 results, and files read per correct file
Look at the missesEach miss is either a missing edge, a string link or a cross-repository link

The point is not a leaderboard. It is to see, on your own code, which kind of link each method misses.

Key points

  • Semantic search finds code that sounds like the question. Graph traversal and symbol navigation find code that is connected to the answer.
  • Names are reused, meaning is spread across files, and chunking breaks structure, so similarity is weaker for code than for prose.
  • The reliable pattern is hybrid: find entry points, resolve to symbols, traverse within a budget, sweep for string links, and return each item with a reason.
  • Code graphs cannot see links made through SQL strings, configuration, events or other repositories. A literal sweep for those names is part of the method.
  • Evaluate retrieval on past changes from your own history, measuring recall of the files that really changed.

This is part 3 of Codebase Intelligence for AI-Assisted Engineering. Previous: What Graphify actually adds to a coding agent. Next: The code works. Nobody understands the system.

How CloudNala can help

CloudNala designs code retrieval for teams whose agents work across large or old repositories. We build a test set from your own change history, measure how semantic, lexical and structural retrieval perform on it, and put a hybrid pipeline in place with the language indexes, graph rebuilds and string sweeps your seams actually need, so the agent's starting set of files comes with reasons rather than similarity scores.


Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book a Codebase Intelligence Review or write to us at consult@cloudnala.co.za