Twenty-Five Hours of Concept Extraction

A planning-heavy week, plus two papers built to PDF.

Concept-guided RAG

I set up the real experiment: concept extraction with Llama 3.2 running locally through Ollama, at temperature 0, under its own experiment ID so it can't be confused with the mock run. Before launching it I measured the cost. SciFact needs 9,672 LLM calls (8,563 document chunks plus 1,109 queries), and one call took 9.3 seconds on my laptop, so a full pass would take about 25 hours. The LLM cache makes the run resumable.

That number is a useful result on its own. Document-side concept extraction is expensive to index, which makes the query-only variant (extract concepts from the query, match against raw text) a much more practical option if it holds up.

Agentic DGG conversion

I scoped a new direction: translating existing DGG models written in Mathematica into FoxFlow and LaTeX through the typed model hub. The first obstacle is input, not output. The notebooks store typeset expression structures rather than plain source, so I'm weighing plain-text exports, the Wolfram Engine, or a dedicated parser. I also found that cycles are a shared limitation: the LaTeX renderer draws them incorrectly and my FoxFlow emitter refuses them, so we need a common notation for them.

Lost in the Middle and RAG saturation

Both papers now build to PDF from source with pandoc and tectonic. The retrieval saturation paper comes to 13 pages with no undefined references. The remaining steps are a venue decision and making the code public.