Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
Obsidian Wiki Ingestion LLM Pipeline
See how a local LLM pipeline autonomously ingests messy files into an organized Obsidian knowledge graph, bypassing auth-walls and creating cross-linked nodes.
An automated ingestion pipeline that monitors a local terminal staging area, parses messy chat dumps or unformatted text files, and compiles them into an organized, semantic markdown vault for Obsidian.
Instead of dealing with manual ClickOps to manage a knowledge base, a local LLM agent (Gemma 4 9B via Ollama) routes incoming files to their correct semantic home based on an atomic directory map. The pipeline automates the entire lifecycle: cleaning and normalising metadata into strict snake_case tags, auto-regenerating the central vault index map on every execution pass, and running an autonomous extraction loop using BeautifulSoup to scrape hyperlinks. Social media auth-walls are bypassed by routing X.com links through an FXTwitter data proxy, after which a secondary text-synthesis agent processes the raw HTML under a strict context-reset block to spawn independent, cross-linked source nodes.
What I will show live: I will drop raw, unformatted code snippets and files containing GitHub Gists and Twitter threads into a staging directory via the terminal. I’ll execute the script, trace the live extraction logs and proxy bypasses, and show the local LLM streaming structural classification decisions on the fly. Finally, I’ll toggle over to the Obsidian UI graph view to demonstrate the newly compiled nodes instantly locking into the centralized map in real-time.