LLM-Based Hierarchical Topic Modeling Tool
The LLM-Based Hierarchical Topic Modeling Tool normalizes raw string values in JSONL files into reusable hierarchical categories using plain-language instructions. It requires Python, an API key for a LiteLLM-supported model, and a Rust compiler for installation, with optional Conda environment support. Users configure model keys in config.py or a .env file before running the provided scripts to transform the data.
- ▪The tool groups related labels into a taxonomy with broad topics at the top and specific subtopics below, using LLM assistance to map raw strings to normalized category paths.
- ▪Installation requires Python 3.9+, an API key for a LiteLLM provider such as OpenRouter, and a Rust compiler to handle any package builds that need compilation.
- ▪Configuration is done by editing config.py or creating a .env file with the appropriate API key, after which the scripts can process JSONL files to produce normalized outputs.
Opening excerpt (first ~120 words) tap to expand
Instruction-Driven Hierarchical Topic Normalizer Hierarchical topic modeling groups related labels into a taxonomy: broad topics at the top and increasingly specific subtopics below them. This tool is a practical, LLM-assisted version for JSONL data: it turns inconsistent raw strings into a reusable hierarchy that you steer with plain-language instructions. For example, it can map bad, neg, and negative to negative, or map search to Features|Search. It is designed for extracted Reddit-study data, but works with any JSONL file containing strings, nested objects, and arrays of objects. The input is a JSONL file with raw string values. The output is another JSONL file with the same records and structure, except that mapped values have been replaced by their normalized category paths.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.