A lightning-fast utility for preparing and organizing your code for use with LLMs like Claude Projects. LMPrep will collect and rename all of your project files to a flat directory, but preserving the structure within the filenames.

For example, a file at src/models/user.py will be renamed to src^models^user.py in the output directory. Be sure to tell the LLM that your files are structured this way!

• Smart File Organization: Automatically flattens complex directory structures while preserving path information in the filenames and in a file tree
• Configurable Filtering: Specify which file extensions to include in your dataset to limit context size
• Path Preservation: Uses customizable delimiters to maintain original path information in filenames
• Git-Aware: Respects .gitignore patterns to exclude unwanted files or secrets
• Flexible Output: Generate individual files or create a zip archive
• Visual Tree View: Visualize your source and output file structure, or send the file tree to the LLM
• Fast & Efficient: Written in Rust for maximum performance

The easiest way to install LMPrep is to get it from PyPi:

pip install lmprep

wheels are built for Windows, Linux, and MacOS.

1. Download the latest release for your platform from Releases:

   • Windows: lm-x86_64-pc-windows-msvc.zip
   • Linux: lm-x86_64-unknown-linux-gnu.tar.gz
   • macOS: lm-x86_64-apple-darwin.tar.gz

2. Install the binary:

Linux/macOS:

# Extract and copy binary tar xzf lm-x86_64-*-*.tar.gz sudo mv lm /usr/local/bin/ # Create config file curl -O https://raw.githubusercontent.com/bcherb2/lmprep/main/src/config-example.yaml mv config-example.yaml ~/.lmprep.yml

Windows (in PowerShell, run as Administrator):

# Extract and copy binary Expand-Archive lm-x86_64-pc-windows-msvc.zip New-Item -ItemType Directory -Force -Path "C:\Program Files\lmprep" Move-Item -Force lm.exe "C:\Program Files\lmprep\lm.exe" $env:Path += ";C:\Program Files\lmprep" # Create config file Invoke-WebRequest -Uri "https://raw.githubusercontent.com/bcherb2/lmprep/main/src/config-example.yaml" -OutFile "$env:USERPROFILE\.lmprep.yml"

3. Verify installation:

lm --help

If you have Rust installed, you can build from source:

git clone https://github.com/bcherb2/lmprep cd lmprep cargo build --release

The binary will be in target/release/lm (or lm.exe on Windows). Follow step 2 above to set up the config file.

# Create a default config file in the current directory lm --init-config # Organize files in current directory lm . # Organize files from a specific directory lm /path/to/source # Use a custom config file lm . -c /path/to/.lmprep.yml # Create a zip archive instead of of individual files lm . --zip

Create a .lmprep.yml file in your home directory to customize behavior, or create one in your project root directory. Here's an example:

allowed_extensions: - py - rs - md - txt ignored_directories: - node_modules delimiter: "^" subfolder: context zip: false tree: true respect_gitignore: true

NOTE: The install script will create a default config file at ~/.lmprep.yml

lm [OPTIONS] [SOURCE] Arguments: [SOURCE] Source directory to organize files from [default: .] Options: -c, --config <FILE> Path to config file -s, --subfolder <NAME> Override the subfolder name from config -z, --zip Create a zip file instead of individual files -t, --tree Show file tree of source and output -v, --verbose Show more detailed output during processing --init-config Create a default config file in the current directory -h, --help Print help -V, --version Print version

To set up for development:

1. Clone the repository
2. Run ./dev-setup.sh

This will build the Rust binary, set up the correct directory structure, and install the package in development mode.

When you make changes to the Rust code, run ./dev-setup.sh again to rebuild and reinstall. Python changes will be picked up automatically due to the development install.

• Code Analysis: Organize your code into a flat structure while preserving context (works especially well with Claude Projects)
• Document Processing: Organize and prepare document collections for processing, logs, etc.
• Version Control: Easily create clean snapshots of your codebase for archival in zip format

1. Install Rust using rustup
2. Clone the repository:

   git clone https://github.com/bcherb2/lmprep.git cd lmprep

3. Build the project:

   cargo build --release

4. The binary will be available at target/release/lm, copy it and add it to your PATH
5. Create the .lmprep.yml file in your home directory or project root

NOTE: see install/BUILD.md for more in depth building instructions.

Q: Why use LMPrep instead of just copying files? A: LMPrep preserves directory structure information in filenames, making it easier for LLMs to understand file relationships and context. Sure, you can do this manually, but it gets tedious.

Q: How does path flattening work? A: A file at src/models/user.py becomes src^models^user.py in the output directory (using default delimiter). Changing the delimiter to + would result in src+models+user.py.

Q: Can I exclude certain files or directories? A: Yes! LMPrep respects .gitignore patterns and allows you to specify allowed file extensions.

Q: Is it safe to use on large directories? A: Yes! LMPrep is written in Rust for performance and memory efficiency, making it suitable for large datasets.

Contributions are welcome! Please feel free to submit a Pull Request.

This project is licensed under the MIT License - see the LICENSE file for details.