Repository navigation
Will you sell your GPT-4-32k API? If you would like to, please contact me. #43
Copy link
Copy link
Open
Description
Activity
EASY!
Just do this:
- Go to:
-
Create an account
-
Connect Capy to GitHub
-
Select the correct GitHub (forked) repo
-
Send a text prompt to the Capy AI chat something like:
"From the github repo I need you to do a complete rewrite of
gpt-repository-loaderas a modular Python package, addressing all open issues and pull requests with a production-ready implementation."-
Wait about 10-20 minutes for Capy to complete
-
Download new updated files from your forked repo
-
Welcome to the world of AI (can be done for any issue / bug, just change the text prompt to what needs to be done)**
This PR implements a complete rewrite of
gpt-repository-loaderas a modular Python package, addressing all open issues and pull requests with a production-ready implementation.Package & Tooling
- Add
pyproject.tomlwith[project]metadata, optional dependencies (git, tokens, clipboard, web), and entry pointsgpt-repository-loader/gpt-repo(Add pyproject.toml #63, 🚀 Add pipx installation support via pyproject.toml #64, Pipx installation #55). - Add
.devcontainer/devcontainer.jsonfor VS Code remote development (Create devcontainer.json #44). - Add GitHub Actions CI workflow testing Python 3.8–3.12 on Ubuntu, macOS, and Windows.
- Remove legacy
gpt_repository_loader.pyscript; refactor intogpt_repository_loaderpackage with modules.
Ignore Engine & Binary Detection
- Implement full gitignore-style pattern matcher (
ignore.py): negation patterns (!), leading slash anchoring, directory-only trailing slashes,**path matching, character classes, comments, and escaping (Add negated-pattern support for .gptignore #46, ignore .gitignore along with .gptignore #24, ignore.tox#21, .gptignore should appropriately interpret leading slashes #40). - Merge
.gitignoreand.gptignoreacross the tree, with package-shipped fallback and CLI--exclude/--includeoverrides (ignore .gitignore along with .gptignore #24, adding a number of features to understand what is being ignored #52, Don't include non relevant files in the scan #56). - Detect binary files via extension allow/deny lists, magic-byte sniffing (PNG, JPEG, GIF, ELF, gzip, zip, etc.), and NUL byte heuristics (Ignore Binary Files #53, An option to compress the file contents #37, Ignore .DS_Store #42, added some essential terms to ignore #47).
- Add
--show-ignoredand--json-reportto list excluded files with matching patterns and reasons (adding a number of features to understand what is being ignored #52).
Token Counting, Limits & Chunking
- Integrate
tiktokenfor accurate token counting when installed; deterministic 4-char fallback otherwise (Add warning about token length being greater than GPT limits #26, Add a token limit flat to the command. #41). - Add
--token-limitflag (per-model defaults via--model), warn when exceeded, and exit code 2 for CI (Add a token limit flat to the command. #41). - Implement
--chunkto split oversized bundles into<output>.chunkNN.txtfiles (In many applications, output.text is too long to input #35). - Track per-file and total token counts, include stats in
--preamble-statsand Web UI.
Compression & Output Formats
- Add per-file compression via
--compress/--encoding-mode gzip+base64; reverse loader decodes transparently (An option to compress the file contents #37). - Support
--format zipto produce a ZIP file containing bothbundle.txtand raw file tree (gpt-4 doesn't seem to think this is better than a zip #48). - Support
--format jsonto emit a JSON document with files, ignored list, warnings, and metadata. - Auto-upgrade to base64 encoding for files containing literal
----or--END--lines.
Remote Repositories & Git Integration
- Accept GitHub/GitLab/Bitbucket/Codeberg URLs,
git@SSH strings, and downloadable archives viaurl_loader.py; clone via GitPython or extract to temp dir (Support URLs #36, Use GitPython library #32, Add flag do the reverse process, write the output file(s) back to the repository #30). - Add
--clone-depthand--branchfor shallow clones. - Use GitPython for
--tracked-onlymode, repo introspection (HEAD SHA, branch, dirty flag), andgit check-ignoreintegration (Use GitPython library #32, Don't include non relevant files in the scan #56).
Metadata
Metadata
Assignees
Labels
No labels
Will you sell your GPT-4-32k API? If you would like to, please contact me.