614

Some time ago I added info(files) that must be private. Removing from the project is not problem, but I also need to remove it from git history.

I use Git and Github (private account).

Note: On this thread something similar is shown, but here is an old file that was added to a feature branch, that branch merged to a development branch and finally merged to master, since this, a lot of changes was done. So it's not the same and what is needed is to change the history, and hide that files for privacy.

Improve this question
8
  • 3
    You would have to rewrite history. For example git rebase then git push -f Commented May 3, 2017 at 14:07
  • The filter-branch method described in the suggested duplicate will do what you want. Commented May 4, 2017 at 6:00
  • 1
    Also stackoverflow.com/a/17890278 which points to the BFG which can be faster than using git filter-branch Commented May 4, 2017 at 6:05
  • but just go faster and do the same, and need to use java i read @Hasturkun Commented May 5, 2017 at 10:16
  • 4
    NONE of theses solutions worked... GIT should have a command for this... it's so useful... Commented May 11, 2021 at 22:49

14 Answers 14

Reset to default
610

I have found this answer and it helped:

git filter-branch --index-filter 'git rm -rf --cached --ignore-unmatch path_to_file' HEAD
Improve this answer
Sign up to request clarification or add additional context in comments.

22 Comments

Warning: This creates a ton of commits and causes divergence. You probably have to force push after, but I was too scared.
Seconding what @sudo said but this did work for my fresh branch that I accidentally committed .env to. Quick and to the point solution.
You can also specify a range of commits as the last argument. If the commit in question was recent, do <previous_hash>..HEAD and save some time.
after this it works only for me git push --force
Current versions of Git say this about filter-branch: "WARNING: git-filter-branch has a glut of gotchas generating mangled history rewrites. Hit Ctrl-C before proceeding to abort, then use an alternative filtering tool such as 'git filter-repo' (github.com/newren/git-filter-repo) instead. See the filter-branch manual page for more details; to squelch this warning, set FILTER_BRANCH_SQUELCH_WARNING=1."
|
335

git-filter-repo

git recommends using the third-party add-on git-filter-repo (when git filter-branch command is executed). There is a long list of reasons why git-filter-repo is better than any other alternatives, my experience is that it is very simple and very fast.

This command removes the file from all commits in all branches:

git filter-repo --invert-paths --path <path to the file or directory>

Multiple paths can be specified by using multiple --path parameters. You can find detailed documentation here: https://www.mankier.com/1/git-filter-repo

Improve this answer

25 Comments

i get error: git: 'filter-repo' is not a git command. See 'git --help'.
@cikatomo It's a third-party tool, you have to install it github.com/newren/git-filter-repo/blob/main/INSTALL.md
This worked but it removed .git so I wonder why not just remove .git manually and re-init?
@cikatomo Another way to install pip install git-filter-repo.
Be very careful with this tool. I almost lost a lot of work. Read the docs and consider backing up all your files to a zip file outside of the directory before proceeding. The tool is bad about not giving you enough warning. It's very much an RTFM situation.
|
187

If you have recently committed that file, or if that file has changed in one or two commits, then I'd suggest you use rebase and cherrypick to remove that particular commit.

Otherwise, you'd have to rewrite the entire history.

git filter-branch --tree-filter 'rm -f <path_to_file>' HEAD

When you are satisfied with the changes and have duly ensured that everything seems fine, you need to update all remote branches -

git push origin --force --all

Note:- It's a complex operation, and you must be aware of what you are doing. First try doing it on a demo repository to see how it works. You also need to let other developers know about it, such that they don't make any change in the mean time.

Improve this answer

16 Comments

after rewrite the entire history, for keep the changes to repository (github) what must be done?
Why does your suggestion use --tree-filter rather than --index-filter like in @PetroFranko's answer?
holy crap, it worked! I mean it was really really simple. I've done it the hard way before, but this was much easier. Tip: the path needs to be relative.
thank you for the "if recent, try to rebase" solution - my "problem" file was added on a separate branch in a recent commit, so i could rewrite history using a commit pre-dating the file addition
@summerisbetterthanwinter copy it before removing may be?
|
89

Remove the file and rewrite total history with new commit hashes:

There are two ways:

  1. Using git-filter-branch:
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch <path to the file or directory>' --prune-empty --tag-name-filter cat -- --all
  1. Using git-filter-repo:
pip3 install git-filter-repo git filter-repo --path <path to the file or directory> --invert-paths
  • now force push the repository: git push origin --force --all
  • tell your collaborators to rebase git pull --rebase.
Improve this answer

11 Comments

@alper you need to replace PATH-TO-YOUR-FILE-WITH-SENSITIVE-DATA with the file to remove eg: README.md if you want to remove it.
For git filter-repo: I am getting following message : Aborting: Refusing to destructively overwrite repo history since this does not look like a fresh clone. (expected freshly packed repo) Please operate on a fresh clone instead. If you want to proceed anyway, use --force.. If I force it I get following: fatal: 'origin' does not appear to be a git repository fatal: Could not read from remote repository.
git filter-branch approach worked for me on mac, while filter-repo approach was removing remote origin
This worked, but I forgot to back up the file first, and now it's gone. :-(
You can specify file path like: "*/a.txt" if you happen to have the same file in different paths or files' path have changed in different commits.
|
56

I read this GitHub article, which led me to the following command (similar to the accepted answer, but a bit more robust):

git filter-branch --force --index-filter "git rm --cached --ignore-unmatch PATH-TO-YOUR-FILE-WITH-SENSITIVE-DATA" --prune-empty --tag-name-filter cat -- --all
Improve this answer

5 Comments

It works better than accepted answer but It also deletes file in locale. Take copy of it before this if you don't want to waste time for rewrite.
this helped me on mac, while filter-repo was removing remote origin
my local file didn't get removed maybe because it was entered in .gitignore
don't forget the "git push --force" after all your purging is done.
Works like a charm, thanks. I also added -r in case of deleting a directory
18

  • First of all, add it to your .gitignore file and don't forget to commit the file :-)

  • You can use this site: http://gitignore.io to generate the .gitignore for you and add the required path to your binary files/folder(s)

  • Once you added the file to .gitignore you can remove the "old" binary file with BFG.

How to remove big files from the repository

You can use git filter-branch or BFG.

BFG Repo-Cleaner

an alternative to git-filter-branch.

The BFG is a simpler, faster alternative to git-filter-branch for cleansing bad data out of your Git repository history:

*** Removing Crazy Big Files***

  • Removing Passwords, Credentials & other Private data

Examples (from the official site)

In all these examples bfg is an alias for java -jar bfg.jar.

# Delete all files named 'id_rsa' or 'id_dsa' : bfg --delete-files id_{dsa,rsa} my-repo.git

Enter image description here

Improve this answer

4 Comments

Is it a third party cleaner?
Is it secure to use?
Indeed, a very "old" tool which is being used by the community for few years. The source is in GitHub so you and the community can browse it.
I just find out that GitHub does not remove deleted commits in case when users request them to run garbage collector, (stackoverflow.com/questions/34582480/remove-commit-for-good/…). I am just get lost where when we use 3rd party tools like GitHub whatever committed, we will always need to ask them to remove it, which is not cool
11

Using the BFG Repo-Cleaner package is another viable alternative to git-filter-branch. Apparently, it is also faster...

Improve this answer

Comments

5

Remove file(s)

bfg --delete-files YOUR-FILE-WITH-SENSITIVE-DATA

Replace all text listed in passwords.txt wherever it can be found in your repository's history, run:

bfg --replace-text passwords.txt

After that you need topush your changes to GitHub/GitLab/BitBucket

git push --force

More about the BFG tool here

Furthermore, since this technique it will rewrite your repository's history, which changes the SHAs for existing commits, you should alter and any dependent commits. So merge and close all open PRs!

Improve this answer

Comments

2

The following commands should be applied one by one in each project in order to remove the history for a specific file, but you have to take backup from the project at the beginning, because the file will be deleted

  • git filter-branch --index-filter "git rm --cached --ignore-unmatch **ProjectFolderName**/src/main/resources/**application-prod.properties**" HEAD

  • git push origin --force --all

  • git update-ref -d refs/original/refs/heads/master

..........................................................................

  • git filter-branch --index-filter "git rm --cached --ignore-unmatch **ProjectFolderName**/src/main/resources/**application.properties**" HEAD

  • git push origin --force --all

  • git update-ref -d refs/original/refs/heads/master

Improve this answer

Comments

1

This command will delete any zip file in your history.

git filter-branch --force --index-filter "git rm --cached --ignore-unmatch *.zip" --prune-empty --tag-name-filter cat -- --all

Replace zip with your files format.

Improve this answer

Comments

0
  1. First install BFG Repo-Cleaner, for example if you use Ubuntu 22.04 or similar:

sudo snap install bfg-repo-cleaner

  1. Then clone your GitHub repository as a mirror, like this:

git clone --mirror [email protected]:YOU/REPO.git

  1. Navigate into folder:

cd REPO

  1. Now you can delete file from Git history, for example:

bfg --delete-files config.php

  1. Now push back changes to GitHub:

git push or git push --force

Improve this answer

Comments

0

I had the same "you must be a on a branch" message. I went to the top of the repository in the Git UI, and just below the repository name I found a pull down where I could change the branch from default to main. This fixed the issue for me.

Enter image description here.

Improve this answer

Comments

0

Squashing your commits from the file creation to its removal is a simple solution to this if it fits your particular scenario. Obviously not suitable if you need to keep the history between these two points.

Improve this answer

Comments

-3

Add the file pattern to your .gitignore. Example:

*.env

Then remove the cached file information using this command(zsh):

bash -c 'git rm --cached *.env'

Improve this answer

2 Comments

Why was this downvoted?
Not sure, I still use this method

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.