5

I receive the following error when pushing my commits

remote: warning: File var/log/system.log is 57.82 MB; this is larger than recommended maximum file size of 50 MB remote: error: GH001: Large files detected. remote: error: Trace: 96d01231dffac3fbc3ba1eb2e9f01a93 remote: error: See http://git.io/iEPt8g for more information. remote: error: File var/report/752246136671 is 100.86 MB; this exceeds github's file size limit of 100 MB

I tried the following the commands listed step by step below:

git push -u origin master

cant find these in git files to commit when i typed git status.

Could you please let me know how to push my changes to repo without these errors ? I guess these files are in github index . I also tried git rm --cached var/log/system.log. but no results.

hitting my head to wall !

UPDATE 1 Kindly find the Gists here based on the two answers from experts below:

UPDATE 2 Kindly find below the git Log details for the both the files that i tried to remove:

ANSWER THAT WORKED

Please find the gist for the final answer that solved my issue

credits to git experts VonC, Holger Just and all other experts who have provided their inputs and ofcourse to stackoverflow.

Improve this question
0

3 Answers 3

Reset to default
8
+50

git rm or git rm --cached isn't enough to remove that file fir the history stored in your repo.

You need to:

  • use BFG Repo Cleaner, as suggested above.

    bfg --strip-blobs-bigger-than 1M my-repo.git

  • use git gc --agrressive --prune=now (after BFG), as detailed in "Reduce git repository size"

  • git push -f to force the new history on your remote repo.
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Is it possible that some other blob > 1MB (say those corresponding to seeds for the db) also get mistakenly deleted while trying to clean up for these 2 files? If yes, how to avoid that using bfg?
@mu無 Sure. You can add filters (--delete-files) to get a more precise action. Or do it the hard way, with a git filter-branch. I wanted to bring attention to the next step (the git gc) in order to force a more compact repack, and making the git push possible.
@VonC Thanks a lot for your answer. I tried the above GIT commands and it worked as you suggested. I have updated my question with the gist containing the git commands that solved my issue. Waiting for SO to allow me to award the bounty.
3

The problem at hand is that while you have removed the relevant files using new commits, there are blobs corresponding to the older commits for the same files in you repo, and the size of 2 of these objects is throwing you a warning and an error. To be able to push again, you need to remove the same as well.

While bfg-repo should work for most people to rectify the situation here, it requires java to be installed and configured on the system and that is not always available.

You have 2 files var/log/system.log and var/report/752246136671 which exceed the limit, so I would suggest using filter-branch for removing them for a git only solution:

git filter-branch --index-filter 'git rm --cached --ignore-unmatch var/log/system.log var/report/752246136671' --tag-name-filter cat -- --all

You might need to force push the changes later (using git push -f origin).

In general, it is a good practice to ignore the *.log, *.info and other log files from the git repo completely by using a *.log entry in your .gitignore file. You can similarly ignore the var/reports folder.

In case there are files which have already been committed to the repo and pushed earlier, you might need to use git rm --cached "*.log" to remove the same and commit those changes to untrack them permanently.

PS: This seems like a magento/joomla installation to me, in that case ignore the var folder completely in your .gitignore, you don't want objects from var/cache or var/session etc to be tracked in the repo.

Improve this answer

1 Comment

I had to try git filter-branch if i cant get the bfg working but BFG worked like a cake. and regarding your suggestions to ignore var folder is most welcome. I ignored it. Thanks again. This question gets the attention of good number of experts with different ways to resolve the problem. +1 for your answer.
2

The message contains information about two files. var/log/system.log generates a warning but it would be pushed. var/report/752246136671 is too large and thus prevents the push. You thus have to delete at least the latter file.

Before Github will let you push, you'll have to remove the file from all commits you want to push. It is not enough to just delete the file in a later commit after having it added before.

According to the article linked in the message, you can perform one of the two recommended operations:

If you have added the file in the most recent commit, you can change it to remove the file:

git rm --cached var/report/752246136671 # Stage our giant file for removal, but leave it on disk git commit --amend -CHEAD # Amend the previous commit with your change # Simply making a new commit won't work, as you need # to remove the file from the unpushed history as well git push # Push our rewritten, smaller commit

Or you can use The BFG to filter your repository and remove the file from all commits. This is required if you have added the file in the git history (as opposed to only in your most recent commit), you have to clean your history. Github will not allow to push the large file in any commit even if it is later removed again. This is because in this case, the file will still be part of the history and will thus bloat the repo.

You can install The BFG from https://github.com/rtyley/bfg-repo-cleaner/releases/latest.

The you can remove any indication of any files larger than 100MB by running this command:

cd /path/to/your/git/repo java -jar bfg.jar --strip-blobs-bigger-than 100M # Git history will be cleaned - files in your latest commit will *not* be touched

Note that this will change history of your repository, leading to a potential force-push. You might thus have to coordinate with your fellow developers.

Also, if you still need the file, you should make a backup before as you won't be able to restore it from git.

Improve this answer

4 Comments

I tried git rm --cached var/report/752246136671 and got the following error "fatal: pathspec 'var/report/752246136671' did not match any files". In fact i have removed the file manually from my server. I feel its somewhere in Git history but am not able to locate it. Kindly find this gist gist.github.com/haijerome/9405492 for more information.
I have updated my answer below with the git log details for both of these files i tried to remove.
As I said in my answer (and is said in the article the error message linked to and I quoted): If you have added the file in your git history, you have to clean it to remove the file completely from all commits. Github recommends the BFG. Please see the update to my answer.
Your answer helped me a lot. Thanks a lot. bfg is the key to solve the major problem. +1 for your answer and for your latest comment.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.