9

I want to be able to run my tests for my project with the current state of the index, ignoring non-committed working changes (I later plan to add this to a pre-commit hook). However, I am having trouble figuring out how to remove and then restore the non index changes in a way that never causes merge conflicts. I need this because it is being run by a script, so it shouldn't alter the repository state when finished.

git stash --include-untracked --keep-index and git stash pop come close, but in many cases it results in merge conflicts, even if no changes where made between the two commands.

For example:

mkdir project; cd project; git init .; # setup the initial project with file.rb and test.rb cat > file.rb <<EOF def func() return 42 end EOF cat > test.rb <<EOF #!/usr/bin/env ruby load './file.rb' if (func() == 42) puts "Tests passed" exit 0 else puts "Tests failed" exit 1 end EOF chmod +x test.rb git add . git commit -m "Initial commit" # now change file.rb and add the change cat > file.rb <<EOF def func() return 10 + 32 end EOF git add file.rb # now make a breaking change to func, and don't add the change cat > file.rb <<EOF def func() return 20 + 32 # not 42 anymore... end EOF

From here I want to run the tests against the current state of the index, and the restore the uncommitted changes. The expected result is for the tests to pass, as the breaking change wasn't added to the index.

The following commands do NOT work:

git commit --include-untracked --keep-index ./test.rb git stash pop

The the problem occurs in the git stash pop - a merge conflict occurs.

The only other solution I could think of was to make a temporary commit, then stash the remaining changes, then rollback the commit with git reset --soft HEAD~, then pop the stash. However that is both cumbersome, and I'm not sure how safe that is to run in a pre-commit hook.

Is there a better solution to this problem?

Improve this question
2
  • Is it because git stash --include-untracked --keep-index doesn't actually do anything and git stash pop is popping off some other previously stashed patch? The problem is that git stash will happily exit with a status of 0, even if nothing had been stashed. You need to look out for the "No local changes to save" message coming from git stash. It's a bit fragile, but I think it's the best you can do short of getting a change introduced to Git to make git stash behave a little differently. Commented Dec 15, 2012 at 8:24
  • No, it is definitely stashing something, and still causing conflict problems. I believe it is because git stash --keep-index still stashes the index even if it doesn't remove the changes from the working copy (going by similar questions). Although the problem with git stash not reporting that the stash wasn't created (or the fact that it doesn't have an option to force a stash to be created even if empty) is another annoyance. Commented Dec 15, 2012 at 12:22

3 Answers 3

Reset to default
9

Like you, I run

git stash --keep-index --include-untracked

I can then run tests and so on.

The next part is tricky. These are some things I tried:

  • git stash pop can fail with conflicts, which is unacceptable.
  • git stash pop --index can fail with conflicts, which is unacceptable.
  • git checkout stash -- . applies all tracked changes (good), but also stages them (unacceptable), and does not restore untracked files from the stash (unacceptable). The stash remains (fine -- I can git stash drop).
  • git merge --squash --strategy-option=theirs stash can fail with conflicts, which is unacceptable, and even when it doesn't conflict it does not restore untracked files from the stash (unacceptable).
  • git stash && git stash pop stash@{1} && git stash pop (trying to apply the changesets in reverse order) can fail with conflicts, which is unacceptable.

But I found a set of commands which does what we want:

# Stash what we actually want to commit git stash # Unstash the original dirty tree including any untracked files git stash pop stash@{1} # Replace the current index with that from the stash which contains only what we want to commit git read-tree stash # Drop the temporary stash of what we want to commit (we have it all in working tree now) git stash drop

For less output, and condensed into one line:

git stash --quiet && git stash pop --quiet stash@{1} && git read-tree stash && git stash drop --quiet

As far as I'm aware, the only thing this doesn't restore is files which were added in the index and then deleted from the working tree (they'll end up added and present) and files which were renamed in the index and then deleted from the working tree (same outcome). For this reason we need to look for files which match these two cases with a line like git status -z | egrep -z '^[AR]D' | cut -z -c 4- | tr '\0' '\n' before the initial stash, and then loop through and delete them after restoring.

Obviously you should only be running the initial git stash --keep-index --include-untracked if the working tree has any untracked files or unstaged changes. To check for that you can use the test git status --porcelain | egrep --silent '^(\?\?|.[DM])' in your script.

I believe this is better than the existing answers -- it doesn't need any intermediate variables (other than whether the tree was dirty or not, and a record of which files need to be deleted after restoring the stash), has fewer commands and doesn't require garbage collection to be switched off for safety. There are intermediate stashes, but I'd argue this this exactly the kind of thing they're for.

Here's my current pre-commit hook, which does everything mentioned:

#!/bin/sh # Do we need to tidy up the working tree before tests? # A --quiet option here doesn't actually suppress the output, hence redirection. git commit --dry-run >/dev/null ret=$? if [ $ret -ne 0 ]; then # Nothing to commit, perhaps. Bail with success. exit 0 elif git status --porcelain | egrep --silent '^(\?\?|.[DM])'; then # There are unstaged changes or untracked files dirty=true # Remember files which were added or renamed and then deleted, since the # stash and read-tree won't restore these # # We're using -z here to get around the difficulty of parsing # - renames (-> appears in the string) # - files with spaces or doublequotes (which are doublequoted, but not when # untracked for unknown reasons) # We're not trying to store the string with NULs in it in a variable, # because you can't do that in a shell script. todelete="$(git status -z | egrep -z '^[AR]D' | cut -z -c 4- | tr '\0' '\n')" else dirty=false fi if $dirty; then # Tidy up the working tree git stash --quiet --keep-index --include-untracked ret=$? # Abort if this failed if [ $ret -ne 0 ]; then exit $ret fi fi # Run tests, remember outcome make precommit ret=$? if $dirty; then # Restore the working tree and index git stash --quiet && git stash pop --quiet stash@{1} && git read-tree stash && git stash drop --quiet restore_ret=$? # Delete any files which had unstaged deletions if [ -n "$todelete" ]; then echo "$todelete" | while read file; do rm "$file" done # Abort if this failed if [ $restore_ret -ne 0 ]; then exit $restore_ret fi fi fi # Exit with the exit status of the tests exit $ret

Any improvements welcome.

Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

Shouldn't your regex be '^(\?\?|MM|\sM)'? To include modified yet completely unstaged files.
I think you're absolutely right. Editing my answer. (Also changing the egrep command so that wc isn't needed.) Additionally we need to look for D in the second column (file was staged but then deleted from the work tree).
In fact the case of AD isn't handled at all by this script. The file's not deleted again during the restore step. I'm working on handling that.
@aiham, I've updated the answer. I'd love if it could be simplified at all. I don't much like how the AD and RD cases are handled but it's the best I could come up with. It might do weird things in the RD case if something was renamed from a filename which looks like a status message such as MM somefile, due to crapness of git-status.
I had one problem with this script. When there are no renamed/deleted files (i.e. $todelete is empty), echo "$todelete" | while read file; do results in a single iteration with $file set to an empty string. This, in turn, leads the script to fail with: "rm: cannot remove '': No such file or directory". Solution that works for me is to pipe echoed $todelete through grep -v "^$" before sending it to while read.
|
4 
$ git config gc.auto 0 # safety play $ INDEX=`git write-tree` $ git add -f . $ WORKTREE=`git write-tree` $ git read-tree $INDEX $ git checkout-index -af $ git clean -dfx $ # your tests here $ git read-tree $WORKTREE $ git checkout-index -af $ git clean -dfx $ git read-tree $INDEX $ git config --unset gc.auto $ # you're back.

The git clean manpage for -x rather elliptically suggests this solution

Improve this answer

13 Comments

I should have used read-tree not reset, my apologies. I fixed it above and removed the (now-)redundant commits when setting INDEX and WORKTREE.
You are right, the ignored files are added back after it is done. Also, adding git checkout $WORKTREE -- 'db/*.sqlite3' just before the # your tests here line fixed the problems with missing databases during the test run. This way also seems more robust with me explicitly choosing what ignored files to use for the test (making bad commit less likely). Thanks for the help!
@kjo You'd need to plug the git added objects into the reference machinery, commit-treeing the written trees and update-refing the commits to some ref -- much as stash does. N.B.: stash keeps the stack in its reflog, and that specific log is protected from ordinary expiration
@kjo actually, come to think of it, you wouldn't need to commit the trees. refs can be to anything at all. So for a first cut you could touch .git/logs/refs/mystash once, then in the above say git update-ref refs/mystash $INDEX; git update-ref refs/mystash $WORKTREE before running the tests, remove the gc.auto manipulation, and in most repos you're good to go.
@kjo or if you're willing to dispense with the stackability, git update-ref refs/preserve-this-tree $WORKTREE; git update-ref refs/preserve-that-tree $INDEX before running the tests will be enough.
|
0

Here is the best solution I have figured out so far. It works in the git hooks pre-commit and commit-msg. I've tested it when there is:

  • Only staged changes
  • Only unstaged changes
  • Both staged and unstaged changes
  • No changes

In all three cases it works correctly. The big downside is it is pretty hacky, creating and removing temp commits and stashes. 

#!/bin/bash git commit -m "FOR_COMMIT_MSG_HOOK" -n commit_status=$? # git stash save always returns 0, even if it # failed creating a stash. So compare the new and old git stash list # output to see if a stash was created current_stash_count="$(git stash list)" git stash save -q -u "FOR_COMMIT_MSG_HOOK" new_stash_count="$(git stash list)" echo "##### Running tests #####" # put testing code here if [[ $commit_status -eq 0 ]]; then git reset --soft 'HEAD~' fi if [[ "$current_stash_count" != "$new_stash_count" ]]; then git stash pop -q fi exit $result
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.