Techniques to avoid entangling git merges when working on an extensive change

The model isn't too bad and in most cases is not really avoidable.

Few suggestions:

1. Do you coordinate any work with other developers? We use Git at work and generally people modify what they need to where they need to but sometimes, if someone says, hey I'm in this area doing so and so. Other developers might hold off working on tickets that touch the same thing.
2. If I'm working on a large feature that requires a long-lived feature branch. I do not merge in on daily basis but definitely frequently. As you pointed out, doing several smaller merges does reduce the surprise factor of OMFG this looks completely different.
3. Use graphical tools. I'm aware that some people still run "git merge", gut a bunch of text files and then manually reconcile them. That sounds absolutely aweful and I would never wish that on anyone. In my previous job, we used Perforce, and when I joined my current team where we use git, the first time I had to do a 3-way merge, I went online and found this blog post. P4merge definitely makes the process of reconciliation so much easier.
4. Another thing you could consider is breaking your large task into smaller ones. Especially if you need to make refactoring or architecture changes. If you rearrange stuff first, make sure it works and push that to common develop branch, you'll get much less surprises if the rest of your work is in few specific areas of the product.
5. To build a bit on previous point, this might also depend on how well your software is designed. Having a modular approach and keeping Open-Close Principle in mind when writing code will definitely help to keep developers from stepping on each others toes too much. However, if you have many monolithic classes that often have to be modified any time you fix a bug or add a feature, that will contribute to your frustration levels. In that case, fixing design is a much bigger question than just git workflow.

In general, the more often you'll merge your changes, the fewer conflicts you'll have in total and thus the less total time will be spent on the merges.

Of course there will be no conflicts when merging independent parts of code. And that does not necessarily say anything about files. If a file contains two functions, one person changes one of them and another changes the other, the automatic merge will resolve it fine in a split second.

So you should:

1. Work on improving general design of the code. The more modular the code is, the easier it is to work on different parts of it without interfering with each other.

2. Coordinate tasks in the team so that tasks that need to touch the same part of the code are not done at the same time. This will of course be made easier by keeping the code modular.

3. Split the tasks to smaller chunks that can be pushed to main development branch (develop or next or whatever you call it). Again keeping the code well modularized will allow you to do localized refactoring in preparation for some feature that does not break anything else and can be pushed to everybody.

4. If you need to modify the same part of the code at the same time and can't make the tasks small, merge between the people working on it during the work, so the later changes are done with the earlier work of the other colleagues already included and don't need merging. Of course you have to merge at points where the code is not broken to the point that the others could not test their parts (but during any larger task you need intermediate points where the code works enough for testing partial changes anyway).

   1. If you can live with all the tasks to only be delivered when all are done, just set up a shared develop branch and push your partial changes there and keep up to date with it.
   2. If you know which feature has to be delivered first, you can keep merging one way from the feature branch of the feature to be done earlier to the one for later. So the first feature can still be delivered immediately when finished while the later saves some merge trouble by merging early.
   3. If you need to keep the features separate, you can still get some help by periodically merging all of them together, resolving any conflicts, recording the resolutions with git rerere(1) and discarding the merge result again. The next merge attempt will only show any new conflicts and when some of the features are finally integrated, the resolutions can again be reused to bring the remaining feature branches up to date.

It should be noted that you would have the same problems in any other version control system, because the funamental working of a 3-way merge is always the same. Git will at least let you easily back off and retry if you make any mistake using the powerful git reflog(1). And the git rerere(1) also helps now and then.

Establish coding standards for your team

It may seem trivial, but I learned to love DVCS (Git, Mercurial) the day our team decided to set coding standards, which were then tuned in our IDEs.

For instance, if you code like this:

// tabs indents if($you+$me==$wow) { $answer='It\'s gonna be great'; return $answer; } 

Whereas you colleague codes like that:

// spaces indents if ($me + $you = $hey) { $answer = "Let\'s talk a bit, won't you?"; return $answer; } 

You're undergoing a LOT of merging problems. So if that's not the case yet, take the time to establish coding standards in your team, and customize your IDEs accordingly to automatize everything.

Another advantage of coding standards

A little off-topic (that's why it's separated from the rest) but useful.

If your team's manager manages your team well, your team probably organizes frequent collaborative code reviews. For example, friday afternoon is code review time.

Establishing coding standards ensures that nobody is pecking anybody else's way of coding, or is having trouble understanding a snippet because of a coding style he is not used to. So, you can rest assured that the code review time will be fully used to discuss real programming matters, not punctuations matters.

Whether you are using git or anything else, if people are making incompatible changes to the codebase in parallel, merging your work will always be painful. This has nothing to do with the git model in particular.

If you find merging is too painful, likely something is wrong with the way you and the other developers are coordinating your efforts.

For example, refactorings which touch a lot of lines in a lot of different files (such as renaming heavily used functions/classes/etc) should not be done as others are working. When you want to make broad changes like that, communicate with the other developers, agree on what will be changed, sync up with everybody, make the change, then sync up again.

The same is true if you want to redesign some part of the code, pulling it all apart and then putting it together in a different way, or possibly rewriting it from scratch. Working independently in parallel, and then merging afterwards, is an unworkable model in that case. The solution is to agree that someone is going to overhaul certain files/classes/modules/etc, sync up, make the changes, then sync up again. After that, you can continue working in parallel.

Although git and other distributed VC tools allow any dev to work on any part of the code at any time, that doesn't mean you should actually do so. There are times when you need to use some discipline, times when you need to communicate more with the other devs and coordinate your efforts. Yes, that coordination has overhead. Yes, it will slow you down sometimes. That's an unavoidable consequence of working as a team rather than an individual. Of course, if you can divide up the work in a way which eliminates the need for fine-grained coordination, that overhead will be much less.

Push your changes to the main development branch at least once a day, preferably more often, and encourage other developers to do the same. As many people say, if it hurts, do it more often.

That means that what you push generally won't be the entire change you intended to make, because you haven't had time to make it. Instead it will be some other changes to the code which will make it easier for you to make the change you want in future.

This workflow requires that each commit must be an incremental change that at least does not make the product worse. For instance you might carry out one refactoring in a commit, when you believe it will help you make a required change later, or implement part of a new feature, making sure that users will not be able to access anything that isn't ready.

This is continuous integratation.