This is just a partial answer.
Manipulate[] is rewriting the code in a way that breaks With[]. If we look that the InputForm of the evaluated code, we see that the code for Control[..] has been tagged with the option ControlPlacement -> 1, which apparently marks it to replace the placeholder Manipulate`Place[1].
With[{$maxterms = 4}, Manipulate[ a[[;; terms]], {{a, ConstantArray[0., $maxterms]}, None}, {{terms, 2}, Range@$maxterms}, Dynamic@ Column@ Table[ With[{i = i}, Control[{ {a, ConstantArray[0., $maxterms], Subscript["a", i]}, (* i not injected *) 0., 1, Manipulator[Dynamic[a[[i]]], ##2] &}] (* i is injected *) ], {i, terms}] ] ] // InputForm
Input form:
Manipulate[ a[[1 ;; terms]], {{a, {0., 0., 0., 0.}}, 0, ControlType -> None}, {{terms, 2}, {1, 2, 3, 4}}, {{a, {0., 0., 0., 0.}, Subscript["a", i$]}, 0., 1, Dynamic[Manipulator[Dynamic[a[[i$]]], ##2] & ], ControlPlacement -> 1}, Dynamic[Column[ Table[With[{i$ = i}, Manipulate`Place[1]], {i, terms}]]] ]
What is still unclear is how With[{i$ = i}, Manipulate`Place[1]] results in the symbol i$ being replaced by the value of i. It seems to have something to do with being wrapped in Dynamic[].
For instance, if we add a Dynamic@ to Subscript["a", i] in the following two ways, only the first of the two works:
Dynamic@Subscript["a", i] (* WORKS (@xavier's workaround) *) Subscript["a", Dynamic@i] (* does NOT work *)

However, both codes work if we use a replacement rule instead of With to inject the value of i; just replace the original Control[..] code inside Table[] with this:
Control[{{a, ConstantArray[0., $maxterms], Subscript["a", Dynamic@i] (* or replace with Dynamic@Subscript["a", i] *) }, 0., 1, Manipulator[Dynamic[a[[i]]], ##2] &}] /. HoldPattern[i] -> i
But the replacement rule method fails just like With in the OP if the control label is just Subscript["a", i], not being wrapped by Dynamic[].
Going back to the input form of the original code, I would expect Manipulate`Place[1] might be replaced by the control. Something like this:
DynamicModule[{terms = 2, a = {0., 0., 0., 0.}}, Dynamic[Column[Table[With[{i$ = i}, Manipulate`Place[1]], {i, terms}]]] /. Manipulate`Place[1] -> Control@{{a, {0., 0., 0., 0.}, Subscript["a", i$]}, 0., 1, Dynamic[Manipulator[Dynamic[a[[i$]]], ##2] &], ControlPlacement -> 1} ]

But as can be seen, that actually works, so something else must be happening.
Dynamic@Subscript["a", i]makes the injection. Could it be that without thisDynamic,Manipulatedoes not bother to inject? This seems to be the case when removingDynamicfromManipulatorand writing insteadManipulator[a[[i]], ##2] &. $\endgroup$Manipulate[]is doing yet. It must be rewriting the code in some way. I guess I'm wondering whether it's a bug or I'm breaking some rule. $\endgroup$