Find a model with transfer function and real data

For "eliminating" the offsets, you can use this kind of circuit.
R2 is lower, added R5 and Voff. Adjust Voff with voltage ~ 2.5 V just before the time = 10.

If you don't adjust Voff, you will have something like this.

If you are impatient and want to estimate the time constant quickly you do not need to wait till you fall to 37%. You can indeed measure the time constant the way you do. The only downside is that your measurement will be less precise.

Initially (for t much smaller then tau) the exp(-t/tau) will be almost linear. E.g. at t=1% of tau exp(-t/tau) is also roughly 1% (0.995%) at t=10% of tau then exp(-t/tau) is 9.52%

So small errors in the how you measure the output voltage will lead to larger errors in how you estimate tau but other then that your method of using the logarithm will work.

For measuring the time constant it also does not matter if the change in output voltage is due to the step function in the input or due to initial changes in other parameters, as long as at the time of measurement the Vcm is stable.

Analysis of the Op Amp circuit with two input signals:

The graph is impossible because the expnents take on very high values, causing both the hyperbolic function and the exponential to diverge and practically overflow.

By changing some parameters you get a plausible answer:

The requested solution is at the end if you want to skip the math.

Therefore, I want to calculate my time constant independently of the common mode value.

The common mode voltage is part of all the node voltages when measured to ground. The only way to remove it is to measure the node voltages relative to the common mode voltage as demonstrated below.

The circuit presented is second order and overdamped. So the proposed first order approximation can sometimes work with appropriate constraints. The secret (and the effort) is to discover the constraints that allow it.

The main difficulty the OP presents is that the output voltage has insufficient time to reach 5 time constants, so the initial condition of the capacitors is not known.

A helpful approach is to measure the input and output relative to the common mode bias voltage \$v_{cm}(t)=V_{CM}=2.5V\$. If this can't be done then subtract the common mode voltage from the voltages measured relative to ground.

^{simulate this circuit – Schematic created using CircuitLab}

Notice that lowercase is used for time varying signals and uppercase used for Laplace domain or constants.

In the Laplace domain \$V_{cm}(s)=\frac{V_{CM}}{s}=\frac{2.5}{s}\$.

To reveal the initial conditions (\$v_{Ci}(0)\$, and \$v_{Cf}(0)\$) at \$t=0\$, the following capacitor definitions are used for \$C_i\$ and \$C_f\$ respectively:

\$V_{Ci}(s)=\frac{1}{C_{i}s}I_{Ci}(s)+\frac{v_{Ci}(0)}{s}\$

\$I_{Cf}(s)=C_{f}sV_{Cf}(s)-C_{f}v_{Cf}(0)\$

Circuit analysis then reveals the output response relative to \$V_{cm}(s)\$.

$$ V_{ocm}(s)=\frac{-\frac{1}{R_{i}C_{f}}s}{\left(s+\frac{1}{R_{f}C_{f}}\right)\left(s+\frac{1}{R_{i}C_{i}}\right)}V_{icm}(s)+\frac{v_{Cf}(0)}{\left(s+\frac{1}{R_{f}C_{f}}\right)}-\frac{\frac{1}{R_{i}C_{f}}v_{Ci}(0)}{\left(s+\frac{1}{R_{f}C_{f}}\right)\left(s+\frac{1}{R_{i}C_{i}}\right)} $$

where \$V_{icm}(s)=\left(V_{i}(s)-V_{cm}(s)\right)\$, and \$V_{ocm}(s)=\left(V_{o}(s)-V_{cm}(s)\right)\$ are the input and output voltages relative to the common mode voltage. Notice that the output voltage \$V_{ocm}(s)\$ is the feed back capacitor voltage.

The input voltage in the OP corresponds to a -0.25 V step relative to \$V_{CM}\$. For an arbitrary step input of height \$\frac{V_A}{s}\$ The time domain solution is: $$ v_{ocm}(t)=\left(\frac{\left(V_{A}+v_{Ci}(0)\right)}{\left(\frac{R_{i}}{R_{f}}-\frac{C_{f}}{C_{i}}\right)}+v_{Cf}(0)\right)e^{\frac{-t}{R_{f}C_{f}}}-\frac{\left(V_{A}+v_{Ci}(0)\right)}{\left(\frac{R_{i}}{R_{f}}-\frac{C_{f}}{C_{i}}\right)}e^{-\frac{-t}{R_{i}C_{i}}} $$

For the values given \$\left(\frac{R_{i}}{R_{f}}-\frac{C_{f}}{C_{i}}\right)\approx-1\$, so: $$ v_{ocm}(t)=\left(-V_{A}+v_{Cf}(0)-v_{Ci}(0)\right)e^{\frac{-t}{R_{f}C_{f}}}+\left(V_{A}+v_{Ci}(0)\right)e^{-\frac{-t}{R_{i}C_{i}}} $$

\$R_{i}C_{i}<<R_{f}C_{f}\$ allows the second order system to be approximated as a first order system tried in the OP. The term on right becomes insignificant for \$t>5R_{i}C_{i}\$. $$ v_{ocm}(t)\approx\left(-V_{A}+v_{Cf}(0)-v_{Ci}(0)\right)e^{\frac{-t}{R_{f}C_{f}}} $$

The three constants must be measured to achieve an accurate calculation.

From a comment:

I want another solution/ method.

Here it is:
Take the ratio of two measurements, then the time constant can be determined. The ratio of two measurements at different times is: $$ \frac{v_{ocm}(t_1)}{v_{ocm}(t_2)}\approx e^{\left(\frac{t_{2}-t_{1}}{R_{f}C_{f}}\right)}\Rightarrow R_{f}C_{f}=\frac{t_{2}-t_{1}}{\ln\frac{v_{ocm}(t_{1})}{v_{ocm}(t_{2})}} $$

The following constraints must be observed:

1. The output voltage must be measured relative to the common mode voltage. VCM cannot be ignored.
2. \$5R_{i}C_{i}<t_{1}<t_{2}<5R_{f}C_{f}\$. This is necessary to maintain the first order approximation.

This result also shows that the the circuit operates independently from the common mode voltage even if it is time varying.