0

Normally I don't ask questions but I'm about at my wits end and foresee myself struggling with this for a while. To the question reviewers, I have not found any questions referring to either the Mongo versions I'm having trouble with nor my error specific to the arbiter node.

TLDR:

The arbiter is failing on an upgrade from v6 to v7 because of an invalid featureCompabilityVersion I've read that it should not care about or use.

Am I missing something obvious relating to the configuration/upgrade of the arbiter node?

Long version:

I'm using Docker Compose to manage a replica set of Mongo databases consisting of 1 primary, 1 secondary, and 1 arbiter. They all use the docker.io/bitnami/mongodb image and have volumes to persist whatever data they have to write. My task is to upgrade their images from v5 to v7 using the Mongo instructions.

Upgrading from v5 to v6 was uneventful since v6 is backwards compatible and just required updating the image version. To upgrade to v7 I logged into the primary node's admin database and ran db.adminCommand( { setFeatureCompatibilityVersion: "6.0" } ) followed by db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1}) to confirm the change took.

I updated the image versions in the compose file from v6 to v7 and restarted. The primary/secondary nodes are fine but the arbiter spews this error before shutting down,

UPGRADE PROBLEM: Found an invalid featureCompatibilityVersion document (ERROR: Location4926900: Invalid featureCompatibilityVersion document in admin.system.version: { _id: "featureCompatibilityVersion", version: "5.0" }. See https://docs.mongodb.com/master/release-notes/6.0-compatibility/#feature-compatibility. :: caused by :: Invalid feature compatibility version value '5.0'; expected '6.0' or '6.3' or '7.0'. See https://docs.mongodb.com/master/release-notes/6.0-compatibility/#feature-compatibility.). If the current featureCompatibilityVersion is below 6.0, see the documentation on upgrading at https://docs.mongodb.com/master/release-notes/6.0/#upgrade-procedures.

Now I've read that the arbiter should contain no data and that the arbiter ignores the FCV in favor of the backwards compatibility reported by the binaries, which if true, should be v6, not v5 as the error states. I tried to authenticate to the arbiter to set the FCV there but I get command setFeatureCompatibilityVersion requires authentication.

Am I missing something obvious relating to the configuration/upgrade of the arbiter node?

Improve this question

2 Answers 2

Reset to default
2

As you have written already, the Arbiter does not store any data. Thus, I would propose simply stop it, drop entire dbPath folder, install new binaries and start it again.

By default it will not affect your application at all.

Improve this answer
1
  • You are a prince among men and I deeply appreciate you taking the time to answer. Removing the arbiter's db files from disk let the container start properly. rs.status() show it's still a part of the replica set. I suppose I might benefit from removing it's volume from the compose file. Commented Mar 4 at 19:19
1

Facing a similar question before when upgrading from 4.2 to 4.4, and after.

context: https://github.com/bitnami/charts/issues/17432#issuecomment-1680889994

If you are using bitnami/mongodb k8s chart and helmfile, what you can do is to use strategicMergePatches to patch the mongodb-arbiter statefulset image to the old version, upgrade data bearing nodes, FCV, and then proceed to upgrade arbiter the container image tag.

Example:

- name: mongodb namespace: default version: 15.6.26 chart: oci://registry-1.docker.io/bitnamicharts/mongodb strategicMergePatches: - apiVersion: apps/v1 kind: StatefulSet metadata: name: mongodb-arbiter namespace: default spec: template: spec: containers: - name: mongodb-arbiter # Workaround the arbiter upgrade issue: it should only updated after mongodb data node update has finished image: docker.io/bitnami/mongodb:<place your previous image tag>
Improve this answer

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.