2

Let me start by saying that I am aware a simple search for "nfs SQLite" surfaces this answer as the top result. In a perfect world, I'd be able to choose to use a better database engine and a better storage solution. This is not a perfect world.

We use NFSv4 for networking files across our infra. The shares are served by an IBM StorageScale appliance installed about 3 years ago.

We also happen to use closed-source software that uses SQLite under the hood for various tasks. Not using this software is entirely out of the question.

As you can imagine, we're not having a great time. We frequently encounter crashes in the application due to "Disk I/O error"s when the software tries interacting with the database. Obviously we checked, and the disks, shares, mounts and OSs are all fine.

The software is claimed by the vendor to be compatible with NFS (which is preposterous, since it uses SQLite), and it is to the best of our abilities configured to maximize this compatibility, but obviously this isn't enough. We know of multiple other companies in the same space as us using this software over NFS with perhaps not incredible success but at least fewer and less frequent issues.

I'm looking for ways to configure our NFS shares and mounts that increase reliability of SQLite databases stored on these shares, and I'm open to suggestions for alternative networked file systems that are known to scale well and support SQLite, though obviously the barrier to entry is much higher.

Here's a sample entry returned by nsfsstat -m for one of our current shares:

 Flags: rw,nodev,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=[censored],local_lock=none,addr=[censored]
Improve this question

1 Answer 1

Reset to default
2

It might make sense to take a step back and look at the given problem from a different angle.

  • Is it technically required that these SQLite databases reside on a shared filesystem?
  • Is there concurrent write access to the same SQLite database from different clients?

Maybe you can get away by keeping the database files locally, regularly replicating them to your NFS storage.

Improve this answer
1
  • 1) as far as we know, yes, it is a technical requirement because it is part of each project's storage, which must be shared. while it could be possible to e.g. symlink it to a local disk, this is the type of things that get you "we won't support you because your setup is non-standard" responses froom vendor support. 2) I'm actually not sure, the software is opaque. there is only ever 1 user (as in, end-user, a person) using the database, but i don't know whether the software spawns multiple clients to the DB - I suspect it does but only in read-only. Commented 10 hours ago

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.