docker on Fedora has conservative ulimits

Today I woke up to alert mails from my monitoring that the InfluxDB instance was restarting now and again. Looking into this I found out that it couldn’t open any more files. Apparently docker (moby-engine) on Fedora 37 has a default ulimit on open file descriptors of 1024 (hard and soft), which means inside a docker container processes are not allowed to have more than 1024 files open at a time.

InfluxDB has a shard-based storage driver which means, it needs to potentially open a lot of shard files. I’d say a standard sized instance can easily exceed 1024 open file descriptors. So for standard installation with reasonable usage 4096 sounds more appropriate.

The current imposed ulimits can be found inside the container with docker{-compose} exec <container-id> ulimits -n <-H|-S>.

Configure ulimits in docker on CLI with --ulimit nofile=4096:4096 and in docker-compose.yml with:

  ...
  ulimits:
    nofile:
      soft: 4096
      hard: 4096

So it seems that ulimits on Fedora Linux using moby-engine are quite conservative. Which is fine by me. It’s good to have it fail when it exceeds sane default usage parameters. In malicious or faulty scenarios I want it to fail sooner than later.