Instability on AL2

[dangeReis]

We have about 4TB of data on a 1TB SSD EBS volume on AWS using Amazon Linux 2.

Everything is working great, except for random lockups every couple of days. Would love to be able to post some stack traces, but can't seem to figure out how to get them.

Linux myserver 4.14.262-200.489.amzn2.x86_64 #1 SMP Fri Feb 4 20:34:30 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

[bghira]

4tb on a 1tb volume?

[dangeReis]

Mostly text files, good compression with zstd. Guess it's more like 6T.

NAME  PROPERTY              VALUE                  SOURCE
tank  type                  filesystem             -
tank  creation              Tue Apr 20 22:14 2021  -
tank  used                  724G                   -
tank  available             261G                   -
tank  referenced            18.5G                  -
tank  compressratio         8.77x                  -
tank  mounted               yes                    -
tank  quota                 none                   default
tank  reservation           none                   default
tank  recordsize            16M                    local
tank  mountpoint            /tank                  default
tank  sharenfs              off                    default
tank  checksum              on                     default
tank  compression           zstd-9                 local
tank  atime                 off                    local
tank  devices               on                     default
tank  exec                  on                     default
tank  setuid                on                     default
tank  readonly              off                    default
tank  zoned                 off                    default
tank  snapdir               hidden                 default
tank  aclmode               discard                default
tank  aclinherit            restricted             default
tank  createtxg             1                      -
tank  canmount              on                     default
tank  xattr                 off                    local
tank  copies                1                      default
tank  version               5                      -
tank  utf8only              off                    -
tank  normalization         none                   -
tank  casesensitivity       sensitive              -
tank  vscan                 off                    default
tank  nbmand                off                    default
tank  sharesmb              off                    default
tank  refquota              none                   default
tank  refreservation        none                   default
tank  guid                  4581886990559413009    -
tank  primarycache          all                    default
tank  secondarycache        all                    default
tank  usedbysnapshots       0B                     -
tank  usedbydataset         18.5G                  -
tank  usedbychildren        705G                   -
tank  usedbyrefreservation  0B                     -
tank  logbias               latency                default
tank  objsetid              54                     -
tank  dedup                 off                    default
tank  mlslabel              none                   default
tank  sync                  disabled               local
tank  dnodesize             legacy                 default
tank  refcompressratio      2.95x                  -
tank  written               18.5G                  -
tank  logicalused           6.16T                  -
tank  logicalreferenced     54.4G                  -
tank  volmode               default                default
tank  filesystem_limit      none                   default
tank  snapshot_limit        none                   default
tank  filesystem_count      none                   default
tank  snapshot_count        none                   default
tank  snapdev               hidden                 default
tank  acltype               off                    default
tank  context               none                   default
tank  fscontext             none                   default
tank  defcontext            none                   default
tank  rootcontext           none                   default
tank  relatime              off                    default
tank  redundant_metadata    all                    default
tank  overlay               on                     default
tank  encryption            off                    default
tank  keylocation           none                   default
tank  keyformat             none                   default
tank  pbkdf2iters           0                      default
tank  special_small_blocks  0                      default

[bghira]

and it happens with recordsize set to a more sane value?

[dangeReis]

That's a good question. I was just thinking about that - would hate to lose all the compression, but if it makes it more stable. What's the max supported value? 1MB? 128K?

Would using a higher (5.x) kernel help?

[dangeReis]

and it happens with recordsize set to a more sane value?

Also would I need to copy all the data off, or can I just set it going forward?

[dangeReis]

also using ashift=12

[joshenders]

Just a heads up...

You've created this issue in the wrong project. This project is for the website zfsonlinux.github.com and not the OpenZFS project.

Additionally, your post lacks sufficient clarity and information which is necessary to debug and resolve your issue. (It currently reads as a support request and GitHub issues are not intended as a support forum.)

Please close this issue and follow up in the appropriate project following this issue template for reporting an issue.

Thanks!