Ammasso 1100 User's Guide download pdf (Page 37)

Each MPI process is allowed to lock down up to VIADEV_MAX_RENDEZVOUS amount of

application buffer memory. Once this limit is reached, IO buffers are evicted and

unregistered on a “least recently used” basis.

The default value for this parameter is 209715200 bytes (200MB).

4.7.2 Tuning suggestions

Based on the default values, one can determine how much memory is being consumed by

each MPI process in a run. Then the desired amount of memory should be determined based

on how much memory your MPI application needs for doing computation. Given you can

actually determine that, you can then adjust these parameters to reduce the amount of

memory used by the MPI RDMA driver.

A few guidelines are offered here:

• Try reducing VIADEV_NUM_RDMA_BUFFERS first. The default for this parameter

doesn’t particularly scale well or evenly as your NP value increases.

• Try reducing VIADEV_MAX_RENDEZVOUS as a next step.

• Keep VIADEV_SQ_DEPTH >= VIADEV_RQ_DEPTH

• Try to avoid lowering VIADEV_RQ_DEPTH. It is probably safe to drop this to

around 64, but avoid going lower if possible.

1 2 ... 32 33 34 35 36 37 38 39 40 41 42 ... 65 66

No comments

Ammasso 1100 User's Guide Page 37