Nginx Architecture Deep Dive: Pool Implementation

This is one of the series of articles that I will write on nginx Architecture. I did a google search but could not find proper documentation on nginx. Although there are few but not as much in detail as I wanted.

This series of articles I am going to write on nginx will be complimentary along with those already existing. I will start with various utilities that are built in the nginx.

In this post, I will describe about nginx memory management utility called 'Pools'

From Usage perspective,  Pools are chunks/blocks of memory that are allocated and managed internally.  Pools try to use contiguous memory and if there is requirement of memory more need than available nginx allocates a new block and adds to the pool.

Nginx can allocate a max size of Page size for each block i.e 4095(4096-1) on x86. To meet the requirements of memory of size more than 4k, nginx uses 'malloc' and manages it.

Here is the pool structure

struct ngx_pool_s {

    ngx_pool_data_t       d;

    size_t                max;

    ngx_pool_t           *current;

    ngx_chain_t          *chain;

    ngx_pool_large_t     *large;

    ngx_pool_cleanup_t   *cleanup;

    ngx_log_t            *log;

};

 ngx_pool_data_s stores all the information related to pool.  It has the following pointers

'last' : Points to the available memory to use in the block
'end': Points to the dead end of that memory block
'next': Points to the next memory block's pool data structure
'failed' : Number of failures to allocate memory in this block. 

'max' is the total size of memory block
'current' points to the first available memory block for use in the pool.  This is always updated to the block which has memory left for use and not failed more than 4 times in allocating memory.
'large' points to the chain of memory blocks allocated by malloc
'cleanup' will hold a function pointer with data to pass to it. Applications can make use of it to set callbacks during pool destruction.
'log' points to logger utility structure which I will describe in coming posts.

Note: nginx allocates more  memory blocks on demand. But it does not provide any function to delete the memory blocks except for those 'large' memory blocks that are wrapped around 'malloc'

Users of the Pools can reset the memory blocks and free up the large buffers.  After a reset of pool, each memory block will have the 'last' pointed to the start of the memory block(i.e actual start + size of (ngx_pool_s))


Following are the pool functions that are used by other nginx modules or utilities:

ngx_palloc()  - To allocate memory of a given size from a given pool
ngx_pnalloc() - similar to above but allocates zeroed memory
ngx_create_pool() - Create a new memory Pool of a given size
ngx_destroy_pool() - Destroy the memory pool - frees up both memory blocks and large buffers
ngx_pool_reset() - Cleans up Large buffers and makes all memory blocks available for use.