The Android ION memory allocator

Back in December 2011, LWN reviewed the list of Android kernel patches in the linux-next staging directory. The merging of these drivers, one of which is a memory allocator called PMEM, holds the promise that the mainline kernel release can one day boot an Android user space. Since then, it has become clear that PMEM is considered obsolete and will be replaced by the ION memory manager. ION is a generalized memory manager that Google introduced in the Android 4.0 ICS (Ice Cream Sandwich) release to address the issue of fragmented memory management interfaces across different Android devices. There are at least three, probably more, PMEM-like interfaces. On Android devices using NVIDIA Tegra, there is "NVMAP"; on Android devices using TI OMAP, there is "CMEM"; and on Android devices using Qualcomm MSM, there is "PMEM" . All three SoC vendors are in the process of switching to ION.

ION heaps

   struct ion_heap_ops { 	int (*allocate) (struct ion_heap *heap, 			 struct ion_buffer *buffer, unsigned long len, 			 unsigned long align, unsigned long flags); 	void (*free) (struct ion_buffer *buffer); 	int (*phys) (struct ion_heap *heap, struct ion_buffer *buffer, 		     ion_phys_addr_t *addr, size_t *len); 	struct scatterlist *(*map_dma) (struct ion_heap *heap, 			 struct ion_buffer *buffer); 	void (*unmap_dma) (struct ion_heap *heap,  	         struct ion_buffer *buffer); 	void * (*map_kernel) (struct ion_heap *heap,  	         struct ion_buffer *buffer); 	void (*unmap_kernel) (struct ion_heap *heap,  	         struct ion_buffer *buffer); 	int (*map_user) (struct ion_heap *heap, struct ion_buffer *buffer, 			 struct vm_area_struct *vma);    }; 

The default ION driver (which can be cloned from here) offers three heaps as listed below:

   ION_HEAP_TYPE_SYSTEM:        memory allocated via vmalloc_user().    ION_HEAP_TYPE_SYSTEM_CONTIG: memory allocated via kzalloc.    ION_HEAP_TYPE_CARVEOUT:	carveout memory is physically contiguous and set aside at boot. 

Using ION from user space

A user space C/C++ program must have been granted access to the /dev/ion device before it can allocate memory from ION. A call to open("/dev/ion", O_RDONLY) returns a file descriptor as a handle representing an ION client. Yes, one can allocate writable memory with an O_RDONLY open. There can be no more than one client per user process. To allocate a buffer, the client needs to fill in all the fields except the handle field in this data structure:

   struct ion_allocation_data {         size_t len;         size_t align;         unsigned int flags;         struct ion_handle *handle;    } 

User-space clients interact with ION using the ioctl() system call interface. To allocate a buffer, the client makes this call:

   int ioctl(int client_fd, ION_IOC_ALLOC, struct ion_allocation_data *allocation_data) 

   int ioctl(int client_fd, ION_IOC_SHARE, struct ion_fd_data *fd_data); 

   struct ion_fd_data {         struct ion_handle *handle;         int fd;    } 

To free the buffer, the second client needs to undo the effect of mmap() with a call to munmap(), and the first client needs to close the file descriptor it obtained via ION_IOC_SHARE, and call ION_IOC_FREE as follows:

     int ioctl(int client_fd, ION_IOC_FREE, struct ion_handle_data *handle_data); 

     struct ion_handle_data { 	     struct ion_handle *handle;      } 

User processes can also share ION buffers with a kernel driver, as explained in the next section.

Sharing ION buffers in the kernel

In the kernel, ION supports multiple clients, one for each driver that uses the ION functionality. A kernel driver calls the following function to obtain an ION client handle:

   struct ion_client *ion_client_create(struct ion_device *dev,                     unsigned int heap_mask, const char *debug_name) 

The first argument, dev, is the global ION device associated with /dev/ion; why a global device is needed, and why it must be passed as a parameter, is not entirely clear. The second argument, heap_mask, selects one or more ION heaps in the same way as the ion_allocation_data. The flags field was covered in the previous section. For smart phone use cases involving multimedia middleware, the user process typically allocates the buffer from ION, obtains a file descriptor using the ION_IOC_SHARE command, then passes the file desciptor to a kernel driver. The kernel driver calls ion_import_fd() which converts the file descriptor to an ion_handle object, as shown below:

    struct ion_handle *ion_import_fd(struct ion_client *client, int fd_from_user); 

Some hardware blocks can only operate on physically-contiguous buffers with physical addresses, so affected drivers need to convert ion_handle to a physical buffer via this call:

   int ion_phys(struct ion_client *client, struct ion_handle *handle, 	       ion_phys_addr_t *addr, size_t *len) 

Needless to say, if the buffer is not physically contiguous, this call will fail.

When handling calls from a client, ION always validates the input file descriptor, client and handle arguments. For example, when importing a file descriptor, ION ensures the file descriptor was indeed created by an ION_IOC_SHARE command. When ion_phys() is called, ION validates whether the buffer handle belongs to the list of handles the client is allowed to access, and returns error if the handle is not on the list. This validation mechanism reduces the likelihood of unwanted accesses and inadvertent resource leaks.

ION provides debug visibility through debugfs. It organizes debug information under /sys/kernel/debug/ion, with bookkeeping information in stored files associated with heaps and clients identified by symbolic names or PIDs.

Comparing ION and DMABUF

ION and DMABUF share some common concepts. The dma_buf concept is similar to ion_buffer, while dma_buf_attachment serves a similar purpose as ion_handle. Both ION and DMABUF use anonymous file descriptors as the objects that can be passed around to provide reference-counted access to shared buffers. On the other hand, ION focuses on allocating and freeing memory from provisioned memory pools in a manner that can be shared and tracked, while DMABUF focuses more on buffer importing, exporting and synchronization in a manner that is consistent with buffer sharing solutions on non-ARM architectures.

The following table presents a feature comparison between ION and DMABUF:

ION and DMABUF can be separately integrated into multimedia applications written using the Video4Linux2 API. In the case of ION, these multimedia programs tend to use PMEM now on Android devices, so switching to ION from PMEM should have a relatively small impact.

Integrating DMABUF into Video4Linux2 is another story. It has taken ten patches to integrate the videobuf2 mechanism with DMABUF; in fairness, many of these revisions were the result of changes to DMABUF as that interface stabilized. The effort should pay dividends in the long run because the DMABUF-based sharing mechanism is designed with DMA mapping hooks for CMA and IOMMU. CMA and IOMMU hold the promise to reduce the amount of carveout memory that it takes to build an Android smart phone. In this email, Andrew Morton was urging the completion of the patch review process so that CMA can get through the 3.4 merge window.

Even though ION and DMABUF serve similar purposes, the two are not mutually exclusive. The Linaro Unified Memory Management team has started to integrate CMA into ION. To reach the state where a release of the mainline kernel can boot the Android user space, the /dev/ion interface to user space must obviously be preserved. In the kernel though, ION drivers may be able to use some of the DMABUF APIs to hook into CMA and IOMMU to take advantage of the capabilities offered by those subsystems. Conversely, DMABUF might be able to leverage ION to present a unified interface to user space, especially to the Android user space. DMABUF may also benefit from adopting some of the ION heap debugging features in order to become more developer friendly. Thus far, many signs indicate that Linaro, Google, and the kernel community are working together to bring the combined strength of ION and DMABUF to the mainline kernel.

The Android ION memory allocator,,5-wow.com