Skip to main content

AI Image Preprocess

APIs to accelerate AI image preprocessing with the following steps:

  • Image feeding from NVMe to DPU memory
  • Image preprocessing (e.g, JPEG decoding, crop, resize)
  • P2P transfer between DPU and GPU memory

Refer to related Context APIs as well.

Example​

Here is an example of using AI image preprocess APIs.

example_image_prep.cc
#include <libmango.h>

#define NUM_BATCH 16

int main()
{
  // create a context with the first DPU device
  mango_context_h context;
  mango_context_create (&context);
  mango_context_set_hw_accel (context, MANGO_HW_ACCEL_DPU, 0)

  // create a GPU handle with the first NVIDIA GPU device
  mango_gpu_h gpu;
  mango_gpu_create (&gpu);
  mango_gpu_open (gpu, MANGO_GPU_NVIDIA, 0)

  // create a GPU buffer
  size_t size = ...;
  void *gpu_buf;
  mango_gpu_alloc_data (gpu, size, &gpu_buf);

  // set GPU data to the context
  mango_context_set_gpu_data (context, gpu, gpu_data, size);

  // load images to the DPU
  char *image_files[NUM_BATCH];
  for (int i = 0; i < NUM_BATCH; i++) {
    image_files[i] = ...;
  }
  mango_context_load_images (context, image_files, NUM_BATCH);

  // decode images in the DPU
  mango_context_decode_images (context, NULL);

  // transfer decoded images from DPU to GPU
  mango_context_images_to_gpu (context, gpu, 0);

  // copy GPU buffer to CPU buffer
  void *cpu_buf = ...;
  mango_gpu_memcpy (gpu, cpu_buf, gpu_buf, size, false);

  // release resources
  mango_gpu_free_data (gpu, gpu_buf);
  mango_gpu_destroy (gpu);
  mango_context_destroy (context);

  return 0;
}

Datatypes​

mango_image_h​

typedef void * mango_image_h;

A handle of mango image.

mango_gpu_h​

typedef void * mango_gpu_h;

A handle of mango gpu.

mango_gpu_type_e​

typedef enum _mango_gpu_type mango_gpu_type_e;

Define GPU types compatible with libmango APIs.

Values

MANGO_GPU_NVIDIANVIDIA GPU
MANGO_GPU_AMDAMD GPU
MANGO_GPU_INTELIntel GPU

Functions​

mango_gpu_num_devices​

int mango_gpu_num_devices(mango_gpu_type_e gpu_type)

Get a number of GPU devices.

Parameters

  • in gpu_type The GPU device type

Returns a number of devices.

mango_gpu_create​

mango_status_e mango_gpu_create(mango_gpu_h *gpu)

Create a GPU handle.

Parameters

  • out gpu The output GPU handle.

Returns 0 on success, Otherwise, a negative error value.

mango_gpu_destroy​

mango_status_e mango_gpu_destroy(mango_gpu_h gpu)

Destroy the GPU handle.

Parameters

  • in gpu The GPU handle to be destroyed.

Returns 0 on success, Otherwise, a negative error value.

mango_gpu_open​

mango_status_e mango_gpu_open(mango_gpu_h gpu,
                              mango_gpu_type_e gpu_type,
                              int id)

Open a GPU device.

Parameters

  • in gpu The GPU handle to be opened.
  • in gpu_type The GPU device type
  • in id The GPU id in the valid range.

Returns 0 on success, Otherwise, a negative error value.

note

'id' should be less than num_devices().

mango_gpu_close​

mango_status_e mango_gpu_close(mango_gpu_h gpu)

Close the GPU device.

Parameters

  • in gpu The GPU handle.

Returns 0 on success, Otherwise, a negative error value.

mango_gpu_alloc_data​

mango_status_e mango_gpu_alloc_data(mango_gpu_h gpu, size_t size, void **data)

Allocate GPU data abuffer.

Parameters

  • in gpu The GPU handle.
  • in size The buffer size.
  • out data The output buffer ptr.

Returns 0 on success, Otherwise, a negative error value.

mango_gpu_free_data​

mango_status_e mango_gpu_free_data(mango_gpu_h gpu, void *data)

Free GPU data abuffer.

Parameters

  • in gpu The GPU handle.
  • in data The GPU data buffer.

Returns 0 on success, Otherwise, a negative error value.

mango_gpu_memcpy​

mango_status_e mango_gpu_memcpy(mango_gpu_h gpu,
                                void *cpu_data,
                                void *gpu_data,
                                size_t size,
                                bool to_gpu)

Perform memory copy between CPU and GPU.

Parameters

  • in gpu The GPU handle.
  • in cpu_data The CPU data buffer.
  • in gpu_data The GPU data buffer.
  • in size The data buffer size.
  • in to_gpu The memcpy direction.

Returns 0 on success, Otherwise, a negative error value.

mango_gpu_get_affinity_mask​

mango_status_e mango_gpu_get_affinity_mask(mango_gpu_h gpu, cpu_set_t *mask)

Get a proper CPU affinity mask for the GPU device.

Parameters

  • in gpu The GPU handle.
  • out mask The CPU affinity mask.

Returns 0 on succes, Otherwise, a negative error value.

note

The number of CPUs and PCIe topology decides the returned mask.