6. Python API

6.1. poprt module

class poprt.Converter(input_shape=None, convert_version=11, precision='fp32', checkpoints=None, eightbitsio=False, fp16_skip_op_types=None, skip_passes=None, custom_pass_config=None, used_passes=[], check=False, disable_fast_norm=False, pack_args=None, fp8_skip_op_names=None, fp8_params=None, quantize=False, enable_insert_remap=False, enable_erf_gelu=False, serialize_matmul=None, serialize_matmul_add=None, remap_mode='after_matmul', max_tensor_size=- 1, infer_shape_ahead=False)

Bases: object

Convert genernal ONNX model to IPU friendly ONNX model.

Parameters

  • input_shape (Dict[str, List[int]]) –

  • convert_version (int) –

  • precision (str) –

  • checkpoints (str) –

  • eightbitsio (bool) –

  • fp16_skip_op_types (str) –

  • skip_passes (str) –

  • custom_pass_config (str) –

  • used_passes (List[str]) –

  • check (bool) –

  • disable_fast_norm (bool) –

  • pack_args (Dict) –

  • fp8_skip_op_names (str) –

  • fp8_params (str) –

  • quantize (bool) –

  • enable_insert_remap (bool) –

  • enable_erf_gelu (bool) –

  • serialize_matmul (Dict[str, str]) –

  • serialize_matmul_add (Dict[str, str]) –

  • remap_mode (str) –

  • max_tensor_size (int) –

  • infer_shape_ahead (bool) –

convert(model, logger)

Convert genernal ONNX model to IPU friendly ONNX model.

Parameters

  • model (ModelProto) – A ONNX ModelProto class object to be converted.

  • logger –

Returns

A ONNX ModelProto class object representing the ONNX model.

Return type

ModelProto

class poprt.Pass(*args, **kwargs)

Bases: object

Abstract base class for passes.

Return type

None

static against_passes(pass_names)

Register against property for a Pass.

Passes can’t work with against passes.

Parameters

pass_names (List[str]) –

Return type

Callable[[Pass], Pass]

static constraint_passes(constraint_name, pass_names)

Register constraints for pass.

valid constraints are against, depend, before

Parameters

  • constraint_name (str) –

  • pass_names (List[str]) –

Return type

Callable[[Pass], Pass]

static get_pass(name, arg=None, *args, **kwargs)

Get Pass by name.

Parameters

  • name (str) –

  • arg (Optional[Any]) –

Return type

Pass

static get_registered_passes()

Get all registered Passes.

Return type

Dict[str, Pass]

static property_register(k, v)

Register a property for Pass.

Parameters

  • k (str) –

  • v (Any) –

Return type

Callable[[Pass], Pass]

static register_pass(pass_name)

Register a Pass.

Parameters

pass_name (str) –

Return type

Callable[[Any], Pass]

6.2. poprt.backend module

poprt.backend.get_session(path_or_bytes, num_devices=1, backend='poprt', output_dir='./', options=None, batches_per_step=1, export_popef=False, align_output_dtype=False)

Create a new session.

Parameters

  • path_or_bytes (Union[str, bytes]) – input onnx model

  • num_devices (int) – number of devices

  • backend (str) – one of ‘poprt’ and ‘onnxruntime’

  • output_dir (str) – output directory

  • options (Optional[Dict[str, Any]]) – options for session

  • batches_per_step (int) – number of batches per step

  • export_popef (bool) – True export PopEF, otherwise not export

  • align_output_dtype (bool) – align output dtype with onnx model

Return type

Session

class poprt.backend.Session(path_or_bytes, num_devices=1, output_dir='./', options=None, batches_per_step=1, export_popef=False)

Bases: object

Abstract base class for Session.

Parameters

  • path_or_bytes (Union[str, bytes]) – input onnx model

  • num_devices (int) – number of devices

  • options (Dict[str, Any]) – options for session

  • output_dir (str) –

  • batches_per_step (int) –

  • export_popef (bool) –

Return type

None

compile_and_export(serilization_path)

Compile the Model.

Parameters

serilization_path (str) – saved path

Return type

None

get_io_info()

Return meta info of input/outputs, include dtype, name, shape.

Return type

Dict

get_outputs()

Get outputs.

Return type

List[Any]

load(**kwargs)

Load Model.

Return type

None

run(output_names, inputs_dict)

Run the model.

Parameters

  • output_names (List[str]) – output tensor names

  • inputs_dict (List[Dict]) – input tensor Dict

set_options(options)

Set options for Session.

Parameters

options (Dict[str, Any]) – session options

Return type

None

set_output_dir(output_dir)

Set output directory.

Parameters

output_dir (str) – output directory

Return type

None

6.3. poprt.quantizer module

poprt.quantizer.quantize(onnx_model, input_model, output_dir, data_preprocess=None, quantize_loss_type='kld', options=None)

Quantize the model according strategy. At now, we only support SimpleQuantizer.

Parameters

  • onnx_model (ModelProto) – onnx ModelProto

  • input_model (str) – the origin model

  • data_preprocess (Optional[str]) – path of pickle format file for data preprocessing, the storage format is {input_name_1: ndarray_1, input_name_2: ndarray_2, …}

  • output_dir (str) – the output dir

  • options (Optional[Dict[str, Any]]) – options

  • quantize_loss_type (str) –

Returns

A quantized onnx ModelProto

Return type

ModelProto

class poprt.quantizer.FP8Quantizer(output_dir, loss_type, data_preprocess=None, options=None)

Return the Input Model.

Parameters

  • output_dir (str) –

  • loss_type (str) –

  • data_preprocess (str) –

  • options (Dict[str, Any]) –