Model format

SSF is framework agnostic and does not enforce a specific model serialization format. The only restriction is that your ML model must be callable from Python. We describe how to adapt any ML model so it can be used in SSF below.

For detailed documentation of all the SSF Config fields, see config detail.

Create an application interface

Create a Python file local to your application code. In that file, create an application specific concrete class inheriting from SSFApplicationInterface:

from ssf.application import SSFApplicationInterface, SSFConfig
class MyApplication(SSFApplicationInterface):
  ...

Implement the required methods:

class MyApplication(SSFApplicationInterface):
    def __init__(self, ssf_config : SSFConfig):
        pass

    def build(self) -> int:
        """
        Build required dependencies.
        This could be building custom-ops, binary executables, model-preprocessing, and so on.
        Consider running the model to capture a PopEF file as an optimisation.
        This is called when `gc-ssf build` is issued.

        Returns:
                0 (RESULT_OK) if successful.
        """

    def startup(self) -> int:
        """
        One-time startup for the application instance.
        Consider priming the application instance by issuing a dummy request.
        This is called during `gc-ssf run` before requests are started.

        Returns:
                0 (RESULT_OK) if successful.
        """

    def request(
        self, params: Union[dict, list], meta: Union[dict, list]
    ) -> Union[dict, list]:
        """
        Request for inference.
        This is called by the dispatcher for each queued request while `gc-ssf run` is running.

        Parameters:
                params (dict | list): Input parameters as a dictionary. Dictionary fields must match inputs declared in the SSF config.
                meta (dict | list): Metadata fields such as `endpoint_id`, `endpoint_version` and `endpoint_index` to support multiple or versioned endpoints.
        Returns:
                Output parameters as a dictionary or list of dictionaries. Dictionary fields must match outputs declared in the SSF config.
        Note:
                When `max_batch_size` in the SSF config is greater than 1 then input parameters will be a list of dictionaries and the respective return values must be a list of dictionaries.
        """

    def shutdown(self) -> int:
        """
        One-time shutdown for the application instance.
        This is called during `gc-ssf run` when requests are stopped.

        Returns:
                0 (RESULT_OK) if successful.
        """

    def watchdog(self) -> int:
        """
        Called after a period of request inactivity to check the application instance is still ready to receive requests.
        If the application instance has an unrecoverable failure then its watchdog can return failure which will cause the server to restart the application instance.
        If failures can not be detected, or they are handled internally, the default implementation can be used.

        Returns:
               0 (RESULT_OK) if successful.
        """

(Optional) You can declare your application's class __init__ method with or without the named argument ssf_config. If used, then the SSF config will be passed in as a copy of the SSF internal run-time configuration. This may be useful to the application if it needs to adapt at run-time based on SSF config details. The following fields are provided:

ssf_config.config_dict : The SSF config (YAML) as a dictionary, after expansion (see NOTES). You may add application-specific custom fields and sections in the SSF config and reference these in your application interface using ssf_config.config_dict. See also, CLI argument --modify-config.
ssf_config.args : The SSF CLI arguments for this SSF invocation, as an argparse Namespace object in ssf_confing.args.
ssf_config.unknown_args : The SSF CLI arguments for this SSF invocation that are unknown. The SSF CLI is tolerant to unknown arguments so it is possible to pass through arguments to your application.

Warnings:

The ssf_config object contains other fields that are implementation dependent and subject to change between versions of SSF.

(Optional) You can also declare a custom factory method named create_ssf_application_instance that will return your SSFApplicationInterface instance. For example, if you need to pass custom arguments to the init method.

def create_ssf_application_instance() -> SSFApplicationInterface`
  return MyApplication()

Or, if you want to pass through the SSF config,

def create_ssf_application_instance(ssf_config) -> SSFApplicationInterface`
  return MyApplication(ssf_config)

See examples/simple/my_application.py for more details.

(Optional) Return code

When you implement SSFApplicationInterface methods, you can use ssf.application_interface.results to set a return code for the build methods .

from ssf.application_interface.results import *

See result codes.

Create an SSF Config

Create an SSF config ssf_config.yaml file next to your application code (where you created the application interface module).

See examples/simple/ssf_config.yaml for more details.

Your SSF config file should define the version of SSF for which this config was written and tested. For example:

ssf_version: 1.0.0

It should also include top-level application details, for example:

application:
  id: simple_test
  name: Test API
  desc: A very simple test API.
  version: 1.0
  module: my_application.py
  ipus: 0

Where:

id: (str) Unique identifier for the application (no spaces)
name: (str) Human readable one-liner title
desc: (str) Human readable expanded description
version: (str) Application-specific versioning
module: (str) The name of your application module in which SSFApplicationInterface implementation can be found
ipus : (int) The number of IPUs required to run one instance of the application; can be zero; if not specified then SSF assumes 1 IPU is required

You should also add a list of endpoints. Multiple endpoints can be declared.

These may have the same (if only version is changing) or different IDs. Calls to the application request interface include metadata such as the endpoint ID, version and list index plus the replica index of the application instance.

For example:

endpoints:

  - id: Test1
    version: 1
    desc: Test1

Where:

id: (str) Endpoint unique id (no spaces)
version: (str) Endpoint version
desc: (str) Human readable description

    inputs:
      - id: x:
        type: Integer
        desc: A value
        example: 10


    outputs:
      - id: requests:
        type: Integer
        desc: Count of requests.

      - id: x_times_1000:
        type: Integer
        desc: Input value x times 1000.

Where:

id: Parameter id (name)
type: Type (one of the SSF defined types)
desc: Human readable one-liner description
example : Example value for this parameter