Class InferenceSession

java.lang.Object
com.codename1.ai.inference.InferenceSession
All Implemented Interfaces:
AutoCloseable
public final class InferenceSession extends Object implements AutoCloseable

Reusable, native on-device session for a TensorFlow Lite model.

Opening and execution are asynchronous because model allocation and delegates can be expensive. Metadata and resize operations are synchronous. A session is not usable after close(); applications should retain and reuse one session instead of reopening the model for every input.

  • Method Details

    • isSupported

      public static boolean isSupported()
      Tests whether the current port includes a native LiteRT runtime.
      Returns:
      true when sessions can be opened on this target

    • open

      public static AsyncResource<InferenceSession> open(ModelSource source, InferenceOptions options)

      Opens and allocates a model session off the EDT.

      The option values are copied before asynchronous backend work is scheduled. Reusing or changing the supplied InferenceOptions after this method returns therefore cannot alter the pending open. Canceling the returned resource prevents session publication; if the native backend finishes opening afterward, its handle is closed automatically.

      Parameters:
      source - bytes, resource, or file containing a .tflite model
      options - execution options; null uses defaults
      Returns:
      an asynchronous session, failed with InferenceException when the model or requested accelerator cannot be used

    • getInputs

      public TensorInfo[] getInputs()
      Returns the model's current input metadata. Shapes reflect the most recent successful resizeInput(String, int[]) call. Metadata cannot be queried while run(Tensor[]) is pending because the native interpreter is mutable and may be updating tensor state.
      Returns:
      a defensive copy of the input metadata array
      Throws:
      IllegalStateException - if the session is closed or a run is pending

    • getOutputs

      public TensorInfo[] getOutputs()
      Returns the model's current output metadata. Backends refresh this information after an invocation so models with dynamically resolved output dimensions report the shape used to decode the returned tensor. Metadata cannot be queried while run(Tensor[]) is pending because the native interpreter is mutable and may be updating tensor state.
      Returns:
      a defensive copy of the output metadata array
      Throws:
      IllegalStateException - if the session is closed or a run is pending

    • run

      public AsyncResource<Tensor[]> run(Tensor[] inputs)
      Copies input tensors to native memory, invokes the model, and returns every output tensor. Named tensors are matched by name; unnamed tensors are matched by position. Calling close() while this operation is pending prevents new work immediately but defers native release until the returned resource succeeds or fails. A session accepts one run at a time because the underlying native interpreter is mutable. The array container is defensively copied before it is handed to the asynchronous backend, so replacing an element after this method returns cannot change the pending invocation. Each Tensor is itself immutable. Canceling the returned resource suppresses result publication but does not interrupt an invocation that has already entered LiteRT. The session remains busy, and a pending close() remains deferred, until the native operation actually succeeds or fails.
      Parameters:
      inputs - one tensor for each model input; null is treated as an empty input array
      Returns:
      asynchronous output tensors in model output order
      Throws:
      IllegalStateException - if the session is closed or already running
      IllegalArgumentException - if an input name, count, or shape does not match the model's current input metadata

    • resizeInput

      public void resizeInput(String name, int[] shape)

      Resizes an input and reallocates native tensors before the next run.

      This method throws while an asynchronous run(Tensor[]) is pending because native runtimes cannot safely reallocate tensors during an invocation.

      Parameters:
      name - model input name, or null for the first input
      shape - new non-negative dimensions
      Throws:
      IllegalStateException - if the session is closed or a run is pending

    • close

      public void close()
      Releases the interpreter, delegates, and any temporary staged model. The original file supplied by ModelSource.file(String) is never deleted. If a run is pending, release is deferred until that run settles. Calling this method more than once has no effect.
      Specified by:
      close in interface AutoCloseable