ecal.calculators.inference

Classes

Inference(model_name, input_size, ...[, ...])

This class is used to estimate the flops of the model inference, which is then used to estimate the energy consumption of the model inference.
class ecal.calculators.inference.Inference(model_name, input_size, num_samples, processor_flops_per_second, processor_max_power, calculator=None)[source]

Bases: object

This class is used to estimate the flops of the model inference, which is then used to estimate the energy consumption of the model inference.

Parameters:
__init__(model_name, input_size, num_samples, processor_flops_per_second, processor_max_power, calculator=None)[source]

Initialize Inference class :param calculator: FLOPCalculator implementation :param model_name: PyTorch model or model name :param input_size: Tuple of input size :param num_samples: int of number of samples :param processor_flops_per_second: float of processor flops per second :param processor_max_power: int of processor max power in watts

Parameters:
calculate_energy()[source]

Calculate the energy usage of the current inference

Returns:

Total energy usage of the current inference in Joules

Return type:

float

calculate_flops()[source]

Calculate total FLOPs for the current inference workload

Returns:

Total FLOPs across num_samples inference calls (a single forward pass’s FLOPs multiplied by num_samples)

Return type:

float