Qwak prediction timers help troubleshoot model prediction latency, measure prediction times, find bottlenecks, and optimize performance.
Configure multiple timers with custom names for clear visibility using either a context manager or a decorator.
After building, deploying and sending inference requests to your model, you can view these timers on the Latency breakdown graph.
Note: The prediction timer graph may display up to 5 timers including the default
predicttimers. This leave room for additional 3 custom timers.
Qwak timers may be configured via a function decorates manager, to easily wrap different methods that are called during the prediction process.
from qwak import qwak_timer from qwak.model.base import QwakModel class MyModel(QwakModel): def build(): pass @qwak_timer("my_custom_timer_name") def pre_process(self, df): # Perform custom processing return df def predict(self, df): self.pre_process(df) return df
Qwak timers may be configured via a context manager, to easily wrap parts of your code that need measurements.
from qwak import qwak_timer from qwak.model.base import QwakModel class MyModel(QwakModel): def build(): pass def predict(self, df): with qwak_timer("my_custom_timer_name"): # Perform custom processing df = df + 1 return df
Measure the end to end inference time, starting at the moment a request arrived, until the inference output.
Measures the the time it takes requests to leave the input queue before reaching the model.
Updated 3 days ago