Metrics

etalon provides a set of metrics to evaluate the performance of LLM inference systems. These metrics are designed to capture the nuances of LLM inference process and its impact on real-time user experience.

etalon also enables visualizing these metrics to understand and compare the performance across different LLM inference systems.

Check the following resources to learn more:

• Metrics used by etalon
  • Time to First Token (TTFT)
  • Time Between Tokens (TBT)
  • Time Per Output Token (TPOT)
  • Normalized Latency
  • fluidity-index
  • fluid token generation rate
• Visualizing Metrics