Metrics Reloaded: The Oracle

Tech Stories Tech Brief By HackerNoon

This story was originally published on HackerNoon at: https://hackernoon.com/metrics-reloaded-the-oracle.
Ground truth and production metrics complement each others for ensuring performance.
Check more stories related to tech-stories at: https://hackernoon.com/c/tech-stories. You can also check exclusive content about #ai-assistant, #performance-evaluation, #conversational-ai-agents, #ground-truth-metrics, #user-perceived-metrics, #user-reported-metrics, #data-points, #automated-lab-testing, and more.
This story was written by: @pmukherjee. Learn more about this writer by checking @pmukherjee's about page, and for more stories, please visit hackernoon.com.
Focus on how to obtain these metrics in a manner that is easy to use, accurate and actionable. Ground truth and production metrics complement each others for ensuring performance. These two metric pipeline should be used to improve each other. Metric collection from production should not affect degrading performance. Metrics should dictate what logs needed to be added. Logs should be harmless and simple. Metrics derivation and logs should be orthogonal. We discuss more details on scaling performance metrics infrastructure in the final part of the trilogy