Tuning the benchmark in real world

Tuning the benchmark in real world

About the session

Ahmed Bashir and Ritesh Goru discuss the context engineering behind a more useful AI teammate, from system prompts and skills to tool guidance, hooks, and model selection.

They connect those design choices with benchmarking and real-world signals: ways to test changes, learn from failures, and improve an agent over time.

Key takeaways

  • Identify the context and guidance choices that shape an agent's responses.
  • Examine how benchmarking can test changes to the AI experience.
  • Use real-world signals and failures to inform iteration.

Agenda

  1. Context engineering for AI teammates

  2. Prompts, skills, tools, hooks, and model selection

  3. Benchmarking and real-world feedback

Speakers

  • Ahmed Bashir

    Ahmed Bashir

    Chief Technology Officer, DevRev

  • RG

    Ritesh Goru