Senior Software Engineer - AI Interaction Evaluator

Accountabilities

• Evaluate AI-generated coding interactions end-to-end, assessing usefulness, correctness at a conceptual level, and alignment with strong engineering reasoning.

• Analyze whether AI explanations, preambles, and reasoning reflect high-quality developer thinking or introduce confusion or weak logic.

• Distinguish between varying levels of output quality and provide structured, opinionated assessments of model performance.

• Identify what makes interactions effective or ineffective, including clarity, trustworthiness, and engineering coherence.

• Provide detailed qualitative feedback on AI behavior, highlighting what worked, what failed, and where reasoning feels misleading or incomplete.

• Contribute to defining evaluation standards for high-quality AI-assisted development workflows.

• Help shape benchmarks for what “good engineering judgment” looks like in AI coding tools.

Requirements

You bring deep software engineering experience and strong technical intuition, enabling you to evaluate code quality and system behavior without relying solely on execution or line-by-line verification. You are comfortable making subjective but structured judgments and articulating why an engineering solution feels right or wrong. You have hands-on experience with modern programming workflows and AI-assisted development tools, and you understand what high-quality engineering output looks like in practice.

• Staff-level or Principal-level software engineering experience (or equivalent senior expertise)

• Strong proficiency in at least one core language: TypeScript/JavaScript or Python

• Experience using modern AI coding tools such as Codex, Claude Code, or Cursor

• Deep familiarity with AI-assisted development workflows and developer tooling

• Ability to evaluate code and system behavior at a conceptual level without full execution

• Strong engineering judgment and ability to define what “good” looks like in ambiguous scenarios

• Clear, direct communication style with ability to provide structured, opinionated feedback

• Experience mentoring engineers or defining engineering best practices (nice to have)

• Exposure to prompt engineering or AI evaluation workflows (nice to have)

Benefits

• Competitive hourly compensation: $100–$200/hour

• Flexible part-time engagement (approximately 10–20 hours/week)

• Remote work with high autonomy and async-friendly collaboration

• Opportunity to influence the development of next-generation AI coding tools

• Short-term contract with potential for extension

• Work directly at the frontier of AI-assisted software engineering evaluation

• Fast onboarding process with immediate start availability

How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
 Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1