Senior Python Software Engineer – Coding-Agent Safety Evaluation

About the role

We are looking for a senior Python software engineer to design and evaluate realistic coding‑agent benchmark tasks. The role blends production engineering, secure software development, repository validation, test design, and safety evaluation to ensure coding agents operate within strict privacy and data‑integrity constraints.

Key responsibilities

• Build production‑like benchmark tasks using repositories, tests, configuration, documentation, and runtime scenarios.
• Create realistic engineering tasks such as bug fixes, feature additions, CI repairs, configuration migrations, integration updates, and runtime‑state fixes.
• Define utility, safety, and alignment requirements that protect system constraints, developer intent, permissions, privacy, and data integrity.
• Develop visible and hidden test suites and produce safe and unsafe reference solutions for controlled evaluation.
• Identify unsafe shortcuts (e.g., disabling tests, weakening assertions, leaking secrets) and package prompts, metadata, runnable repositories, evaluators, reference patches, scoring rubrics, and calibration notes.
• Analyze coding‑agent rollouts for utility completion and safety violations.
• Collaborate with engineering, QA, security, and client teams to improve evaluator reliability and benchmark difficulty.

Required profile

• Bachelor's or Master's degree in Computer Science or a related technical discipline.
• Minimum 8 years of hands‑on software engineering experience in tier‑one product companies or technology startups.
• Strong professional Python experience plus proficiency in at least one of Java, Go, or C++.
• Experience building and maintaining production software used by real customers.
• Solid background in software architecture, debugging, and performance optimization.
• Production‑engineering expertise covering deployment pipelines, monitoring, logging, observability, metrics, distributed tracing, high availability, fault tolerance, scalability, and disaster recovery.
• Secure‑software experience identifying vulnerabilities and implementing remediation in areas such as injection, authentication/authorization, memory safety, race conditions, deserialization, secrets management, and input validation.

Required skills

• Python
• Java
• Go
• C++
• Production engineering
• Deployment pipelines
• Monitoring, logging, observability, metrics, distributed tracing
• High availability, fault tolerance, scalability, disaster recovery
• Secure software development
• Vulnerability identification and remediation
• Injection prevention
• Authentication and authorization
• Memory safety
• Race condition handling
• Deserialization safety
• Secrets management
• Input validation