AI Quality Analyst – Bulgarian (Remote, $15/hr)

About the role

We are looking for a detail‑oriented AI Quality Analyst fluent in Bulgarian to evaluate and improve the performance of our conversational AI models. This remote, hourly‑contract position involves assessing model responses for personalization, relevance, and accuracy.

Key responsibilities

• Evaluate AI model outputs for grounding, integration, helpfulness, and personalization quality.
• Design and execute multi‑turn prompts based on personal context to test model capabilities.
• Identify hallucinations, incorrect personalization, and poor inferences in responses.
• Perform side‑by‑side comparisons of model outputs to determine quality and effectiveness.
• Write clear, structured rationales for response evaluations and rankings.
• Extract and verify debug information to ensure proper data source usage.
• Maintain strict data hygiene and accurate documentation of evaluations.
• Collaborate with cross‑functional teams to enhance AI model performance.

Required profile

• Strong proficiency in written and spoken Bulgarian.
• Experience in data annotation, AI evaluation, content moderation, or a related field.
• Analytical mindset with the ability to assess nuanced AI responses.
• Creative ability to design multi‑turn prompts based on personal context.
• Understanding of personalization concepts and ability to spot forced or incorrect personalization.
• High attention to detail and excellent written communication.
• Self‑motivated and able to work independently in a remote setting.
• Availability for full‑time work with at least 4 hours overlap with PST.
• Bachelor’s degree or equivalent analytical experience.

Required skills