Overview
STEM Careers Canada Jobs in Cranston, USA at Rex.zone
About The Role
This remote, full-time role supports AI/ML training workflows through high-quality data labeling, RLHF-style preference ranking, prompt evaluation, and QA evaluation. You will apply clear rubrics and annotation guidelines to improve training data quality and measurable model performance.
Key Responsibilities
- Perform high-accuracy data labeling for text, image, and multimodal datasets
- Execute RLHF evaluation (preference comparisons, rubric-based scoring)
- Run prompt evaluation and response grading for LLM evaluation
- Apply content safety labeling and policy-aware categorization
- Maintain annotation guidelines compliance and propose guideline improvements
- Conduct QA evaluation, audits, and disagreement resolution to ensure training data quality
- Document edge cases clearly and support distributed, asynchronous collaboration
Required Qualifications
- Mid-Senior experience in evaluation, annotation programs, or engineering-adjacent data operations
- Strong consistency applying detailed rubrics with low error rates
- Familiarity with LLM training pipelines and preference-based grading (RLHF)
- Working knowledge of NLP concepts (e.g., named entity recognition)
- Exposure to computer vision annotation workflows preferred
- Strong written communication for QA notes and edge-case documentation
Compensation
Competitive base pay: $30–$50 per hour.
Remote, full-time; async-first collaboration.
#J-18808-Ljbffr
Title: STEM Careers Canada
Company: Rex.zone
Location: Cranston, USA
Category:
