<strong>About The Role<br><br></strong>Support AI/ML training pipelines by performing remote data annotation and evaluation across NLP, computer vision, and content safety. You will label and review datasets, complete RLHF preference ranking, and deliver rubric-based prompt and QA evaluations to improve training data quality and model performance.<br><br><strong>What You Will Do<br><br></strong><ul><li>Execute data labeling for text, image, and multimodal tasks using web-based tools</li><li>Perform named entity recognition (NER), text classification, and span labeling as required</li><li>Complete RLHF pairwise comparisons and preference ranking with clear rationale notes</li><li>Run prompt evaluation using rubrics for helpfulness, correctness, and safety</li><li>Conduct QA evaluation, sampling, and audits to ensure guideline compliance and consistency</li><li>Annotate computer vision data (bounding boxes, polygons, segmentation masks) when assigned</li><li>Perform content safety labeling for sensitive or policy-violating material when needed</li><li>Document edge cases, track disagreements, and propose guideline clarifications</li><li>Collaborate asynchronously with distributed teams to meet throughput and quality targets<br><br></li></ul><strong>Required Qualifications<br><br></strong><ul><li>Experience with professional data annotation, data labeling, or evaluation workflows</li><li>Strong written English for rubric-based judgments and structured feedback</li><li>Ability to follow detailed annotation guidelines and maintain consistency under QA review</li><li>Familiarity with NLP concepts (classification, NER) and/or computer vision annotation fundamentals</li><li>Comfort working with online labeling tools, spreadsheets, and documentation</li><li>Reliable remote work setup and availability for full-time schedules<br><br></li></ul><strong>Preferred Qualifications<br><br></strong><ul><li>Hands-on experience with RLHF evaluation, prompt evaluation, or LLM output grading</li><li>Experience conducting QA evaluation, calibration sessions, or inter-annotator agreement checks</li><li>Understanding of LLM failure modes (hallucinations, prompt injection, unsafe outputs)</li><li>Exposure to taxonomy/ontology or guideline authoring<br><br></li></ul><strong>Compensation<br><br></strong>Hourly pay range: <strong>$30–$50 USD (HOURLY)</strong>, based on task complexity, performance, and project needs.<br><br><strong>How To Apply<br><br></strong>Apply via Rex.zone and include your annotation experience plus examples of RLHF, prompt evaluation, and QA evaluation work that demonstrates strong guideline adherence and training data quality focus.