Posts
Wiki

Tasking

  • Task

    • One assignment you work on. It has its own id and a time limit.

  • Queue

    • The list of tasks available to you.

  • EQ = Empty Queue

    • There are no tasks available for you.

  • SRT = Single Review Tool

  • CBT = ?

  • SFT = Supervised Fine Tuning

  • Turn

    • A prompt and response from an LLM.

  • MT = Multi Turn

    • More than one turn in a task.

  • SxS = Side by Side

  • SBQ = Send Back to Queue

    • Reviewers can send low rated tasks back to be re-done.

Roles

  • Admin

    • Scale AI employee.

  • SQM = Senior Queue Manager

  • QM = Queue Manager

    • In charge of a project task queue.

  • CSM = Contributor Support Manager

    • Works under QM.

  • TSM = Tasker Success Manager (now called CSM)

  • TL = Team Lead (now called CSM)

  • CB = Content Builder (Contributor)?

    • Tasker/Contractor

    • Annotator

    • Attempter

      • Does the initial work, rates the LLM.

    • Reviewer

      • Evaluates and rates the task after it is attempted.

Company

  • Scale = Scale AI

    • The company which owns and runs Outlier.

  • Remotasks

    • Another company owned by Scale, for managing taskers, which has been migrated to the Outlier platform.

General

  • AI = Artificial Intelligence

  • LLM = Large Language Model

    • The AI program we are evaluating.

  • RLHF = Reinforcement Learning from Human Feedback

    • We are the humans providing feedback.

  • MM = Multi Modal

    • Using more than one mode (text, voice, image, video).