Issues

Open and talk about labeling issues for all reviewers and managers to see.

Labelers, reviewers, and managers can open issues on labeling tasks to have a conversation about a particular asset or label.

Labelers can open issues on the tasks they have been assigned to, for example, to clarify how a certain asset needs to be labeled. Mentioned reviewers and project managers will get notified of the new issue so that they may take action and respond.

Reviewers can open issues on tasks to, for example, communicate to the labeler how they should label similar tasks in the future. Labelers get notified of the opened issues opened this way.

Users can have real-time conversations on issue threads, mention people to be notified, and delete/resolve issues.

Opening and Viewing Issues on Ango Hub

How to Open an Issue

General Task Issue

Users can open issues that do not point to a specific region of the asset, and are about the task in general.

To do so, open the task you'd like to open an issue in. From the right-hand sidebar, click on Issues and write your issue in the box. Mention other users if necessary by @ing them, and press Enter. You can also press "I" on your keyboard if no object is selected to quickly open a task-level issue.

You can only mention users which are also members of the project.

When you get a reply, you will get a notification.

You can edit or delete the issue using the Edit and Delete buttons on the issue.

Specific Area Issue

In visual labeling tasks, you can place an issue marker on the image to direct attention to a particular region of the asset related to your issue. This is available for images, videos, PDF, and medical data.

To do so, open the task in which you'd like to open an issue, and click on the speech bubble icon in the top-right corner of the screen. Then, click on the asset region of interest. Type your issue and press Enter.

Issues created this way will also be visible in the right-hand sidebar.

If an issue is obstructing your view, you can hide it by clicking on the icon, and you can unhide it by clicking it again from the sidebar.

For medical data, you can additionally draw a line when the issue cursor is active, to further point at a specific area when opening a Specific Area issue:

Object-Level Issues

To open an issue tied to a particular object, right-click on the object, expand the context menu, click on the three dots and choose Open Issue:

Alternatively, from the Objects list on the bottom left of the screen, click on the three dots on the row of the object you'd like to open the issue on, then click on Create Issue.

Alternatively, click on an object, then press "I" on your keyboard.

Classification-level Issue

Click on the three dots next to the classification answer you'd like to create an issue about, and click on Create Issue:

Multi-Image Issues

In multi-image assets, you may open an issue spanning multiple images.

To do so, navigate to the image where this issue starts occurring. Then, select the issue "speech bubble" icon and click anywhere on the asset to open the Open Issue dialog:

At the End selector, pick the image number where the issue ends. Click on the arrow to create the issue. You'll be able to see the issue page range when browsing issues in the issue sidebar, as well as in the issue export.

In the issue export:

Issue object with page range in issue export

{
  "_id": "65364c3bc1407000158d96d3",
  "points": [],
  "status": "Open",
  "errorType": "Comment",
  "labelTask": "651d596274373600156401ad",
  "asset": {
    "_id": "651d596274373600156401a9",
    "dataset": [
      "URLs"
    ],
    "batches": [],
    "externalId": "my-asset-external-id-1",
    "data": "https://angohub-public-assets.s3.eu-central-1.amazonaws.com/3e6e15b9-c32c-4b73-97f9-dbda784926ab.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIATAGM6WLISC5CRH7S%2F20231004%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20231004T122403Z&X-Amz-Expires=120000&X-Amz-Signature=d8f4e7ba5794faf2e5b27a83375b93bc234935c930bf371fdcb936e66fd8f012&X-Amz-SignedHeaders=host&x-id=GetObject",
    "head": "https://angohub-public-assets.s3.eu-central-1.amazonaws.com/3e6e15b9-c32c-4b73-97f9-dbda784926ab.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIATAGM6WLISC5CRH7S%2F20231004%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20231004T122403Z&X-Amz-Expires=120000&X-Amz-Signature=d8f4e7ba5794faf2e5b27a83375b93bc234935c930bf371fdcb936e66fd8f012&X-Amz-SignedHeaders=host&x-id=GetObject"
  },
  "project": "64e5e49785eb730015b4eb7b",
  "position": "[266.8706411698538,280.4731774415406]",
  "content": "I'm not sure how to annotate this car between these frames.",
  "stage": "Complete",
  "contentMentions": [],
  "page": 0, // Page start, zero-indexed
  "pageEnd": 3, // Page end, zero-indexed
  "createdBy": "lorenzo@example.net",
  "organization": "64db3c87c14b35001503e10b",
  "createdAt": "2023-10-23T10:34:35.018Z",
  "updatedAt": "2023-10-23T10:34:35.018Z",
  "comments": [],
  "stageId": "Complete"
}

Viewing and Responding to Issues

To view and respond to issues in the project, go to the Issues tab and click on the issue of interest:

You will be directed to the task where the issue was opened, and the issue will be highlighted for you. To respond, simply type your response in the box and press Enter:

If you believe your issue was resolved, click on Resolve on the top-right corner of the issue.