Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

github: implement label functionality #64

Open
hyangah opened this issue Nov 26, 2024 · 16 comments
Open

github: implement label functionality #64

hyangah opened this issue Nov 26, 2024 · 16 comments
Assignees

Comments

@hyangah
Copy link
Contributor

hyangah commented Nov 26, 2024

We plan to classify issues and documents and label them. For GitHub issues, we need to

  • post labels: Like internal/commentfix, or internal/related, this can be implemented in a separate package (internal/label or internal/classify).

  • retrieve all available labels in the project.

@hyangah hyangah added this to the Labeling & Classification milestone Nov 26, 2024
@jba jba self-assigned this Nov 27, 2024
gopherbot pushed a commit that referenced this issue Dec 10, 2024
Add internal/labels, which can classify issues.
The categories are taken from internal/rules.

Add a page to gaby for manual review of the classification
decisions.

For #64.

Change-Id: Iaea615e9586f7f1af28c2876c4dbc35cea34d44e
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/634935
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Tatiana Bradley <[email protected]>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/634976 mentions this issue: gaby: labels page: skip labels authored by bots

gopherbot pushed a commit that referenced this issue Dec 11, 2024
For #64.

Change-Id: I9db76f39f41d2ad734257beb01b55d6de6705cfa
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/634976
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/635455 mentions this issue: internal/github: operations on repo labels

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/635457 mentions this issue: internal/devtools/labeleval: evaluate labels

gopherbot pushed a commit that referenced this issue Dec 12, 2024
Support creating, changing and listing a repo's labels.

For #64.

Change-Id: I9e3f5dd8b734e3d381a7886938d545744ed7f469
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/635455
Reviewed-by: Tatiana Bradley <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 12, 2024
Provide a CLI for comparing the LLM's chosen labels for issues with
the desired labels.

For #64.

Change-Id: I28f643ea0e5c6aebf62b8b49b8cd630c9ef556c2
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/635457
Reviewed-by: Tatiana Bradley <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/635855 mentions this issue: internal/labels,labeleval: minor changes

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/635875 mentions this issue: internal/labels: clean issue body

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/635975 mentions this issue: internal/devtools/cmd/labeleval: add more issues

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/635876 mentions this issue: internal/devtools/cmd/labelhist: show label history

gopherbot pushed a commit that referenced this issue Dec 13, 2024
Add a CLI tool for displaying the the labels added to and removed
from an issue.

This will be useful to evaluate labels added by gaby.

Example:

> go run ./internal/devtools/cmd/labelhist 69000
69000:
  2024-08-21T19:56:28Z cherrymui  +NeedsInvestigation
  2024-08-22T19:03:13Z dmitshur   -NeedsInvestigation
  2024-08-22T19:03:13Z dmitshur   +NeedsFix
  2024-08-22T19:03:30Z dmitshur   +Testing

For #64.

Change-Id: Ide5f429df92bfebd7978a8dfbb46b93708a6ab3d
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/635876
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Tatiana Bradley <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 13, 2024
Recategorize some bugs and clarify the meaning of a category.

After these changes, during one run, the only failure was 60017,
classified as a language proposal instead of a bug. But arguably
it _is_ a proposal ("Will be good to see composite literal being
supported directly inside range expression"), even though one based on
a misunderstanding.

Other runs reported other failures; the results are not repeatable.

For #64.

Change-Id: I4a3338741c2be3f60fcb1f1007677765ca9f4f92
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/635855
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Tatiana Bradley <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 13, 2024
Remove HTML comments and some headings that come
from the issue template.

This fixes the classification of 69040, an empty issue (see
internal/devtools/cmd/labeleval). Previously, the LLM was
confused by the comments and the presence of some headers.

For #64.

Change-Id: I82a68bba3f3aac9365b4d14035ed976b405651dc
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/635875
Reviewed-by: Tatiana Bradley <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 14, 2024
Also add a new category, featureRequest.

For #64.

Change-Id: I0409da33b4431d39bf72c3646b878c4fef3d8ea8
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/635975
Reviewed-by: Tatiana Bradley <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/636476 mentions this issue: internal/labels: copy descriptions from repo

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/636715 mentions this issue: internal/labels: switch config to YAML

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/636716 mentions this issue: internal/labels: add extra description

gopherbot pushed a commit that referenced this issue Dec 16, 2024
For the labels that exist in the golang/go repo, copy their descriptions
into our config. Also, reword existing label descriptions to follow the
common pattern used in the repo.

The labeleval tool shows that these changes make minimal differences:

--- /tmp/before-desc-changes.txt	2024-12-13 16:32:46.735167620 -0500
+++ /tmp/after-desc-changes.txt	2024-12-13 16:37:44.330625628 -0500
@@ -3,7 +3,7 @@
 69003  PASS:5 FAIL:0   accessRequest:5
 69005  PASS:5 FAIL:0   libraryProposal:5
 69008  PASS:5 FAIL:0   languageProposal:5
-69011  PASS:1 FAIL:4   question:1  bug:1  languageProposal:3
+69011  PASS:0 FAIL:5   bug:3  languageProposal:2
 69012  PASS:5 FAIL:0   bug:5
 69014  PASS:5 FAIL:0   bug:5
 69015  PASS:5 FAIL:0   bug:5

For #64.

Change-Id: Iaf7e0b3542cc2f852eb28ca8e178a0af307cebfa
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/636476
Reviewed-by: Ian Lance Taylor <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 16, 2024
Switch to YAML from JSON so we can add comments and include
multiline text.

For #64.

Change-Id: Ibbfaf4ec66d8f7bc064d21257943166b26617770
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/636715
Reviewed-by: Ian Lance Taylor <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 16, 2024
The description in the issue tracker might not be enough information.
Allow extra descriptive text about a category to be fed to the LLM.

Here I've included information about proposal scope from
https://go.dev/s/proposal. Although it didn't really help
with evaluation, I still think this feature is useful.

(Nothing I wrote, including phrases like "not gopls", could dissuade
the LLM from classifying go.dev/issue/69058, requesting a new gopls
feature, as a tool proposal.)

For #64.

Change-Id: I11563e065097e1176bd144f12c96c7f0285beaf2
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/636716
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Tatiana Bradley <[email protected]>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637096 mentions this issue: internal/labels: Labeler, constructor and config

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637155 mentions this issue: internal/labels: sync labels

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637097 mentions this issue: internal/labels: labeler: skipping

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637517 mentions this issue: internal/labels: labeler: Run

gopherbot pushed a commit that referenced this issue Dec 18, 2024
First of several CLs implementing Labeler, akin to related.Poster.

For #64.

Change-Id: I621b882cf6a779f8e81042e6d5654c0b2a51c8fc
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637096
Reviewed-by: Tatiana Bradley <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 18, 2024
Add a method that will reconcile the issue tracker labels
with the configured labels.

For #64.

Change-Id: I51492fcfdd0ced31257b0355a8570489d3402499
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637155
Reviewed-by: Tatiana Bradley <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 18, 2024
Add the initial part of Run, and the logic for skipping issues.

This is all copied almost verbatim from related.go.

For #64.

Change-Id: I6156c40310d7306449cbdfbd5fbc44e9b39a05fd
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637097
Reviewed-by: Tatiana Bradley <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
gopherbot pushed a commit that referenced this issue Dec 18, 2024
Add Run method to Labeler.

Based heavily on related.Poster.Run.

For #64.

Change-Id: I4818214eb0f8bd00294cde47d41cad943b28d74e
Reviewed-on: https://go-review.googlesource.com/c/oscar/+/637517
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Tatiana Bradley <[email protected]>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637855 mentions this issue: internal/labels: labeler: modify GitHub

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/637856 mentions this issue: internal/labels: generalize to multiple projects

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants