gabyhelp feedback

[rsc]

I am experimenting with running a new bot @gabyhelp (which I am writing) in the Go issue tracker to try to help automate various mundane things that a machine can do reasonably well. Some aspects of the work may involve AI/LLMs; some will not.

Please thumbs up or thumbs down individual comments from the bot depending on whether they are correct/helpful or not. If the bot is getting in the way, we obviously want to know and fix it.

When the bot posts a comment, it will link back to this discussion page, which is a good place to leave non-boolean feedback. Thanks!

This discussion is also a good place to post ideas for things you wish a bot would do for you in the issue tracker.

To snoop around in all the issues where the bot has commented, you can use this search.

[rsc]

Right now, @gabyhelp is editing recently modified/posted comments and issues. Anywhere the plain text "CL nnn" appears, it changes the text to be a link to go.dev/cl/nnn. I've meant to do this for many years now, so that's the first chore.

[qiulaidongfeng]

See https://go.dev/doc/contribute#commit_messages , The rest of the description elaborates and should provide context for the change and explain what it does. Write in complete sentences with correct punctuation, just like for your comments in Go. Don't use HTML, Markdown, or any other markup language.
Does this mean that the Contribution Guide need to be revised？

[thepudds]

Hi @qiulaidongfeng, I think that section of the Contribution Guide is talking about avoiding HTML and Markdown in commit messages, vs. Russ above is talking about automatically editing GitHub issues.

[rsc]

@qiulaidongfeng The bot accidentally edited your PR description, because GitHub pretends that every PR is an Issue. The bot is only meant to be editing Issues, and I changed it to explicitly ignore PRs. Apologies!

[qiulaidongfeng]

For example :
@gabyhelp rewrite

CL 512155

to

[CL 512155](https://go.dev/cl/512155)

in #67902.

[CL 512155](https://go.dev/cl/512155)

isn't Markdown?

[rsc]

Yes, that's the PR that it accidentally edited. Again, my apologies.

[rsc]

@gabyhelp has also started posting to new issues (plus the issues created in the last two days, to get started) with lists of similar issues, if it finds any that are similar enough. If not, it will stay quiet.

Here is one success story: #67887 (comment). And here is another: #67897 (comment). This one is only a partial success because @thepudds got there first, but only because the bot gave him a head start. 😄

[dolmen]

This post by @gabyhelp is definitely useful: #67943 (comment).

[dmitshur]

In #68028 (comment), it's visible that gabyhelp currently doesn't filter out PRs in the list of potential "similar issues", which it probably should. GitHub API treats issues and PRs similarly, but PRs are a subset of CLs.

[rsc]

I think the only issue is the title (Similar Issues). In general if something is very related I think we should surface it. If an issue gets filed and gabyhelp posts hey there is a PR that seems to be about this issue, that's very helpful. The plan is to include all Gerrit CLs eventually too. (At that point we'd probably want to deduplicate GitHub PRs against Gerrit CLs.)

[dolmen]

@gabyhelp 's success is more about GitHub search being a big fail.

As an example I have just filled #68102 because a search of GOBIN didn't show anything relevant. See https://github.com/golang/go/issues?q=is%3Aissue+GOBIN+-author%3Adolmen

[magical]

I imagine this is on your list already, but in case not: another thing that would be nice would be for the bot to rewrite google-internal urls to their public counterparts.

[rsc]

Indeed. Thanks for the reminder. Done! And apologies again for having the internal-vs-external split at all. I'm just as annoyed by it as all of you. (It's literally just the same site with a different URL to force a different, stricter authentication path.)

[randall77]

It might be nice to have gabyhelp not generate emails when posting to issues. I see its updates, particularly related issues posts, as not needing a "ping". It's just additional information put there when you otherwise navigate to an issue.

[rsc]

@prattmic Sigh, I knew that GitHub used to format mails that way, but I thought surely by now they would have fixed it - I reported it to them at least a year three years ago! I work around that in the proposal minutes by writing out [TITLE](https://blahblah) explicitly. I guess I have to do that here too. Thanks for letting me know.

[rsc]

@prattmic This should be fixed now.

[hyangah]

On the other hand I miss the github's formatting that shows whether the issues are closed or not clearly, and gets updated. Is there a way to mix both somehow?

[rsc]

Sure. Once I get better tests written I will add "(closed)" markers.

[cespare]

@rsc but that doesn't cover "and gets updated" part. I guess the "(closed)" markers will just be a snapshot of the state at whatever time the bot processed that comment?

IMO all these options are worse than going back to the initial state where we let GitHub dynamically format the issue links and sacrifice a bit of readability in the emails.

[seankhliao]

I wonder if can be smart enough to recognize misformatted text, especially code blocks. We often see issues with no fences around code blocks or using single backticks (I assume this is from jira's formatting).

[deefdragon]

Related, I would love if it could add syntax highlighting to existing code-blocks that don't have it. It makes it so much easier for me to read when the highlighting is applied.

[rsc]

@seankhliao Can you point to specific issues with this problem? Thanks.

@deefdragon Different people have different but very strong opinions about syntax highlighting. I'm going to focus on clearer wins.

[seankhliao]

• unfenced code: Bug Report: JSON Deserialization Case Sensitivity Issue in Go #67689 , runtime: time.Sleep problem on Windows 10, but not Windows 7 or Linux #61042 , go v1.17.x breaks the (os.exec) Cmd.Start() functionality on Windows #49044
• unfenced stacktrace: net/http: Transport memory leak #43966 , Panic when trying to serve http with embedded assets #43761
• jira style(?) code blocks: crypto/rsa: can generate digital signature when hash algorithm (digest method) is set to MD5 in FIPS mode #45565

[seankhliao]

you'll need to look at the edit history for these, since I found them with a google search of "edited by seankhliao"

[ianlancetaylor]

Just a note that the similar issues list in #67861 was not helpful other than the reference to #33818. The issue #67861 was about changing sort.Sort (which we can't do), and the related issues are about other changes to sorting. We don't necessarily have to do anything here, the other issues weren't outrageously wrong, they just weren't helpful.

[cespare]

In #67948 (comment), the similar issues were not helpful.

The first listed issue, #56471, is very related -- as I wrote in the description, I created #67948 as a spin-off from #56471. Perhaps gabyhelp could be updated not to list issues which are already mentioned in the description?

The other four listed issues are superficially similar but not actually related in a useful way.

[jfrech]

I would feel alienated were I nudged to interact with an LLM by merely browsing Go development. I hope you will confine your bot to deterministic, comprehensible actions only.

[rsc]

I probably share your skepticism and concerns, but I don't fully understand the specific scenario you are concerned about. Can you say more?

[Merovius]

I think one thing I don't want to happen is a bot ask me clarifying questions, for example.

Another example of something I've seen LLMs used for which I consider insulting would be a bot re-phrasing comments.

I guess personally I'm mostly fine with AI use-cases I'd call "classification" but would feel patronized by any use-case that tries to emulate (or interlope in) human interaction.

[mvdan]

Would this bot eventually replace the rather basic "Gopher Robot" at https://go-review.googlesource.com/q/owner:[email protected]? It currently seems to rely on a few heuristics to tell PR/CL authors about common mistakes, such as in https://go-review.googlesource.com/c/build/+/591455. It seems rather similar to the bot discussed here.

[rsc]

I think in the long term yes we'd want it to be able to do everything the current Gopherbot does. Right now we're focusing on what it can do that's new.

[ianlancetaylor]

In #67934 gabyhelp correctly reported that the issue was similar to #67933. That is because they are both backport requests for the same original issue. Perhaps gabyhelp can skip backport issues.

[findleyr]

Thanks @seankhliao, you beat me to it. It would be nice if gabyhelp could recognize issues that are 'process'.

[rsc]

I already disabled it on watchflakes issues, and I will disable it on backports issues too.

I am happy to disable it on gopls release issues too. Is that any issue with title prefix "x/tools/gopls: release version v"?

[findleyr]

@rsc yes all gopls release issues follow that format.

[rsc]

Thanks. Updated to skip both backports and gopls releases:

	if strings.Contains(issue.Body, `— [watchflakes](https://go.dev/wiki/Watchflakes)`) ||
		strings.HasSuffix(issue.Title, ` backport]`) ||
		strings.HasPrefix(issue.Title, `x/tools/gopls: release version v`) {
		continue
	}

Eventually we will hopefully be able to tell the bot to do this kind of thing. 😄

[seankhliao]

also package removal requests?
#68003 (comment)

[dolmen]

The Go project has a redirector for permalinks.
Examples: https://golang.org/s/generatedcode https://go.dev/s/generatedcode (I don't know which one should be preferred nowadays).

It would be helpful if @gabyhelp could replace links to targets of the redirector (and hopefully, any previous targets if the short link has an history) by the short permalink.

[thanm]

For my newly filed issue #67953, the related issues that @gabyhelp annotated it with (in #67953 (comment)) are not really all that helpful. Yes, they are all problems with coverage, but don't really relate to the very specific problem at hand.

[rsc]

The related issues for my issue #67961 (filed using gopls bug) are all empty instances of the gopls bug template. They are 90% similar to mine. Maybe I was too concise in my report!

Probably we need to look for empty templates anyway for auto-closing and then exclude them. (It's easy to tell, the empty templates are all 99% similar to each other.)

[rsc]

Another potential thing to explore is trying to help deduplicate the automated crashes in https://github.com/golang/vscode-go/issues.

[MikeMitchellWebDev]

As a relative newcomer to filing issues, I found the Gaby bot helpful for re-categorizing the issue I am filing (i.e editing my own title after the Gaby bot makes its post) (#67984)

[findleyr]

In #67995 (comment), Gaby provided a remarkably accurate summary of the timeline of our feelings about file watching in gopls: the initial issue where we discussed the philosophy of client-side file watching, and then the most relevent issues where we struggled to work around client-side issues, culminating in the current https://go.dev/issue/67995 which I filed to explore server-side file watching.

I experienced delight :).

[thepudds]

In #61897 (comment), @gabyhelp appears to have incorrectly edited the text CL 591096 to link to https://go.dev/cl/ (without a CL number).

[rsc]

My fault, I noticed and fixed the bug pretty quickly after deploying that. I think that was the only one. And now I have tests. Just hadn't quite gotten to those yet.

[prattmic]

#68003 (comment) shows an escaping issue. Brackets in the issue title seem to have broken the markdown in the comment.

[rsc]

Oops, should be fixed now, thanks.

[griesemer]

#68054 (comment) was added by gabyhelp as similar issues. They are similar in the sense that they are compiler panics but they are otherwise very much unrelated.

[rsc]

The code running @gabyhelp is posted on my personal GitHub for now. See https://pkg.go.dev/rsc.io/gaby for docs and https://github.com/rsc/gaby for code. In the near future, we will move it somewhere more official and start taking contributions.

[rsc]

The @gabyhelp comment on #68212 is a nice illustration of why automatic "closed as duplicate" is so difficult. That one is a duplicate of the top related issue, but the problem is that the fix wasn't backported to 1.22 and 1.21.

[randall77]

#68220 is pretty clearly a dup of #68170, but gabyhelp didn't find it.

[rsc]

Thanks! I tried manually expanding the playground link in #68170 into an inline code snippet, and after that #68220 is a close enough match to be considered "related". I have been thinking about doing that anyway to make issues a bit more self-contained. Probably we can do that in the comment fixer before the indexing runs.

[peterGo]

[willfaught]

The name gaby is short for “Go AI Bot”

Where does the "y" come from?

[rsc]

gab is taken so i added a y.

[jfesler]

i'm glad we focus on the important things.

[mauri870]

Gaby posted a response with broken markdown in #68250 (comment)