Official LLM support

[Burnsedia]

Describe the project you are working on

a node that connect to a local llm

Describe the problem or limitation you are having in your project

I want to use local open source LLMs to make some of the contect, the docs on the plugins that do so are in need of better docs

Describe the feature / enhancement and how it helps to overcome the problem or limitation

the ability to use llm in godot nativly on device

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

a node that takes a prompt and handles the rest

If this enhancement will not be used often, can it be worked around with a few lines of script?

There are plugins but I fell it would drive adoption

Is there a reason why this should be core and not an add-on in the asset library?

my game npc can hold a converstaion

[Meorge]

Even if it wasn't opposed and/or disliked by many game devs, this definitely sounds like something that would fall in the domain of plugins, perhaps via GDExtension.

[AThousandShips]

It's exactly the area for a plugin, it's a perfect example of "this can be used for game development, but it isn't needed for game development", so even beyond the legal issues of these things it just isn't something that belongs in the engine

[Burnsedia]

ok, there are plugins, I just want a more official one. The one I tried had hard to follow docs.

…

On Mon, Sep 30, 2024 at 2:33 PM Malcolm Anderson ***@***.***> wrote: Even if it wasn't opposed and/or disliked by many game devs, this definitely sounds like something that would fall in the domain of plugins, perhaps via GDExtension. — Reply to this email directly, view it on GitHub <#10864 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH4ZR5YATWAKVMYYAPU3CCTZZGKPTAVCNFSM6AAAAABPD2VTRKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBTHA4TOOJXHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Meorge]

The quality of the documentation from a node officially supported by Godot will still be only as good as the people maintaining it are interested in it for. In other words, if no one is interested in maintaining an official Godot implementation, then it will languish too.

On the practical side of implementing it, there's also the issue of which LLMs to provide APIs for. If there was some kind of universal API that Godot could hook up to, then it'd be much more feasible, but I'm not aware of one. So maintainers would have to either choose a subset of LLMs as the ones to support, and then ensure relative feature and stability parity between them.

I think your best bet would be to suggest improvements to the documentation for the plugins you found, and/or contribute to them by improving the documentation as you learn more about them.

[Calinou]

Moving to discussion, as there is no concrete technical implementation provided.

[Jesusemora]

Llms weight gigabytes and take entire server rooms to generate. And the AI boom is a fad that has already bankrupted many companies.

[Burnsedia]

I don't think it is a fad, I use it in my company

…

On Tue, Oct 1, 2024, 2:37 a.m. Jesusemora ***@***.***> wrote: Llms weight gigabytes and take entire server rooms to generate. And the AI boom is a fad that has already bankrupted many companies. — Reply to this email directly, view it on GitHub <#10865 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH4ZR534C3STLNVTIXMY5Z3ZZI7KLAVCNFSM6AAAAABPEIQ3FCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBQGUYTOMI> . You are receiving this because you authored the thread.Message ID: <godotengine/godot-proposals/repo-discussions/10865/comments/10805171@ github.com>