Debug studio

[zolinthecow]

Motivation

When developing apps (especially more complex ones) with SGLang's frontend it's a bit hard to tell what is actually getting passed into the generate/ endpoint. This PR introduces a "prompt debugging studio" that allows you to turn on a debug mode and will start intercepting generate requests and saving them to a debug server, where you can go to a web interface and view all the prompts and responses for SGL functions.

For example, if you start the debug server:

python -m sglang.launch_debug_server

It will start a debug server on http://0.0.0.0:56765

Then, you can add a debug region to an sgl function:

@sgl.function
def text_qa(s, question):
    s.begin_debug_region("TEXT_QA")
    s += "Q: " + question + "\n"
    s += "A:" + sgl.gen("answer", stop="\n")

state = text_qa.run(
    question="What is the capital of France?",
    temperature=0.1,
    stream=True
)

The s.begin_debug_region will start debugging under the prompt name "TEXT_QA". You can then go to http://localhost:56765 (will have to ssh forward if you're developing on a remote server) and you should see prompts start appearing

Modifications

• Adds BeginDebugRegion and EndDebugRegion to SGLExpr for the frontend language
• Adds launch_debug_server which under the hood relies on enochian-studio (one of my packages) for the debug server
  • enochian-studio can be found here, I can say it's up to date since I maintain it, along with Etched
• Adds posting to the debug server if enabled for all backends

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[zolinthecow]

^ Couldn't reproduce CI error locally :(

[zhaochenyang20]

@zolinthecow Nice work! I will start to review this PR this weekend. Stay tuned!