-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support stream: true for the completions endpoint #11
Comments
It seems you might be able to by setting the HTTParty connection adapter to support the HTTP gem and then perhaps use https://github.com/launchdarkly/ruby-eventsource which I see you've worked on? |
@bf4 Interesting suggestion. I don't have time to work on it right now, but I'd welcome a PR if you're so inclined. |
Took a stab at running through the connection set up.. not sure if the difference in assumption with how Net::HTTP builds a connection with the uri, while HTTP waits until the actual request is easy to resolve. Faraday wrapping HTTP might be a better match since the adapter needs to mimic Net::HTTP, but otherwise, got pretty close, I think. investigative spike def connection
# https://github.com/jnunemaker/httparty/blob/30f2ec15917ef75934e7e7b139d9c92c22141093/lib/httparty/connection_adapter.rb
# https://github.com/httprb/http/blob/462d711040735be3661068e9e52eb14a4269f1f4/spec/lib/http_spec.rb#L14
# HTTP::Client.new
http = ::HTTP
# host = clean_host(uri.host)
# port = uri.port || (uri.scheme == 'https' ? 443 : 80)
if options.key?(:http_proxyaddr)
http = http.via(
options[:http_proxyaddr],
options[:http_proxyport],
options[:http_proxyuser],
options[:http_proxypass]
)
# http = Net::HTTP.new(
# host,
# port,
# options[:http_proxyaddr],
# options[:http_proxyport],
# options[:http_proxyuser],
# options[:http_proxypass]
# )
# else
# http = Net::HTTP.new(host, port)
end
# http.use_ssl = ssl_implied?(uri)
attach_ssl_certificates(http, options)
timeout_options = {}
if add_timeout?(options[:timeout])
# http.open_timeout = options[:timeout]
# http.read_timeout = options[:timeout]
timeout_options[:connect] = options[:timeout]
timeout_options[:read] = options[:timeout]
from_ruby_version('2.6.0', option: :write_timeout, warn: false) do
# http.write_timeout = options[:timeout]
timeout_options[:write] = options[:timeout]
end
end
if add_timeout?(options[:read_timeout])
# http.read_timeout = options[:read_timeout]
timeout_options[:read] = options[:read_timeout]
end
if add_timeout?(options[:open_timeout])
# http.open_timeout = options[:open_timeout]
timeout_options[:connect] = options[:open_timeout]
end
if add_timeout?(options[:write_timeout])
from_ruby_version('2.6.0', option: :write_timeout) do
# http.write_timeout = options[:write_timeout]
timeout_options[:write] = options[:write_timeout]
end
end
if timeout_options.any?
http = http.timeout(**timeout_options)
end
# if add_max_retries?(options[:max_retries])
# from_ruby_version('2.5.0', option: :max_retries) do
# http.max_retries = options[:max_retries]
# end
# end
# if options[:debug_output]
# http.set_debug_output(options[:debug_output])
#
# logger = Logger.new(STDOUT)
# http = HTTP.use(logging: {logger: logger})
# end
# if options[:ciphers]
# http.ciphers = options[:ciphers]
# end
# Bind to a specific local address or port
#
# @see https://bugs.ruby-lang.org/issues/6617
# if options[:local_host]
# from_ruby_version('2.0.0', option: :local_host) do
# http.local_host = options[:local_host]
# end
# end
# if options[:local_port]
# from_ruby_version('2.0.0', option: :local_port) do
# http.local_port = options[:local_port]
# end
# end
http
end |
The completions endpoint supports a streaming configuration, which may be valuable in situations where low latency is a priority and a stream of partial values to the caller is useful (i.e. for a chat client). This configuration uses server-sent events (SSEs) to incrementally provide updates to the caller.
I'm not entirely sure how to use the underlying HTTParty stack with server-sent events, assuming that's even possible. Needs further research and thought.
The text was updated successfully, but these errors were encountered: