Skip to content

Commit

Permalink
Update openai_gen.py to include reasoning behind should_manually_add_…
Browse files Browse the repository at this point in the history
…dot to openai_gen
  • Loading branch information
LakshyAAAgrawal committed Nov 21, 2023
1 parent 8814efc commit 022c65e
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions src/monitors4codegen/monitor_guided_decoding/openai_gen.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,9 @@ def convert_bytesrep_to_bytes(x: str) -> bytes:
dot_found = True
break

# When "stop" sequence is sent to openai model, it will not generate text beyond the text sequence within the "stop" parameter.
# However, when it stops because of the "stop" sequence, the returned text does not contain the stop sequence, and only includes
# text upto the stop sequence. So, the following code determines if the stop sequence "." needs to be added manually.
should_manually_add_dot = None
if response.choices[0].finish_reason == 'stop':
if dot_found:
Expand Down

0 comments on commit 022c65e

Please sign in to comment.