How do generate a list of variable length where each element follows a regex

[prnvbn]

I want to generate a list of elements of variable length (this is decided based on additional context provided) and also want each element in the list to follow a particular regex. A trivial example is generating a list of single digit numbers

llama2 =  models.Transformers("meta-llama/Llama-2-7b-hf", echo=False)
with block("my_text"):
    lm = llama2 
    lm = llama2 + '1 2' + one_or_more(gen(regex='\d', max_tokens=20))

I would assume that this generates something like 1 2 3 4

However, it generates 1 2 3 and stops. Is there a way to achieve what I want?

[honne23]

I have the same question, the only solution I can think of right now is to prompt the llm to decide how large the list should be using gen then use the control flow to then generate a list of the length decided by the model.

[honne23]

Here's a working example

@guidance
def decide(lm: models.Model):
    lm: models.Model = lm + "How many developers deserve a raise? Total: " + gen(regex='\d+', name="total")
    return lm

@guidance(stateless=True)
def gen_names(lm: models.Model, total: int):
    for i in range(total):
        lm = lm + f""" "{select(options=["Adrian", "George", "Harry", "Jeremy"], name="names", list_append=True)}" """
    return lm

llm = llm + f"""
Adrian and George are really good developers. Harry and Jeremy are very bad and should not get a raise."""
llm: models.Model = llm + decide()
llm: models.Model = llm + gen_names(total=int(llm._variables["total"]))
print(llm._variables)

Produces: {'total': '3', 'names': ['Adrian', 'Harry', 'Jeremy']}

[honne23]

Other methods I tried can produce unlimited length lists

[aadityabhatia]

Good prompt is the key here. One option would be to prompt the LLM to output "END" when it has completed its list. Another is to increase the temperature and tune other parameters to lean towards random sampling. Giving examples to the LLM really helps as well.