Help in Building an Educational Product

[brunoalbertopeixoto]

@Vali-98

Hello, my name is Alberto, and I am a master's student in Brazil with no programming experience. I need to build an educational product for the completion of my course. My proposal is a chatbot that generates a problem from a prompt and, through a series of guided questions, leads the student to reflect on the presented issue. The questions should be fixed, but the chatbot's response to the student's answer should not be. The product is designed as an activity for foreign language teaching. If possible, I would like the chatbot to have the ability to clone voices and use TTS (text-to-speech), but text alone would be sufficient. The idea is for it to work offline.

However, since LLMs (Large Language Models) are resource-intensive and mobile hardware, especially that of students, is not very powerful, I thought of using Oobabooga (LLM webui) (oobabooga/text-generation-webui) or Jan (https://jan.ai/) or other existing alternatives so that the heavier LLM functions run on a PC acting as an API for ChatterUI. I thought of making a fork of ChatterUI, already configured and adjusted to the learning layout, to make it easier for students and teachers to just install and start using it. After configuring it, I would like to disable some functions and windows of ChatterUI. Since I don't know how to program, I plan to use GitHub Copilot to help with the more structural changes to the forked app.

However, I am facing basic issues with ChatterUI and with the structure of the guided interaction flow for the chatbot. I would like to know if you can assist.

The first difficulty seems simple: How should I configure the mobile phone so the chatbot responds with voice? I believe the voice is from the phone's TTS, but the app did not recognize the installed voices. I need a realistic voice. I am using this one: https://github.com/jing332/tts-server-android because the voices are good from Microsoft Edge. Do I need to install something or configure it? How should I do that?

[brunoalbertopeixoto]

@Vali-98 Another difficulty I’m facing is how to configure remote access via server as an API using the LLM webui applications mentioned (Oobabooga, Jan, and others) with ChatterUI. Could someone provide a detailed step-by-step guide that works for different LLM webui alternatives with the possibility of generating an API server?

The biggest issue I’m facing is configuring the personality and flow of the questions and chatbot-user interaction. Since there is a script of questions to be presented in a specific order, both Jan and ChatterUI have an "Instructions" tab that I try to use to configure this flow of conversation and interaction between the chatbot and the user, but I’m not having success. I don’t know which coding format I should use to make this work. Essentially, the chatbot starts by introducing itself, then presents an activity involving a problem generated by a prompt. After that, there are a series of ten questions, but these questions should not all appear at once because they are part of a guided questionnaire. The questions must be presented one at a time by the chatbot, and the user responds one by one. After each response, the chatbot provides feedback considering the context of previous answers, the conversation, and the problem presented at the start. I tried using the coding <assistant_start> and <assistant_end>, but it didn’t work correctly in Jan or ChatterUI. The interaction doesn’t happen as I expected, there’s a delay in responses, and sometimes it freezes the application.

<im_start|>assistant
Hello! My name is Carlos. I am your virtual assistant robot to discuss sibling problems.
<im_end|>
<im_start|>assistant
Here is a problem. Read it carefully as a team so that you can understand it properly:
<im_end|>
<im_start|>assistant
Prompt: Create a common sibling problem. The problem should: Be appropriate for students aged 15-22; Not present an obvious solution; Not include questions for students to practice their own questioning; Be set in a family context.
<im_end|>
<im_start|>assistant
Here is a problem for your analysis:
[Include the generated problem from the given prompt].
<im_end|>
<im_start|>assistant
To help you think about solutions to the problem, reflect as a team on the following questions:
<im_end|>
<im_start|>assistant
Considering concrete examples of sibling problems, what questions would you formulate to gather data not presented here that is essential to understand the context of the information?
<im_end|>
<im_start|>assistant
What strategies can you use to determine the best solution?
<im_end|>
<im_start|>user
[Student's response]
<im_end|>
<im_start|>assistant
[Chatbot's feedback]
<im_end|>

...... From here on, there is another series of questions where there are turns between the chatbot asking the question, the user providing the answer, and the chatbot's feedback considering the response, the question, and the generated problem text. Each new interaction will have a new problem generated by the prompt.

Could someone guide me on the correct way to code and program this interaction? What coding should I use and how should I do it?

Although the developers of Jan have several proposed updates, Oobabooga (LLM webui) has all the configurations I’m interested in working with for the development of teaching activities in a future update of my educational product, such as voice cloning and training the chatbot from texts.

[Vali-98]

Hey there Alberto, thanks for the interest in this project.

I've read through the requirements of your project, and I'm afraid ChatterUI is not a good starting point. There are many issues which I can forsee:

My proposal is a chatbot that generates a problem from a prompt and, through a series of guided questions, leads the student to reflect on the presented issue.

I am not certain that LLMs (especially lower power local ones) can achieve this goal consistently. How much research have you done in this part of the project? As far as I know, multi-lingual performance of open-weight LLMs is rather poor.

The first difficulty seems simple: How should I configure the mobile phone so the chatbot responds with voice? I believe the voice is from the phone's TTS, but the app did not recognize the installed voices.

I'm fairly the app you have listed should work. Perhaps you need to restart ChatterUI once you enable that TTS engine.

Another difficulty I’m facing is how to configure remote access via server as an API using the LLM webui applications mentioned (Oobabooga, Jan, and others) with ChatterUI. Could someone provide a detailed step-by-step guide that works for different LLM webui alternatives with the possibility of generating an API server?

I wouldn't exactly know since every project implements their API differently. In general however, most APIs should be compatible with the standard OpenAI-style Chat Completions. You could refer to the API documentation of the software you are using to do so.

Jan and ChatterUI have an "Instructions" tab that I try to use to configure this flow of conversation and interaction between the chatbot and the user, but I’m not having success.

Just as a tip, Instruct formatting is more about compliance with a LLM's prompt format.

I don’t know which coding format I should use to make this work. Essentially, the chatbot starts by introducing itself, then presents an activity involving a problem generated by a prompt. After that, there are a series of ten questions, but these questions should not all appear at once because they are part of a guided questionnaire.

This would require some custom functionality:

1. We need some method of scripting questions.
2. Adding questions after a feedback of a student's response is sent.

This isn't impossible to do, but It will take a lot of effort to rework the app to create this functionality.

[brunoalbertopeixoto]

@Vali-98

I have until March to apply this product to a class, and I'm concerned. If you have a practical solution or a path I should follow, I would appreciate your advice.

[Vali-98]

I will be blunt and say that I am skeptical of the entire project premise. I have no idea if LLMs can even fit the usecase you need it to.

Secondly, though I feel that the functionality you have proposed is possible to be added within the alotted timeframe, it is likely unfeasible without some level of programming knowledge. ChatterUI seems simple on the surface, but it gets exceedingly complex underneath.

[brunoalbertopeixoto]

@Vali-98
Regarding the interaction flow, I'm trying to make it work, but it still doesn't work as I wanted. The AI ​​asks questions beyond the ones I put in the flow. This makes it very complex for high school students. Also, since I'm using LLMs on the phone, the flow is still slow. I managed to make the TTS voice work, but I was looking for a realistic Open Source TTS voice that would work offline.

Serei franco e direi que sou cético em relação a toda a premissa do projeto. Não tenho ideia se os LLMs podem se encaixar no caso de uso que você precisa.

Em segundo lugar, embora eu ache que a funcionalidade que você propôs pode ser adicionada dentro do prazo estipulado, provavelmente é inviável sem algum nível de conhecimento de programação. O ChatterUI parece simples na superfície, mas fica extremamente complexo por baixo.

[Vali-98]

You may just need to configure the AI to respond with feedback only. Some instruct tuned models will comply if you ask them to only give feedback and never ask questions.