[Sugar-devel] Proposal for Adding AI chatbot in Chat Activity (GSoC 2024)

[Debasis Sikdar]

Okay sure I'll add in the changes.

Thank you again,
Debasis


On Sat, Mar 23, 2024, 18:25 Chihurumnaya Ibiam <ibiam at sugarlabs.org> wrote:

> The info you've shared above would be good info to add to your proposal,
> I'll suggest you look at the code
> in speech manager and the changes you'll need to make to it and then also
> clearly write about that too in your
> proposal.
>
> Hope this helps!
>
> --
>
> Ibiam Chihurumnaya
> ibiam at sugarlabs.org
>
>
>
> On Fri, Mar 22, 2024 at 6:31 PM Debasis Sikdar <
> debasis.sikder123 at gmail.com> wrote:
>
>> I am sorry that I misunderstood you. So, I was talking about the task 2
>> of GSOC,
>>
>> I'd like to implement a chatbot for the Chat Activity. Using a free LLM
>> service like Gemini Ai, where users can have a conversation with the bot
>> using text or voice and get a text or voice reply (For children with
>> disability).
>>
>> For this to work,
>>
>> The speech.py (Speech Manager) needs some improvement.
>>
>> It has TTS functionality but, I also need voice to text functionality.
>>
>> So, I will add that in the speech.py using libraries like PocketSphinx or
>> SpeechRecognition.
>>
>> The input from user will be added with some prompts like (Generate the
>> text which is safe for a child, and with a readability of Coleman Lieu
>> index less than 5) this is what readability checker you were referring to
>> was, it is a way we can generate easy to read and appropriate output from
>> an AI chatbot.
>>
>> I'll also save the chat in the sugar-database so that we keep a record in
>> their local system, so it does not violate any privacy policy.
>>
>> This functionality will be added to the chat bot that I’ll be happy to
>> integrate in the chat activity.
>>
>> As for the UI, it will be based on the current sugar's chat activity UI.
>>
>> Refer to the image below hope you like it. If you have further issues,
>> kindly let me know I hope I was able to understand you correctly this time.
>>
>> Thank you,
>>
>> Debasis
>>
>> On Thu, Mar 21, 2024, 22:18 Chihurumnaya Ibiam <ibiam at sugarlabs.org>
>> wrote:
>>
>>> I'd suggested that you look into using the speech manager if it'll need
>>> to be modified for your needs.
>>>
>>> For the UI, I didn't say anything about Buttons or Windows, I'd asked
>>> about you sharing designs if you had any.
>>>
>>> If I understand correctly, you're saying you'll build a readability
>>> checker to add to the activty.
>>>
>>> I don't really understand the image you've attached, when replying to
>>> threads, use reply-all so the convo would stay in the thread.
>>>
>>> --
>>>
>>> Ibiam Chihurumnaya
>>> ibiam at sugarlabs.org
>>>
>>>
>>>
>>> On Wed, Mar 20, 2024 at 4:13 PM Debasis Sikdar <
>>> debasis.sikder123 at gmail.com> wrote:
>>>
>>>> Hello Sir,
>>>>
>>>> I am Debasis Sikdar, I wrote a review for you yesterday and after your
>>>> reply I was highly motivated.
>>>>
>>>> I read your mail and understood your requirements. I made the changes
>>>> and implemented the changes in my rough sketch I have attached below for
>>>> the same.
>>>>
>>>> In brief,
>>>> I changed the database from SQLite to sugar-database (DS) and
>>>> understood it's working.
>>>> I learnt about the UI using PyGTK and mentioned the creation of Buttons
>>>> and Windows as required.
>>>> I also added a readability checker using Coleman–Liau index for the
>>>> readability of text for children. (0-8 range).
>>>> I changed my implementation of voice support from pyttsx3 to sugar's
>>>> speech.py library.
>>>>
>>>> Thank you so much for your time and consideration,
>>>> Debasis
>>>>
>>>> On Wed, Mar 20, 2024, 18:09 Chihurumnaya Ibiam <ibiam at sugarlabs.org>
>>>> wrote:
>>>>
>>>>> You've not stated if you a proposal review, but I'm guessing that's
>>>>> why you shared it and I have some comments;
>>>>>
>>>>> You've said you'll add support for voice chat using speech recognition
>>>>> - this isn't a tool - and pyttsx3, we have
>>>>> Speech Manager
>>>>> <https://github.com/sugarlabs/sugar-toolkit-gtk3/blob/master/src/sugar3/speech.py>
>>>>> in the sugar-toolkit-gtk3, maybe look at how you can leverage that and
>>>>> improve it if necessary.
>>>>>
>>>>> You've mentioned using SQLite for a local database, Sugar uses
>>>>> sugar-datastore
>>>>> <https://github.com/sugarlabs/sugar-toolkit-gtk3/blob/master/src/sugar3/datastore/datastore.py>
>>>>> to store any data an activity needs
>>>>> to store, you should look at using it.
>>>>>
>>>>> You've also said you'll be designing a UI, but haven't shared any
>>>>> ideas you have and it'll be great if you shared any.
>>>>>
>>>>> You've mentioned abstracting away adding extra prompts to make answers
>>>>> simpler for kids, but you've not mentioned
>>>>> anything about how you intend to achieve this abstraction.
>>>>>
>>>>> Hope this helps!
>>>>>
>>>>> --
>>>>>
>>>>> Ibiam Chihurumnaya
>>>>> ibiam at sugarlabs.org
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 19, 2024 at 7:06 PM Debasis Sikdar <
>>>>> debasis.sikder123 at gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> Dear Team,
>>>>>>
>>>>>> I am Debasis Sikdar, a sophomore in KIIT University, Odisha. I am
>>>>>> pursuing a B.Tech. in Computer Science and Engineering. I have been
>>>>>> targeting GSoC for a year now and I am willing to learn more and contribute
>>>>>> to Open-Source.
>>>>>>
>>>>>> Topic: Adding an AI chatbot to the Topic: Adding an AI chatbot to the
>>>>>> Chat Activity
>>>>>>
>>>>>> Tech-stack: Python, Google.GenerativeAI, UI/UX, PyGTK, SQLite, Pyttsx3
>>>>>>
>>>>>> Timeline:
>>>>>>
>>>>>> Week 1:
>>>>>>
>>>>>> Exploring Gemini API's features and getting familiar with the Sugar
>>>>>> Labs codebase.
>>>>>>
>>>>>> Week 2-3:
>>>>>>
>>>>>> I'll begin integration of the Gemini API and PyGTK for the UI, I can
>>>>>> add support for voice chat using Speech Recognition and pyttsx3.
>>>>>>
>>>>>> Week 4-5:
>>>>>>
>>>>>> Chatbot Training & Refinement so that in any case it does not provide
>>>>>> any harmful or bad response, I'll use safety prompts and filter the output
>>>>>> i.e., text or sound generated.
>>>>>> I'll save the chats in a local database (not shared due to privacy
>>>>>> concerns) using SQLite for the user to access any chat they previously had.
>>>>>> After 1st Evaluation.
>>>>>>
>>>>>> Week 6-7:
>>>>>>
>>>>>> Testing on Different I/O and making sure that answers are made
>>>>>> simpler for children, I will add extra prompts for this purpose which will
>>>>>> be abstracted away.
>>>>>>
>>>>>> Week 8-9:
>>>>>>
>>>>>> The project is finally coming to an end. After integrating, training
>>>>>> and testing we are left with handling of edge cases and UI.
>>>>>> In this week I'll also be designing the UI with PyGTK.
>>>>>>
>>>>>> Week 9-10:
>>>>>>
>>>>>> Testing the whole project after deploying and fixing any errors
>>>>>> generated in the process.
>>>>>> The project can now be merged with the Source.
>>>>>> This is my plan for the project listed, there can be minor changes to
>>>>>> the plan if required.
>>>>>>
>>>>>> I am grateful to Sugar Labs and GSoC for offering this opportunity
>>>>>> and I am looking forward to a favourable response.
>>>>>>
>>>>>> Thank You,
>>>>>> _______________________________________________
>>>>>> Sugar-devel mailing list
>>>>>> Sugar-devel at lists.sugarlabs.org
>>>>>> http://lists.sugarlabs.org/listinfo/sugar-devel
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20240324/5cd566dd/attachment-0001.htm>