[Sugar-devel] Proposal for Adding AI chatbot in Chat Activity (GSoC 2024)

[Chihurumnaya Ibiam]

The info you've shared above would be good info to add to your proposal,
I'll suggest you look at the code
in speech manager and the changes you'll need to make to it and then also
clearly write about that too in your
proposal.

Hope this helps!

-- 

Ibiam Chihurumnaya
ibiam at sugarlabs.org



On Fri, Mar 22, 2024 at 6:31 PM Debasis Sikdar <debasis.sikder123 at gmail.com>
wrote:

> I am sorry that I misunderstood you. So, I was talking about the task 2
> of GSOC,
>
> I'd like to implement a chatbot for the Chat Activity. Using a free LLM
> service like Gemini Ai, where users can have a conversation with the bot
> using text or voice and get a text or voice reply (For children with
> disability).
>
> For this to work,
>
> The speech.py (Speech Manager) needs some improvement.
>
> It has TTS functionality but, I also need voice to text functionality.
>
> So, I will add that in the speech.py using libraries like PocketSphinx or
> SpeechRecognition.
>
> The input from user will be added with some prompts like (Generate the
> text which is safe for a child, and with a readability of Coleman Lieu
> index less than 5) this is what readability checker you were referring to
> was, it is a way we can generate easy to read and appropriate output from
> an AI chatbot.
>
> I'll also save the chat in the sugar-database so that we keep a record in
> their local system, so it does not violate any privacy policy.
>
> This functionality will be added to the chat bot that I’ll be happy to
> integrate in the chat activity.
>
> As for the UI, it will be based on the current sugar's chat activity UI.
>
> Refer to the image below hope you like it. If you have further issues,
> kindly let me know I hope I was able to understand you correctly this time.
>
> Thank you,
>
> Debasis
>
> On Thu, Mar 21, 2024, 22:18 Chihurumnaya Ibiam <ibiam at sugarlabs.org>
> wrote:
>
>> I'd suggested that you look into using the speech manager if it'll need
>> to be modified for your needs.
>>
>> For the UI, I didn't say anything about Buttons or Windows, I'd asked
>> about you sharing designs if you had any.
>>
>> If I understand correctly, you're saying you'll build a readability
>> checker to add to the activty.
>>
>> I don't really understand the image you've attached, when replying to
>> threads, use reply-all so the convo would stay in the thread.
>>
>> --
>>
>> Ibiam Chihurumnaya
>> ibiam at sugarlabs.org
>>
>>
>>
>> On Wed, Mar 20, 2024 at 4:13 PM Debasis Sikdar <
>> debasis.sikder123 at gmail.com> wrote:
>>
>>> Hello Sir,
>>>
>>> I am Debasis Sikdar, I wrote a review for you yesterday and after your
>>> reply I was highly motivated.
>>>
>>> I read your mail and understood your requirements. I made the changes
>>> and implemented the changes in my rough sketch I have attached below for
>>> the same.
>>>
>>> In brief,
>>> I changed the database from SQLite to sugar-database (DS) and understood
>>> it's working.
>>> I learnt about the UI using PyGTK and mentioned the creation of Buttons
>>> and Windows as required.
>>> I also added a readability checker using Coleman–Liau index for the
>>> readability of text for children. (0-8 range).
>>> I changed my implementation of voice support from pyttsx3 to sugar's
>>> speech.py library.
>>>
>>> Thank you so much for your time and consideration,
>>> Debasis
>>>
>>> On Wed, Mar 20, 2024, 18:09 Chihurumnaya Ibiam <ibiam at sugarlabs.org>
>>> wrote:
>>>
>>>> You've not stated if you a proposal review, but I'm guessing that's why
>>>> you shared it and I have some comments;
>>>>
>>>> You've said you'll add support for voice chat using speech recognition
>>>> - this isn't a tool - and pyttsx3, we have
>>>> Speech Manager
>>>> <https://github.com/sugarlabs/sugar-toolkit-gtk3/blob/master/src/sugar3/speech.py>
>>>> in the sugar-toolkit-gtk3, maybe look at how you can leverage that and
>>>> improve it if necessary.
>>>>
>>>> You've mentioned using SQLite for a local database, Sugar uses
>>>> sugar-datastore
>>>> <https://github.com/sugarlabs/sugar-toolkit-gtk3/blob/master/src/sugar3/datastore/datastore.py>
>>>> to store any data an activity needs
>>>> to store, you should look at using it.
>>>>
>>>> You've also said you'll be designing a UI, but haven't shared any ideas
>>>> you have and it'll be great if you shared any.
>>>>
>>>> You've mentioned abstracting away adding extra prompts to make answers
>>>> simpler for kids, but you've not mentioned
>>>> anything about how you intend to achieve this abstraction.
>>>>
>>>> Hope this helps!
>>>>
>>>> --
>>>>
>>>> Ibiam Chihurumnaya
>>>> ibiam at sugarlabs.org
>>>>
>>>>
>>>>
>>>> On Tue, Mar 19, 2024 at 7:06 PM Debasis Sikdar <
>>>> debasis.sikder123 at gmail.com> wrote:
>>>>
>>>>>
>>>>> Dear Team,
>>>>>
>>>>> I am Debasis Sikdar, a sophomore in KIIT University, Odisha. I am
>>>>> pursuing a B.Tech. in Computer Science and Engineering. I have been
>>>>> targeting GSoC for a year now and I am willing to learn more and contribute
>>>>> to Open-Source.
>>>>>
>>>>> Topic: Adding an AI chatbot to the Topic: Adding an AI chatbot to the
>>>>> Chat Activity
>>>>>
>>>>> Tech-stack: Python, Google.GenerativeAI, UI/UX, PyGTK, SQLite, Pyttsx3
>>>>>
>>>>> Timeline:
>>>>>
>>>>> Week 1:
>>>>>
>>>>> Exploring Gemini API's features and getting familiar with the Sugar
>>>>> Labs codebase.
>>>>>
>>>>> Week 2-3:
>>>>>
>>>>> I'll begin integration of the Gemini API and PyGTK for the UI, I can
>>>>> add support for voice chat using Speech Recognition and pyttsx3.
>>>>>
>>>>> Week 4-5:
>>>>>
>>>>> Chatbot Training & Refinement so that in any case it does not provide
>>>>> any harmful or bad response, I'll use safety prompts and filter the output
>>>>> i.e., text or sound generated.
>>>>> I'll save the chats in a local database (not shared due to privacy
>>>>> concerns) using SQLite for the user to access any chat they previously had.
>>>>> After 1st Evaluation.
>>>>>
>>>>> Week 6-7:
>>>>>
>>>>> Testing on Different I/O and making sure that answers are made simpler
>>>>> for children, I will add extra prompts for this purpose which will be
>>>>> abstracted away.
>>>>>
>>>>> Week 8-9:
>>>>>
>>>>> The project is finally coming to an end. After integrating, training
>>>>> and testing we are left with handling of edge cases and UI.
>>>>> In this week I'll also be designing the UI with PyGTK.
>>>>>
>>>>> Week 9-10:
>>>>>
>>>>> Testing the whole project after deploying and fixing any errors
>>>>> generated in the process.
>>>>> The project can now be merged with the Source.
>>>>> This is my plan for the project listed, there can be minor changes to
>>>>> the plan if required.
>>>>>
>>>>> I am grateful to Sugar Labs and GSoC for offering this opportunity and
>>>>> I am looking forward to a favourable response.
>>>>>
>>>>> Thank You,
>>>>> _______________________________________________
>>>>> Sugar-devel mailing list
>>>>> Sugar-devel at lists.sugarlabs.org
>>>>> http://lists.sugarlabs.org/listinfo/sugar-devel
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20240323/62458ced/attachment-0001.htm>