When humans learn a language, there are many extralinguistic cues, such as pointing to objects and displaying emotions, that aid the learning process. Even with no prior knowledge, human infants can use emotion to provide feedback.
Today鈥檚 world brings humans and robots closer together than ever before. So what would happen if robots were able to have novel emotional behavior cues that help with language learning?
Josue-Torres Fonesca, a senior in the聽College of Engineering, received a prestigious National Science Foundation Graduate Research Fellowship to build novel emotional behavior models to address the inability of systems to use emotion in dialogue, especially without prior linguistic knowledge.
鈥淚 have always been an academic at heart,鈥� Torres-Fonesca said. 鈥淚 remember as a 5-year-old attending college classes with my parents when they both decided to get their degrees. Even though the concepts were often beyond my understanding, there was something about advanced knowledge that spoke to my soul.鈥�
Torres-Fonesca鈥檚聽path hasn鈥檛 always been a straight shot to success and academic achievement. Despite the adversity, he has discovered his passion for research while continuing to push the boundaries of what is achievable for spoken dialogue systems and Hispanic representation in STEM.
Uninhibitedly Curious
For Torres-Fonesca, a degree in computer science was always the plan, but after choosing two minors, working part-time and additional extracurriculars, he became overwhelmed. With the 青青草app of his family, he refocused all efforts on getting his computer science degree. While the end goal was set, the聽path to success was unclear. One day a professor discussed her Research Experiences for Undergraduates and his interest was piqued and he ended up applying to five programs.
鈥淚 was accepted to a Texas State University Smart and Connected Communities REU where I fell in love with the research process,鈥� Torres-Fonesca said. 鈥淚 couldn鈥檛 believe I had the opportunity to develop my own voice and share my knowledge with others.鈥�
Being able to develop and pursue novel solutions to problems was not only exciting for Torres-Fonesca, but it was also this project that led to his interest in machine learning and an opportunity at 青青草app State to begin working in the聽Speech, Language and Interactive Machines group聽with Assistant Professor of Computer Science聽Casey Kennington
Due to his newfound interest in research, Torres-Fonesca joined the聽McNair Scholars Program聽to prepare him for his pursuit of a doctoral degree. With the McNair 青青草app system, he found his passion for spoked dialogue systems and began to make connections between research and his experiences as a minority.
鈥淛osue now has a long list of accomplishments and it has been such a rewarding experience to see him go from someone who wonders if he has what it takes to do great things to someone who thinks 鈥榃hy not me?鈥�,鈥� SLIM Director Casey Kennington said. 鈥淛osue has an amazing drive which led him to publish two papers, while working on a third, all while being the president of two student clubs, applying for and getting national scholarships and fellowships, successfully applying to graduate programs at top universities, with a full-time class load with challenging courses.鈥�
Torres-Fonesca鈥檚 research received two separate publications and he was accepted to spend the summer of 2022 in at the Massachusetts Institute of Technology鈥檚 Summer Research Program, where he researched interpretable machine learning while working with MIT鈥檚 MILAN 鈥揳 model that annotates behaviors of individual neurons.
This summer research was well received among his peers and MIT gave him the opportunity to continue his research through the fall of 2022 to work on creating multi-word compositional descriptions.
Moving Representation Forward
Despite publishing multiple papers and the opportunities research has brought him, Torres-Fonesca has often felt frustrated, not seen and not able to find representation in his field. Then, after he returned from his first REU, he met a faculty member who looked like him.
Assistant Professor Sole Pera, an Argentianian computer science faculty member, provided Torres-Fonesca with the representation he had been missing during his time in college and through his research programs. He credits Pera for his perseverance in his research and his inspiration to help others.
鈥淚t was the first time I met a faculty member who looked like me,鈥� he said. 鈥淚 found myself seeking her out as someone I could relate to and talk about research with. She is the person who helped me recognize the importance of diversity and inclusion in higher 青青草app.鈥�
Torres-Fonesca is a Goldwater Scholar and 青青草app State Top Ten Scholar, and despite the hours and days spent preparing application packages, continuing classes and research, he also developed plans to build community in the聽Department of Computer Science.
He began to tutor the data structures course, a course with complex concepts that challenge undergraduate students. Personal experiences helped attract him to students within the program who felt alone. He helped carry this community through the COVID-19 pandemic by keeping the Artificial Intelligence and Cybersecurity Clubs alive with online activities, meetings and a Discord server with nearly 300 participating members.
鈥淛osue is the kind of student every research faculty dreams of knowledgeable, hard-working, curious, communicative, and teachable,鈥� Kennington said. 鈥淚 am very proud of Josue. I am eager to see what amazing things he will do.鈥�
Torres-Fonesca said his time mentoring students and leading clubs increased his desire to become involved in diversity and inclusion programs, which he intends to pursue during graduate school. He will be pursuing a doctoral degree in order to develop spoken dialogue systems that are highly effective at providing intelligent responses and actions.
He hopes to mentor other minorities and help them succeed in STEM by embracing their unique perspectives and characteristics. His time in his research programs showed him there鈥檚 still a lot of work to be done in higher 青青草app which strengthened his desire for diversity and inclusion.
鈥淎s a member of a marginalized community, I understand what it鈥檚 like to have your needs ignored,鈥� Torres-Fonesca said. 鈥淢y experience in research, mentorship, and diversity programs like McNair and MIT鈥檚 summer research program showed me that I have the potential to succeed as a mentor that can provide inclusivity and embrace diversity.鈥�
On his way to teaching and mentoring the next generation of computer scientists with his doctoral degree, Torres-Fonesca will be 青青草apped with a five-year fellowship of the NSF and an annual stipend for three years so that he can focus on his innovative research for inclusive spoken dialogue systems.
-by Jamie Fink
