Projects

Thursday December 5^th, 2024 2:00PM Concurrent Session II

Imagine if CUNY became a hub of innovation, developing tools designed by CUNY students for its community and industry. YrkGPT is being created by York students in collaboration with York faculty and IT, an AI-powered tool similar to ChatGPT. YrkGPT, a Large Language Model, trained on data from around 3,000 endpoints from the publicly accessible York College website, features 7.6 billion parameters, allowing it to accurately generate responses related to York College. This inference ability is bolstered by Retrieval Augmented Generation, giving it access to a constant stream of current and relevant information.

YrkGPT will provide one stop solution to anyone seeking information on York College. YrkGPT will answer questions related to students’ admissions, financial aid, navigating the campus and much more. Faculty and staff will be able to find answers to perform their daily tasks, such as contact information, event location etc.

Once successfully launched, YrkGPT has the potential to be replicated across CUNY campuses and administrative units. The CUNY IT Conference provides a perfect platform to showcase YrkGPT’s innovative approach to advancing education and operational efficiency, setting the stage for broader adoption across the university system.

Abu Kamruzzaman, Assistant Professor, Department of Business and Economics, School of Business and Information Systems

Elizabeth Quaye, Assistant Professor, Department of Accounting and Finance

Gregory Vega, Interim Assistant Vice President/Chief Information Officer, IT

Eric Tyrer, Associate Director, IT, Web Services and Business Intelligence Department

Darryl Nurse, Student, Math & Computer Science Department

All of York College

This presentation, titled "YorkGPT: A Comprehensive Overview," details the development of YorkGPT, an innovative AI project at York College. Greg initiated the presentation by highlighting the collaborative nature of the project, specifically mentioning the partnership between the IT department and Professor Abu's class, which provided mentorship and fresh perspectives.

Professor Abu then elaborated on the vision for CUNY innovation, emphasizing YorkGPT as an AI-powered tool similar to ChatGPT, developed by York students, faculty, and IT. He explained that this Large Language Model, featuring 7.6 billion parameters and trained on data from approximately 3,000 York College website endpoints, utilizes Retrieval Augmented Generation (RAG) for enhanced and current information retrieval.

Eric discussed the critical need for computational power, leading to the acquisition of a computer with two NVIDIA H100 GPUs. He further detailed the challenges of navigating infrastructure transitions, including initial preparations for OpenStack, network conflicts, the eventual VMware contract renewal, and the subsequent relocation of the GPU server to the VMware cluster, which presented new licensing and configuration hurdles. Ultimately, a new VMware cluster was established, equipped with the two high-memory GPUs, allowing for flexible partitioning.

Hector then covered the significant hardware and software configuration challenges encountered, emphasizing the extensive learning, testing, development, and configuration involved. He outlined the key software components utilized, including N8N, Crawl4AI, Plone REST API, PostgreSQL, and WebUI, and explained that YorkGPT comprises both public and private data models, with a specific focus on creating a private GPT to safeguard sensitive information.

Rafael concluded by emphasizing the commitment to data security, highlighting the implementation of controlled access mechanisms to protect confidential student and institutional information. He summarized the collaborative and technically demanding yet rewarding development process, underscoring the creation of a robust AI system that meets institutional needs while prioritizing data security. The presentation concluded with a look towards future prospects and potential advancements for YorkGPT, followed by a Q&A session.