Keywords

recommender systems, pretrained language models, intent detection

Abstract

Pretrained language models have started a new era in artificial intelligence in general and natural language processing (NLP) in particular. With the advent of transformer architecture, pretrained language models are at the forefront of driving advancements in machine intelligence. Scaled dot product attention sits at the core of transformer architecture. In scaled dot product attention, each token gets a chance to attend to every other token in the context. This attention process yields the affinities or correlations that a token exhibits with other tokens, in the context. The correlation vector is normalized into a distribution, which is employed to gauge the effect of attended tokens on the current token. By doing so, we get a context-aware embedding for the current token. Transformer architecture revolutionized the design of modern pretrained language models. The progress is not limited to tasks related to NLP; another notable area that is benefiting from the advancements brought about by the transformer architecture is the design of modern sequential recommender systems. The generalized pretrained transformer (GPT) is a transformer-inspired neural architecture that popularized the idea of autoregressive design for language modeling. In the autoregressive design strategy, the goal is to predict the next token in the sequence based on the sequence history. In this dissertation, we show that pretrained language models can be employed for designing a zero-shot intent detector. Secondly, by realizing the gap and unavailability of a large-scale dataset to encourage research in mobile application recommender systems, we curate a large-scale dataset, MobileRec, and release it for the recommender systems community. Third, we propose several strategies to incorporate the user's interaction rhythm into a sequential mobile application recommender system by employing the autoregressive design paradigm from pretrained language models. We also incorporate our proposed positional morphing neural architecture into sequential recommender systems to enhance their learning capabilities.

Completion Date

2024

Semester

Spring

Committee Chair

Foroosh, Hassan

Degree

Doctor of Philosophy (Ph.D.)

College

College of Engineering and Computer Science

Department

Department of Computer Science

Degree Program

Computer Science

Format

application/pdf

Language

English

Rights

In copyright

Release Date

November 2024

Length of Campus-only Access

None

Access Status

Doctoral Dissertation (Open Access)

Campus Location

Orlando (Main) Campus

Accessibility Status

Meets minimum standards for ETDs/HUTs

Download

Share

COinS