A Comparative Study of Demonstration Selection for Practical Large Language Models-based Next POI Prediction

Researchers conducted a comparative analysis of demonstration selection strategies for using large language models to predict users' next point-of-interest (POI) based on historical location data. The study found that simple heuristic methods like geographical proximity and temporal ordering outperform complex embedding-based approaches in both computational efficiency and prediction accuracy, with LLMs using these heuristics sometimes matching fine-tuned model performance without additional training.

This research addresses a practical challenge in applying large language models to location prediction tasks, where the selection of in-context learning examples significantly influences model performance. The authors' systematic comparison reveals a counterintuitive finding: sophisticated embedding-based and task-specific selection methods, which require substantial computational resources, are consistently outperformed by straightforward heuristic approaches grounded in geographical and temporal proximity.

The work builds on growing interest in in-context learning as an alternative to traditional supervised fine-tuning, which demands labeled training data and computational overhead. Prior research had explored various selection strategies but lacked comprehensive comparative analysis, leaving practitioners uncertain about optimal approaches for real-world deployment. This study fills that gap by benchmarking methods across three real-world datasets, establishing clear empirical guidance.

The practical implications are significant for application developers and researchers. Organizations implementing POI prediction systems can reduce computational costs and infrastructure requirements while maintaining or improving accuracy by adopting simple heuristic selection methods. The finding that LLMs with these demonstrations can match fine-tuned models without additional training democratizes access to high-performance location prediction, particularly valuable for companies with limited training resources.

Future investigations should explore whether these heuristic findings generalize across other LLM applications beyond POI prediction, examine hybrid approaches combining heuristics with embedding methods, and investigate why simpler methods outperform complex alternatives—potentially revealing fundamental insights about in-context learning effectiveness.

• →Simple geographical and temporal heuristics outperform complex embedding-based demonstration selection methods for POI prediction
• →LLMs using heuristic-selected demonstrations achieve performance comparable to fine-tuned models without requiring additional training
• →Heuristic approaches reduce computational costs while improving prediction accuracy and practical applicability
• →Current embedding-based methods add computational complexity without corresponding performance benefits in location prediction tasks
• →Findings suggest practitioners should prioritize implementation simplicity when applying LLMs to sequence prediction problems