The Role of Textual Context in LLM-Based Time Series Forecasting - A Study of Prompt Content and Backbone Scale

Abstract

Large language models (LLMs) are increasingly used as frozen backbones in time series forecasting, yet it remains unclear whether these models can benefit from the textual prompts that accompany the numerical input, and if so, what kind of text is needed and at what scale the benefit emerges. Prior work found that replacing prompts with random text has no effect, but tested only static prompts at small backbone scales. This thesis systematically investigates these questions through a controlled ablation study using a Time-LLM-inspired architecture on hourly bike rental demand. Four prompt configurations (no prompt, static domain description, sample-wise calendar context, and shuffled calendar context) are evaluated at two backbone scales (Qwen2.5-0.5B and Qwen2.5-7B), yielding eight experiments with all other variables held constant. The findings show that the usefulness of the prompt depends on both the type of information provided and the backbone’s scale. Generic dataset-level descriptions offer little value, whereas sample-specific context can improve forecasting by helping the model capture irregular patterns. This benefit is more evident for larger backbones and appears to transfer beyond the main experimental setting. Overall, the results suggest that frozen LLMs can benefit from text in forecasting, but only when that information is relevant to each sample and the backbone is large enough to act on it.