34. Chronos forecasting#
The paper behind this approach is here: https://arxiv.org/pdf/2403.07815
The paper introduces Chronos, a framework that adapts language model architectures for time series forecasting. The key aspects are:
Approach:
Converts time series data into tokens through scaling and quantization
Uses existing transformer language model architectures with minimal modifications
Trains models on both real and synthetic time series data
Key Components:
TSMixup: Data augmentation technique that combines patterns from different time series
KernelSynth: Synthetic data generation using Gaussian processes
Based on T5 transformer architecture, with models ranging from 20M to 710M parameters
Main Results:
Significantly outperforms traditional statistical models and task-specific deep learning models on in-domain datasets
Achieves competitive zero-shot performance on unseen datasets compared to models specifically trained on those datasets
Fine-tuning further improves performance on new datasets
Key Benefits:
Eliminates need for task-specific training
Simplifies forecasting pipelines
Shows language model architectures can handle time series without specialized modifications
Provides strong zero-shot forecasting capabilities
Limitations:
Inference speed slower than some specialized models
Edge cases with very sparse or high-variance time series
Limited by available high-quality public time series data
The paper demonstrates that language model architectures can be effectively adapted for time series forecasting with minimal modifications, potentially simplifying real-world forecasting applications.
See Chronos: amazon-science/chronos-forecasting
Docs: https://auto.gluon.ai/stable/tutorials/timeseries/forecasting-chronos.html
!pip install chronos-forecasting
Requirement already satisfied: chronos-forecasting in /usr/local/lib/python3.11/dist-packages (1.5.0)
Requirement already satisfied: accelerate<1,>=0.32 in /usr/local/lib/python3.11/dist-packages (from chronos-forecasting) (0.34.2)
Requirement already satisfied: torch<3,>=2.0 in /usr/local/lib/python3.11/dist-packages (from chronos-forecasting) (2.6.0+cu124)
Requirement already satisfied: transformers<4.48,>=4.30 in /usr/local/lib/python3.11/dist-packages (from chronos-forecasting) (4.47.1)
Requirement already satisfied: numpy<3.0.0,>=1.17 in /usr/local/lib/python3.11/dist-packages (from accelerate<1,>=0.32->chronos-forecasting) (2.0.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.11/dist-packages (from accelerate<1,>=0.32->chronos-forecasting) (24.2)
Requirement already satisfied: psutil in /usr/local/lib/python3.11/dist-packages (from accelerate<1,>=0.32->chronos-forecasting) (5.9.5)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from accelerate<1,>=0.32->chronos-forecasting) (6.0.2)
Requirement already satisfied: huggingface-hub>=0.21.0 in /usr/local/lib/python3.11/dist-packages (from accelerate<1,>=0.32->chronos-forecasting) (0.29.3)
Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.11/dist-packages (from accelerate<1,>=0.32->chronos-forecasting) (0.5.3)
Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.18.0)
Requirement already satisfied: typing-extensions>=4.10.0 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (4.12.2)
Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.4.2)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.1.6)
Requirement already satisfied: fsspec in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (2025.3.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.4.127)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.4.127)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.4.127)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.4.5.8)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (11.2.1.3)
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (10.3.5.147)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (11.6.1.9)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.3.1.170)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.2 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (0.6.2)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.4.127)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.4.127)
Requirement already satisfied: triton==3.2.0 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.2.0)
Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->chronos-forecasting) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy==1.13.1->torch<3,>=2.0->chronos-forecasting) (1.3.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.11/dist-packages (from transformers<4.48,>=4.30->chronos-forecasting) (2024.11.6)
Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from transformers<4.48,>=4.30->chronos-forecasting) (2.32.3)
Requirement already satisfied: tokenizers<0.22,>=0.21 in /usr/local/lib/python3.11/dist-packages (from transformers<4.48,>=4.30->chronos-forecasting) (0.21.1)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.11/dist-packages (from transformers<4.48,>=4.30->chronos-forecasting) (4.67.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch<3,>=2.0->chronos-forecasting) (3.0.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->transformers<4.48,>=4.30->chronos-forecasting) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->transformers<4.48,>=4.30->chronos-forecasting) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->transformers<4.48,>=4.30->chronos-forecasting) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->transformers<4.48,>=4.30->chronos-forecasting) (2025.1.31)
34.1. Forecasting Passenger Traffic#
import pandas as pd # requires: pip install pandas
import torch
from chronos import BaseChronosPipeline
import matplotlib.pyplot as plt # requires: pip install matplotlib
import numpy as np
pipeline = BaseChronosPipeline.from_pretrained(
"amazon/chronos-t5-small", # use "amazon/chronos-bolt-small" for the corresponding Chronos-Bolt model
device_map="cuda", # use "cpu" for CPU inference
torch_dtype=torch.bfloat16,
)
/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning:
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
warnings.warn(
34.2. Forecasting Passenger Traffic#
df = pd.read_csv(
"https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv"
)
# context must be either a 1D tensor, a list of 1D tensors,
# or a left-padded 2D tensor with batch as the first dimension
# quantiles is an fp32 tensor with shape [batch_size, prediction_length, num_quantile_levels]
# mean is an fp32 tensor with shape [batch_size, prediction_length]
quantiles, mean = pipeline.predict_quantiles(
context=torch.tensor(df["#Passengers"]),
prediction_length=12,
quantile_levels=[0.1, 0.5, 0.9],
)
df.head()
| Month | #Passengers | |
|---|---|---|
| 0 | 1949-01 | 112 |
| 1 | 1949-02 | 118 |
| 2 | 1949-03 | 132 |
| 3 | 1949-04 | 129 |
| 4 | 1949-05 | 121 |
df.tail()
| Month | #Passengers | |
|---|---|---|
| 139 | 1960-08 | 606 |
| 140 | 1960-09 | 508 |
| 141 | 1960-10 | 461 |
| 142 | 1960-11 | 390 |
| 143 | 1960-12 | 432 |
from chronos import ChronosPipeline, ChronosBoltPipeline
print(ChronosPipeline.predict.__doc__) # for Chronos models
print(ChronosBoltPipeline.predict.__doc__) # for Chronos-Bolt models
Get forecasts for the given time series.
Refer to the base method (``BaseChronosPipeline.predict``)
for details on shared parameters.
Additional parameters
---------------------
num_samples
Number of sample paths to predict. Defaults to what
specified in ``self.model.config``.
temperature
Temperature to use for generating sample tokens.
Defaults to what specified in ``self.model.config``.
top_k
Top-k parameter to use for generating sample tokens.
Defaults to what specified in ``self.model.config``.
top_p
Top-p parameter to use for generating sample tokens.
Defaults to what specified in ``self.model.config``.
limit_prediction_length
Force prediction length smaller or equal than the
built-in prediction length from the model. False by
default. When true, fail loudly if longer predictions
are requested, otherwise longer predictions are allowed.
Returns
-------
samples
Tensor of sample forecasts, of shape
(batch_size, num_samples, prediction_length).
Get forecasts for the given time series.
Refer to the base method (``BaseChronosPipeline.predict``)
for details on shared parameters.
Additional parameters
---------------------
limit_prediction_length
Force prediction length smaller or equal than the
built-in prediction length from the model. False by
default. When true, fail loudly if longer predictions
are requested, otherwise longer predictions are allowed.
Returns
-------
torch.Tensor
Forecasts of shape (batch_size, num_quantiles, prediction_length)
where num_quantiles is the number of quantiles the model has been
trained to output. For official Chronos-Bolt models, the value of
num_quantiles is 9 for [0.1, 0.2, ..., 0.9]-quantiles.
Raises
------
ValueError
When limit_prediction_length is True and the prediction_length is
greater than model's trainig prediction_length.
forecast_index = range(len(df), len(df) + 12)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]
plt.figure(figsize=(8, 4))
plt.plot(df["#Passengers"], color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
34.3. Forecasting Stocks#
# Code to use yfinance to download bitcoin data from FRED ticker CBBTCUSD
!pip install yfinance
import yfinance as yf
Requirement already satisfied: yfinance in /usr/local/lib/python3.11/dist-packages (0.2.54)
Requirement already satisfied: pandas>=1.3.0 in /usr/local/lib/python3.11/dist-packages (from yfinance) (2.2.2)
Requirement already satisfied: numpy>=1.16.5 in /usr/local/lib/python3.11/dist-packages (from yfinance) (2.0.2)
Requirement already satisfied: requests>=2.31 in /usr/local/lib/python3.11/dist-packages (from yfinance) (2.32.3)
Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.11/dist-packages (from yfinance) (0.0.11)
Requirement already satisfied: platformdirs>=2.0.0 in /usr/local/lib/python3.11/dist-packages (from yfinance) (4.3.6)
Requirement already satisfied: pytz>=2022.5 in /usr/local/lib/python3.11/dist-packages (from yfinance) (2025.1)
Requirement already satisfied: frozendict>=2.3.4 in /usr/local/lib/python3.11/dist-packages (from yfinance) (2.4.6)
Requirement already satisfied: peewee>=3.16.2 in /usr/local/lib/python3.11/dist-packages (from yfinance) (3.17.9)
Requirement already satisfied: beautifulsoup4>=4.11.1 in /usr/local/lib/python3.11/dist-packages (from yfinance) (4.13.3)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.11/dist-packages (from beautifulsoup4>=4.11.1->yfinance) (2.6)
Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.11/dist-packages (from beautifulsoup4>=4.11.1->yfinance) (4.12.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.11/dist-packages (from pandas>=1.3.0->yfinance) (2.8.2)
Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.11/dist-packages (from pandas>=1.3.0->yfinance) (2025.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests>=2.31->yfinance) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests>=2.31->yfinance) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests>=2.31->yfinance) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests>=2.31->yfinance) (2025.1.31)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.11/dist-packages (from python-dateutil>=2.8.2->pandas>=1.3.0->yfinance) (1.17.0)
# Download Stock data
df = yf.download("C", start="2020-01-01", end="2025-03-20")
# Print the downloaded data
df.head()
YF.download() has changed argument auto_adjust default to True
[*********************100%***********************] 1 of 1 completed
| Price | Close | High | Low | Open | Volume |
|---|---|---|---|---|---|
| Ticker | C | C | C | C | C |
| Date | |||||
| 2020-01-02 | 66.661324 | 66.685942 | 65.676541 | 65.758605 | 12728900 |
| 2020-01-03 | 65.405746 | 66.078679 | 65.200584 | 65.487816 | 12437400 |
| 2020-01-06 | 65.200569 | 65.282639 | 64.601499 | 64.601499 | 10059500 |
| 2020-01-07 | 64.634323 | 65.077474 | 64.609705 | 65.069266 | 10469100 |
| 2020-01-08 | 65.126709 | 65.651924 | 64.568667 | 64.642523 | 11292400 |
f_len = 32
col_name = 'Close'
quantiles, mean = pipeline.predict_quantiles(
context=torch.tensor(df[col_name].dropna().values), # Convert the Pandas Series to a NumPy array and drop NaN values before creating the tensor
prediction_length=f_len,
quantile_levels=[0.1, 0.5, 0.9],
)
forecast_index = range(len(df), len(df) + f_len)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]
plt.figure(figsize=(8, 4))
plt.plot(np.array(df[col_name]), color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
34.4. Treasuries (10 yr)#
df = pd.read_csv('https://fred.stlouisfed.org/graph/fredgraph.csv?bgcolor=%23ebf3fb&chart_type=line&drp=0&fo=open%20sans&graph_bgcolor=%23ffffff&height=450&mode=fred&recession_bars=on&txtcolor=%23444444&ts=12&tts=12&width=1320&nt=0&thu=0&trc=0&show_legend=yes&show_axis_titles=yes&show_tooltip=yes&id=DFII10&scale=left&cosd=2020-03-20&coed=2025-03-20&line_color=%230073e6&link_values=false&line_style=solid&mark_type=none&mw=3&lw=3&ost=-99999&oet=99999&mma=0&fml=a&fq=Daily&fam=avg&fgst=lin&fgsnd=2020-02-01&line_index=1&transformation=lin&vintage_date=2025-03-21&revision_date=2025-03-21&nd=2003-01-02')
df.head()
| observation_date | DFII10 | |
|---|---|---|
| 0 | 2020-03-20 | 0.17 |
| 1 | 2020-03-23 | -0.04 |
| 2 | 2020-03-24 | -0.13 |
| 3 | 2020-03-25 | -0.19 |
| 4 | 2020-03-26 | -0.24 |
f_len = 64
col_name = 'DFII10'
quantiles, mean = pipeline.predict_quantiles(
context=torch.tensor(df[col_name].dropna().values), # Convert the Pandas Series to a NumPy array and drop NaN values before creating the tensor
prediction_length=f_len,
quantile_levels=[0.1, 0.5, 0.9],
)
forecast_index = range(len(df), len(df) + f_len)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]
plt.figure(figsize=(8, 4))
plt.plot(np.array(df[col_name]), color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
34.5. Cryptocurrencies#
df = pd.read_csv('https://fred.stlouisfed.org/graph/fredgraph.csv?bgcolor=%23ebf3fb&chart_type=line&drp=0&fo=open%20sans&graph_bgcolor=%23ffffff&height=450&mode=fred&recession_bars=on&txtcolor=%23444444&ts=12&tts=12&width=1320&nt=0&thu=0&trc=0&show_legend=yes&show_axis_titles=yes&show_tooltip=yes&id=CBBTCUSD&scale=left&cosd=2020-03-20&coed=2025-03-20&line_color=%230073e6&link_values=false&line_style=solid&mark_type=none&mw=3&lw=3&ost=-99999&oet=99999&mma=0&fml=a&fq=Daily%2C%207-Day&fam=avg&fgst=lin&fgsnd=2020-02-01&line_index=1&transformation=lin&vintage_date=2025-03-21&revision_date=2025-03-21&nd=2014-12-01')
df.head()
| observation_date | CBBTCUSD | |
|---|---|---|
| 0 | 2020-03-20 | 6215.40 |
| 1 | 2020-03-21 | 6168.72 |
| 2 | 2020-03-22 | 5791.69 |
| 3 | 2020-03-23 | 6583.38 |
| 4 | 2020-03-24 | 6742.21 |
f_len = 64
col_name = 'CBBTCUSD'
quantiles, mean = pipeline.predict_quantiles(
context=torch.tensor(df[col_name].dropna().values), # Convert the Pandas Series to a NumPy array and drop NaN values before creating the tensor
prediction_length=f_len,
quantile_levels=[0.1, 0.5, 0.9],
)
forecast_index = range(len(df), len(df) + f_len)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]
plt.figure(figsize=(8, 4))
plt.plot(np.array(df[col_name]), color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
