34. Chronos forecasting#

The paper behind this approach is here: https://arxiv.org/pdf/2403.07815

The paper introduces Chronos, a framework that adapts language model architectures for time series forecasting. The key aspects are:

  1. Approach:

  • Converts time series data into tokens through scaling and quantization

  • Uses existing transformer language model architectures with minimal modifications

  • Trains models on both real and synthetic time series data

  1. Key Components:

  • TSMixup: Data augmentation technique that combines patterns from different time series

  • KernelSynth: Synthetic data generation using Gaussian processes

  • Based on T5 transformer architecture, with models ranging from 20M to 710M parameters

  1. Main Results:

  • Significantly outperforms traditional statistical models and task-specific deep learning models on in-domain datasets

  • Achieves competitive zero-shot performance on unseen datasets compared to models specifically trained on those datasets

  • Fine-tuning further improves performance on new datasets

  1. Key Benefits:

  • Eliminates need for task-specific training

  • Simplifies forecasting pipelines

  • Shows language model architectures can handle time series without specialized modifications

  • Provides strong zero-shot forecasting capabilities

  1. Limitations:

  • Inference speed slower than some specialized models

  • Edge cases with very sparse or high-variance time series

  • Limited by available high-quality public time series data

The paper demonstrates that language model architectures can be effectively adapted for time series forecasting with minimal modifications, potentially simplifying real-world forecasting applications.

  1. See Chronos: amazon-science/chronos-forecasting

  2. Docs: https://auto.gluon.ai/stable/tutorials/timeseries/forecasting-chronos.html

%%time
!pip install chronos-forecasting
Collecting chronos-forecasting
  Downloading chronos_forecasting-1.5.3-py3-none-any.whl.metadata (31 kB)
Requirement already satisfied: accelerate<2,>=0.32 in /usr/local/lib/python3.12/dist-packages (from chronos-forecasting) (1.10.1)
Requirement already satisfied: torch<3,>=2.0 in /usr/local/lib/python3.12/dist-packages (from chronos-forecasting) (2.8.0+cu126)
Requirement already satisfied: transformers<5,>=4.48 in /usr/local/lib/python3.12/dist-packages (from chronos-forecasting) (4.56.1)
Requirement already satisfied: numpy<3.0.0,>=1.17 in /usr/local/lib/python3.12/dist-packages (from accelerate<2,>=0.32->chronos-forecasting) (2.0.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from accelerate<2,>=0.32->chronos-forecasting) (25.0)
Requirement already satisfied: psutil in /usr/local/lib/python3.12/dist-packages (from accelerate<2,>=0.32->chronos-forecasting) (5.9.5)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.12/dist-packages (from accelerate<2,>=0.32->chronos-forecasting) (6.0.2)
Requirement already satisfied: huggingface_hub>=0.21.0 in /usr/local/lib/python3.12/dist-packages (from accelerate<2,>=0.32->chronos-forecasting) (0.35.0)
Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.12/dist-packages (from accelerate<2,>=0.32->chronos-forecasting) (0.6.2)
Requirement already satisfied: filelock in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.19.1)
Requirement already satisfied: typing-extensions>=4.10.0 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (4.15.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (75.2.0)
Requirement already satisfied: sympy>=1.13.3 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (1.13.3)
Requirement already satisfied: networkx in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.5)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.1.6)
Requirement already satisfied: fsspec in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (2025.3.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.6.77 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.6.77)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.6.77 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.6.77)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.6.80 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.6.80)
Requirement already satisfied: nvidia-cudnn-cu12==9.10.2.21 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (9.10.2.21)
Requirement already satisfied: nvidia-cublas-cu12==12.6.4.1 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.6.4.1)
Requirement already satisfied: nvidia-cufft-cu12==11.3.0.4 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (11.3.0.4)
Requirement already satisfied: nvidia-curand-cu12==10.3.7.77 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (10.3.7.77)
Requirement already satisfied: nvidia-cusolver-cu12==11.7.1.2 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (11.7.1.2)
Requirement already satisfied: nvidia-cusparse-cu12==12.5.4.2 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.5.4.2)
Requirement already satisfied: nvidia-cusparselt-cu12==0.7.1 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (0.7.1)
Requirement already satisfied: nvidia-nccl-cu12==2.27.3 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (2.27.3)
Requirement already satisfied: nvidia-nvtx-cu12==12.6.77 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.6.77)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.6.85 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (12.6.85)
Requirement already satisfied: nvidia-cufile-cu12==1.11.1.6 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (1.11.1.6)
Requirement already satisfied: triton==3.4.0 in /usr/local/lib/python3.12/dist-packages (from torch<3,>=2.0->chronos-forecasting) (3.4.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.12/dist-packages (from transformers<5,>=4.48->chronos-forecasting) (2024.11.6)
Requirement already satisfied: requests in /usr/local/lib/python3.12/dist-packages (from transformers<5,>=4.48->chronos-forecasting) (2.32.4)
Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /usr/local/lib/python3.12/dist-packages (from transformers<5,>=4.48->chronos-forecasting) (0.22.0)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.12/dist-packages (from transformers<5,>=4.48->chronos-forecasting) (4.67.1)
Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /usr/local/lib/python3.12/dist-packages (from huggingface_hub>=0.21.0->accelerate<2,>=0.32->chronos-forecasting) (1.1.10)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.12/dist-packages (from sympy>=1.13.3->torch<3,>=2.0->chronos-forecasting) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.12/dist-packages (from jinja2->torch<3,>=2.0->chronos-forecasting) (3.0.2)
Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests->transformers<5,>=4.48->chronos-forecasting) (3.4.3)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.12/dist-packages (from requests->transformers<5,>=4.48->chronos-forecasting) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests->transformers<5,>=4.48->chronos-forecasting) (2.5.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.12/dist-packages (from requests->transformers<5,>=4.48->chronos-forecasting) (2025.8.3)
Downloading chronos_forecasting-1.5.3-py3-none-any.whl (29 kB)
Installing collected packages: chronos-forecasting
Successfully installed chronos-forecasting-1.5.3
CPU times: user 2.21 s, sys: 874 ms, total: 3.09 s
Wall time: 13 s

34.1. Set Up#

import pandas as pd  # requires: pip install pandas
import torch
from chronos import BaseChronosPipeline
import matplotlib.pyplot as plt  # requires: pip install matplotlib
import numpy as np

pipeline = BaseChronosPipeline.from_pretrained(
    "amazon/chronos-t5-small",  # use "amazon/chronos-bolt-small" for the corresponding Chronos-Bolt model
    device_map="cuda",  # use "cpu" for CPU inference
    torch_dtype=torch.bfloat16,
)

/usr/local/lib/python3.12/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: 
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
  warnings.warn(
`torch_dtype` is deprecated! Use `dtype` instead!

34.2. Forecasting Passenger Traffic#

df = pd.read_csv(
    "https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv"
)

# context must be either a 1D tensor, a list of 1D tensors,
# or a left-padded 2D tensor with batch as the first dimension
# quantiles is an fp32 tensor with shape [batch_size, prediction_length, num_quantile_levels]
# mean is an fp32 tensor with shape [batch_size, prediction_length]
quantiles, mean = pipeline.predict_quantiles(
    context=torch.tensor(df["#Passengers"]),
    prediction_length=12,
    quantile_levels=[0.1, 0.5, 0.9],
)
df.head()
df.tail()
from chronos import ChronosPipeline, ChronosBoltPipeline

print(ChronosPipeline.predict.__doc__)  # for Chronos models
print(ChronosBoltPipeline.predict.__doc__)  # for Chronos-Bolt models
        Get forecasts for the given time series.

        Refer to the base method (``BaseChronosPipeline.predict``)
        for details on shared parameters.

        Additional parameters
        ---------------------
        num_samples
            Number of sample paths to predict. Defaults to what
            specified in ``self.model.config``.
        temperature
            Temperature to use for generating sample tokens.
            Defaults to what specified in ``self.model.config``.
        top_k
            Top-k parameter to use for generating sample tokens.
            Defaults to what specified in ``self.model.config``.
        top_p
            Top-p parameter to use for generating sample tokens.
            Defaults to what specified in ``self.model.config``.
        limit_prediction_length
            Force prediction length smaller or equal than the
            built-in prediction length from the model. False by
            default. When true, fail loudly if longer predictions
            are requested, otherwise longer predictions are allowed.

        Returns
        -------
        samples
            Tensor of sample forecasts, of shape
            (batch_size, num_samples, prediction_length).
        

        Get forecasts for the given time series.

        Refer to the base method (``BaseChronosPipeline.predict``)
        for details on shared parameters.
        Additional parameters
        ---------------------
        limit_prediction_length
            Force prediction length smaller or equal than the
            built-in prediction length from the model. False by
            default. When true, fail loudly if longer predictions
            are requested, otherwise longer predictions are allowed.

        Returns
        -------
        torch.Tensor
            Forecasts of shape (batch_size, num_quantiles, prediction_length)
            where num_quantiles is the number of quantiles the model has been
            trained to output. For official Chronos-Bolt models, the value of
            num_quantiles is 9 for [0.1, 0.2, ..., 0.9]-quantiles.

        Raises
        ------
        ValueError
            When limit_prediction_length is True and the prediction_length is
            greater than model's trainig prediction_length.
        
forecast_index = range(len(df), len(df) + 12)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]

plt.figure(figsize=(8, 4))
plt.plot(df["#Passengers"], color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
_images/9d8098eb6a3526919343fb328ad4245a392938709f1770a06340af47ba7f8db4.png

34.3. Forecasting Stock Prices#

# Code to use yfinance to download bitcoin data from FRED ticker CBBTCUSD

!pip install yfinance

import yfinance as yf
Requirement already satisfied: yfinance in /usr/local/lib/python3.12/dist-packages (0.2.66)
Requirement already satisfied: pandas>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from yfinance) (2.2.2)
Requirement already satisfied: numpy>=1.16.5 in /usr/local/lib/python3.12/dist-packages (from yfinance) (2.0.2)
Requirement already satisfied: requests>=2.31 in /usr/local/lib/python3.12/dist-packages (from yfinance) (2.32.4)
Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.12/dist-packages (from yfinance) (0.0.12)
Requirement already satisfied: platformdirs>=2.0.0 in /usr/local/lib/python3.12/dist-packages (from yfinance) (4.4.0)
Requirement already satisfied: pytz>=2022.5 in /usr/local/lib/python3.12/dist-packages (from yfinance) (2025.2)
Requirement already satisfied: frozendict>=2.3.4 in /usr/local/lib/python3.12/dist-packages (from yfinance) (2.4.6)
Requirement already satisfied: peewee>=3.16.2 in /usr/local/lib/python3.12/dist-packages (from yfinance) (3.18.2)
Requirement already satisfied: beautifulsoup4>=4.11.1 in /usr/local/lib/python3.12/dist-packages (from yfinance) (4.13.5)
Requirement already satisfied: curl_cffi>=0.7 in /usr/local/lib/python3.12/dist-packages (from yfinance) (0.13.0)
Requirement already satisfied: protobuf>=3.19.0 in /usr/local/lib/python3.12/dist-packages (from yfinance) (5.29.5)
Requirement already satisfied: websockets>=13.0 in /usr/local/lib/python3.12/dist-packages (from yfinance) (15.0.1)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4>=4.11.1->yfinance) (2.8)
Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4>=4.11.1->yfinance) (4.15.0)
Requirement already satisfied: cffi>=1.12.0 in /usr/local/lib/python3.12/dist-packages (from curl_cffi>=0.7->yfinance) (2.0.0)
Requirement already satisfied: certifi>=2024.2.2 in /usr/local/lib/python3.12/dist-packages (from curl_cffi>=0.7->yfinance) (2025.8.3)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas>=1.3.0->yfinance) (2.9.0.post0)
Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas>=1.3.0->yfinance) (2025.2)
Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests>=2.31->yfinance) (3.4.3)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.12/dist-packages (from requests>=2.31->yfinance) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests>=2.31->yfinance) (2.5.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.12/dist-packages (from cffi>=1.12.0->curl_cffi>=0.7->yfinance) (2.23)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas>=1.3.0->yfinance) (1.17.0)
# Download Stock data
df = yf.download("GOOGL", start="2023-08-31", end="2025-09-01")

# Print the downloaded data
df.head()
/tmp/ipython-input-1492018986.py:2: FutureWarning: YF.download() has changed argument auto_adjust default to True
  df = yf.download("GOOGL", start="2023-08-31", end="2025-09-01")

[*********************100%***********************]  1 of 1 completed
Price Close High Low Open Volume
Ticker GOOGL GOOGL GOOGL GOOGL GOOGL
Date
2023-08-31 135.237244 137.054710 134.859842 135.078336 30053800
2023-09-01 134.730759 136.518432 133.926309 136.518432 21543700
2023-09-05 134.839981 135.485523 133.658130 134.512240 19403100
2023-09-06 133.538986 135.594799 132.754389 135.088298 18684500
2023-09-07 134.333496 134.651311 132.039321 132.674937 18844300
f_len = 32
col_name = 'Close'

quantiles, mean = pipeline.predict_quantiles(
    context=torch.tensor(df[col_name].dropna().values[-126:]), # Convert the Pandas Series to a NumPy array and drop NaN values before creating the tensor
    prediction_length=f_len,
    quantile_levels=[0.1, 0.5, 0.9],
)
forecast_index = range(len(df[-126:]), len(df[-126:]) + f_len)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]

plt.figure(figsize=(8, 4))
plt.plot(np.array(df[col_name][-126:]), color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
_images/08fc71be318ab817344c653d88bf01f4235a4d396573d211f36254b018307a1a.png

34.4. Stock Returns#

f_len = 21
col_name = 'Close'
ts = df[col_name].pct_change().dropna().values

quantiles, mean = pipeline.predict_quantiles(
    context=torch.tensor(ts), # Convert the Pandas Series to a NumPy array and drop NaN values before creating the tensor
    prediction_length=f_len,
    quantile_levels=[0.1, 0.5, 0.9],
)
forecast_index = range(len(ts), len(ts) + f_len)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]

plt.figure(figsize=(8, 4))
plt.plot(np.array(ts), color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
_images/64d9e31c067ab02ed87a3ade0663be65d8382a50df17305f2b7b3d45a3d1837a.png

34.5. Treasuries (10 yr)#

df = pd.read_csv('https://fred.stlouisfed.org/graph/fredgraph.csv?bgcolor=%23ebf3fb&chart_type=line&drp=0&fo=open%20sans&graph_bgcolor=%23ffffff&height=450&mode=fred&recession_bars=on&txtcolor=%23444444&ts=12&tts=12&width=1320&nt=0&thu=0&trc=0&show_legend=yes&show_axis_titles=yes&show_tooltip=yes&id=DFII10&scale=left&cosd=2020-03-20&coed=2025-03-20&line_color=%230073e6&link_values=false&line_style=solid&mark_type=none&mw=3&lw=3&ost=-99999&oet=99999&mma=0&fml=a&fq=Daily&fam=avg&fgst=lin&fgsnd=2020-02-01&line_index=1&transformation=lin&vintage_date=2025-03-21&revision_date=2025-03-21&nd=2003-01-02')
df.head()
observation_date DFII10
0 2020-03-20 0.17
1 2020-03-23 -0.04
2 2020-03-24 -0.13
3 2020-03-25 -0.19
4 2020-03-26 -0.24
f_len = 64
col_name = 'DFII10'

quantiles, mean = pipeline.predict_quantiles(
    context=torch.tensor(df[col_name].dropna().values), # Convert the Pandas Series to a NumPy array and drop NaN values before creating the tensor
    prediction_length=f_len,
    quantile_levels=[0.1, 0.5, 0.9],
)
forecast_index = range(len(df), len(df) + f_len)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]

plt.figure(figsize=(8, 4))
plt.plot(np.array(df[col_name]), color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
_images/a6eb040631b0f5e675d34e33ec7088732c3fd98e052cad8db1dcc2f254d9921d.png

34.6. Cryptocurrencies#

df = pd.read_csv('https://fred.stlouisfed.org/graph/fredgraph.csv?bgcolor=%23ebf3fb&chart_type=line&drp=0&fo=open%20sans&graph_bgcolor=%23ffffff&height=450&mode=fred&recession_bars=on&txtcolor=%23444444&ts=12&tts=12&width=1320&nt=0&thu=0&trc=0&show_legend=yes&show_axis_titles=yes&show_tooltip=yes&id=CBBTCUSD&scale=left&cosd=2020-03-20&coed=2025-03-20&line_color=%230073e6&link_values=false&line_style=solid&mark_type=none&mw=3&lw=3&ost=-99999&oet=99999&mma=0&fml=a&fq=Daily%2C%207-Day&fam=avg&fgst=lin&fgsnd=2020-02-01&line_index=1&transformation=lin&vintage_date=2025-03-21&revision_date=2025-03-21&nd=2014-12-01')
df.head()
observation_date CBBTCUSD
0 2020-03-20 6215.40
1 2020-03-21 6168.72
2 2020-03-22 5791.69
3 2020-03-23 6583.38
4 2020-03-24 6742.21
f_len = 64
col_name = 'CBBTCUSD'

quantiles, mean = pipeline.predict_quantiles(
    context=torch.tensor(df[col_name].dropna().values), # Convert the Pandas Series to a NumPy array and drop NaN values before creating the tensor
    prediction_length=f_len,
    quantile_levels=[0.1, 0.5, 0.9],
)
forecast_index = range(len(df), len(df) + f_len)
low, median, high = quantiles[0, :, 0], quantiles[0, :, 1], quantiles[0, :, 2]

plt.figure(figsize=(8, 4))
plt.plot(np.array(df[col_name]), color="royalblue", label="historical data")
plt.plot(forecast_index, median, color="tomato", label="median forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
plt.legend()
plt.grid()
plt.show()
_images/d627e15c3b0c8ab6a8a1740c48791c988cf15ea83574f1dd25411757469c5f20.png