Do a rolling with a left-exclusive window and then a backwards asof_joinjoin_asofto to add the extra value you want outside the window back in:
(df.rolling("timestamp", period="5m", closed="right") .agg("value") .join_asof(df, left_on="timestamp", right_on=pl.col.timestamp.dt.offset_by("5m")) .select( pl.col.timestamp, pl.when(pl.col.value_right.is_null()) .then(pl.col.value) .otherwise(pl.concat_list(pl.col.value_right, pl.col.value)) ) .with_columns(rolling_value=pl.col.value.list.mean()) )