GroupedCandleUniverse#

API documentation for tradingstrategy.candle.GroupedCandleUniverse Python class in Trading Strategy framework.

class GroupedCandleUniverse[source]#

Bases: PairGroupedUniverse

A candle universe where each trading pair has its own candles.

This is helper class to create foundation for multi pair strategies.

For the data logistics purposes, all candles are lumped together in single columnar data blobs. However, it rarely makes sense to execute operations over different trading pairs. :py:class`GroupedCandleUniverse` creates trading pair id -> candle data grouping out from raw candle data.

Usage:

# Get candles for SUSHI-USDT

exchange_universe = client.fetch_exchange_universe()
raw_pairs = client.fetch_pair_universe().to_pandas()
raw_candles = client.fetch_all_candles(TimeBucket.d7).to_pandas()

pair_universe = PandasPairUniverse(raw_pairs)
candle_universe = GroupedCandleUniverse(raw_candles)

# Do some test calculations for a single pair
sushi_swap = exchange_universe.get_by_chain_and_name(ChainId.ethereum, "sushi")
sushi_usdt = pair_universe.get_one_pair_from_pandas_universe(sushi_swap.exchange_id, "SUSHI", "USDT")

raw_candles = client.fetch_all_candles(TimeBucket.d7).to_pandas()
candle_universe = GroupedCandleUniverse(raw_candles)
sushi_usdth_candles = candle_universe.get_candles_by_pair(sushi_usdt.pair_id)

__init__(df, time_bucket=TimeBucket.d1, timestamp_column='timestamp', index_automatically=True, fix_wick_threshold=(0.1, 1.9), fix_inbetween_threshold=(- 0.99, 5.0), primary_key_column='pair_id', remove_candles_with_zero_volume=True, forward_fill=False, bad_open_close_threshold=3.0, autoheal_pair_limit=1500, forward_fill_until=None, min_max_price=(1e-08, 1000000.0))#

Set up new candle universe where data is grouped by trading pair.

Parameters:

df (DataFrame) – DataFrame backing the data.
time_bucket (TimeBucket) –
What bar size candles we are operating at. Default to daily.

TODO: Currently not used. Will be removed in the future versions.
timestamp_column (str) – What column use to build a time index. Used for QStrader / Backtrader compatibility.
index_automatically (bool) – Convert the index to use time series. You might avoid this with QSTrader kind of data.
fix_wick_threshold (tuple | None) –
Apply abnormal high/low wick fix filter.

Percent value of maximum allowed high/low wick relative to close. By default fix values where low is 90% lower than close and high is 90% higher than close.

See fix_bad_wicks() for more information.
bad_open_close_threshold (float | None) – See fix_bad_wicks().
primary_key_column (str) – The pair/reserve id column name in the dataframe.
remove_zero_candles –
Remove candles with zero values for OHLC.

To deal with abnormal data.
forward_fill (bool) –
Forward-will gaps in the data.

See forward fill and forward filling data for more information.
autoheal_pair_limit –
Don’t try to autoheal data if the candle universe is too large.

Autohealing is very taxing operation and should not be performed on large universes. Instead you should preprocess the universe to a candles Parquet file and load directly from there.
autoheal_limit – If we have more than
fix_inbetween_threshold (tuple | None) –
remove_candles_with_zero_volume (bool) –
forward_fill_until (datetime.datetime | pandas._libs.tslibs.timestamps.Timestamp | None) –

Methods

`__init__`(df[, time_bucket, ...])	Set up new candle universe where data is grouped by trading pair.
`calculate_returns`([column])	Calculate timeframe returns for all trading pairs.
`clear_cache`()	Clear candles cached by pair.
`create_empty`()	Return an empty GroupedCandleUniverse
`create_empty_qstrader`()	Return an empty GroupedCandleUniverse.
`create_from_multiple_candle_dataframes`(dfs)	Construct universe based on multiple trading pairs.
`create_from_single_pair_dataframe`(df[, bucket])	Construct universe based on a single trading pair data.
`forward_fill`([columns, drop_other_columns])	Forward-fill sparse OHLCV candle data.
`get_all_pairs`([max_count])	Go through all liquidity samples, one DataFrame per trading pair.
`get_all_samples_by_range`(start, end)	Get list of candles/samples for all pairs at a certain range.
`get_all_samples_by_timestamp`(ts)	Get list of candles/samples for all pairs at a certain timepoint.
`get_candle_count`()	Return the dataset size - how many candles total
`get_candles_by_pair`(pair)	Get all price candles for a single trading pair.
`get_closest_price`(pair, when[, kind, ...])	Get the available price for a trading pair at a specific timepoint or some candles before the timepoint.
`get_columns`()	Get column names from the underlying pandas.GroupBy object
`get_last_entries_by_pair_and_timestamp`(pair, ...)	Get samples for a single pair before a timestamp.
`get_pair_count`()	Return the number of pairs in this dataset.
`get_pair_ids`()	Get all pairs present in the dataset
`get_price_with_tolerance`(pair, when, tolerance)	Get the price for a trading pair at a specific time point, or before within a time range tolerance.
`get_prior_timestamp`(ts)	Get the first timestamp in the index that is before the given timestamp.
`get_sample_count`()	Return the dataset size - how many samples total for all pairs
`get_samples_by_pair`(pair_id)	Get samples for a single pair.
`get_single_pair_data`([timestamp, ...])	Get all candles/liquidity samples for the single alone pair in the universe by a certain timestamp.
`get_single_value`(asset_id, when, ...[, ...])	Get a single value for a single pair/asset at a specific point of time.
`get_timestamp_range`([use_timezone, ...])	Return the time range of data we have for.
`is_forward_filled`()	Check if the data was forward filled after the data loading.
`iterate_samples_by_pair_range`(start, end)	Get list of candles/samples for all pairs at a certain range.

Attributes

get_candle_count()[source]#

Return the dataset size - how many candles total

Return type:: int

get_candles_by_pair(pair)[source]#

Get all price candles for a single trading pair.

Example in a trading strategy that uses multiple pairs:

pair_description = (ChainId.centralised_exchange, "binance", "BTC", "USDT")
pair = strategy_universe.data_universe.pairs.get_by_human_description(pair_description)
candles_df = strategy_universe.data_universe.candles.get_candles_by_pair(pair)

first_close = candles_df.iloc[0]["close"]
first_close_at = candles_df.index[0]
print(f"Pair {pair} first close price {first_close} at {first_close_at}")

This method returns candles data and not timestamp cropped like get_closest_price(), which is more suited for strategy decision cycle like workflows.

Parameters:

pair (PrimaryKey | tradingstrategy.pair.DEXPair) – Trading pair internal id or DEXPair instance

Returns:

Pandas dataframe object with the following columns.

Pandas DataFrame generated with pandas.core.groupby.DataFrameGroupBy. Return None if there is no candle data for this pair_id.

timestamp
open
high
low
close

Return type:

Optional[DataFrame]

get_closest_price(pair, when, kind='close', look_back_time_frames=5)[source]#

Get the available price for a trading pair at a specific timepoint or some candles before the timepoint.

Warning

This is a slow lookup method. You might want to use get_price_with_tolerance() instead.

Parameters:

pair (int | tradingstrategy.pair.DEXPair) – Trading pair id or pair object.
when (pandas._libs.tslibs.timestamps.Timestamp | datetime.datetime) –
Timestamp to query.

This is rounded down to the nearest time bucket. E.g. when using 1d candles, 2023-09-30 22:04:00.383776 will be rounded down to 2023-09-30 00:00.
kind – One of OHLC data points: “open”, “close”, “low”, “high”
look_back_timeframes – If there is no liquidity sample available at the exact timepoint, look to the past to the get the nearest sample. For example if candle time interval is 5 minutes and look_back_timeframes is 10, then accept a candle that is maximum of 50 minutes before the timepoint.

Returns:

We always return a price. In the error cases an exception is raised.

Raises:

CandleSampleUnavailable – There was no samples available with the given condition.

Return type:

float

get_price_with_tolerance(pair, when, tolerance, kind='close', pair_name_hint=None, ignore_forward_fill=False)[source]#

Get the price for a trading pair at a specific time point, or before within a time range tolerance.

The data may be sparse data. There might not be sample available in the same time point or immediate previous time point. In this case the method looks back for the previous data point within tolerance time range.

This method should be relative fast and optimised for any price, volume and liquidity queries.

Example:

test_price, distance = universe.get_price_with_tolerance(
    pair_id=1,
    when=pd.Timestamp("2020-02-01 00:05"),
    tolerance=pd.Timedelta(30, "m"))

# Returns closing price of the candle 2020-02-01 00:00,
# which is 5 minutes off when we asked
assert test_price == pytest.approx(100.50)
assert distance == pd.Timedelta("5m")

Parameters:

pair (int | tradingstrategy.pair.DEXPair) – Trading pair id
when (pandas._libs.tslibs.timestamps.Timestamp | datetime.datetime) – Timestamp to query
kind – One of OHLC data points: “open”, “close”, “low”, “high”, “volume”
tolerance (Timedelta) – If there is no liquidity sample available at the exact timepoint, look to the past to the get the nearest sample. For example if candle time interval is 5 minutes and look_back_timeframes is 10, then accept a candle that is maximum of 50 minutes before the timepoint.
pair_name_hint (str | None) –
What should we call this pair in the error messages.

If not given, try to figure out from the context.
ignore_forward_fill (bool) –
Ignore any data generated by forward fill.

Ignore rows where column foraward_filled is set to True.

Returns:

Return (price, delay) tuple. We always return a price. In the error cases an exception is raised. The delay is the timedelta between the wanted timestamp and the actual timestamp of the candle.

Candles are always timestamped by their opening.

Raises:

CandleSampleUnavailable – There were no samples available with the given condition.

Return type:

Tuple[float, Timedelta]

calculate_returns(column='close')[source]#

Calculate timeframe returns for all trading pairs.

Returns:

Series with MultiIndex (pair_id, timestamp) and returns for each candle.

Frequency remains original.

Return type:

Series

static create_empty()[source]#

Return an empty GroupedCandleUniverse

Return type:: GroupedCandleUniverse

static create_empty_qstrader()[source]#

Return an empty GroupedCandleUniverse.

TODO: Fix QSTrader to use “standard” column names.

Return type:: GroupedCandleUniverse