load_partial_data#

API documentation for tradeexecutor.strategy.trading_strategy_universe.load_partial_data Python function.

load_partial_data(client, execution_context, time_bucket, pairs, universe_options, liquidity=False, liquidity_time_bucket=None, liquidity_query_type=OHLCVCandleType.tvl_v1, preloaded_tvl_df=None, stop_loss_time_bucket=None, required_history_period=None, lending_reserves=None, lending_candle_types=(<LendingCandleType.supply_apr: 'supply_apr'>, <LendingCandleType.variable_borrow_apr: 'variable_borrow_apr'>), start_at=None, end_at=None, name=None, candle_progress_bar_desc=None, lending_candle_progress_bar_desc=None, pair_extra_metadata=False, vaults=None, vault_bundled_price_data=False, round_start_end=True)[source]#

Load pair data for given trading pairs.

A loading function designed to load OHLCV data up to 1000 pairs. Instead of loading all pair data over Parquet datasets, load only specific pair data from their corresponding JSONL endpoints, streaming the data and caching them as local files.

Because or partial data processing, this function has less chance to run out of memory than unlike tradeexecutor.strategy.trading_strategy_universe.load_all_data().

Example of loading spot-only data:

… code-block:: python

from tradeexecutor.strategy.trading_strategy_universe import load_partial_data from tradingstrategy.transport.cache import OHLCVCandleType

def create_trading_universe(

ts: datetime.datetime, client: Client, execution_context: ExecutionContext, universe_options: UniverseOptions,

) -> TradingStrategyUniverse:

dataset = load_partial_data(

client=client, time_bucket=Parameters.candle_time_bucket, pairs=pairs_df, execution_context=execution_context, universe_options=universe_options, liquidity=True, liquidity_time_bucket=TimeBucket.d1, liquidity_query_type=OHLCVCandleType.tvl_v2,

)

strategy_universe = TradingStrategyUniverse.create_from_dataset(

dataset, reserve_asset=”0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913”, # USDC on Base forward_fill=True, # We got very gappy data from low liquid DEX coins

)

return universe

Example of loading spot pair and Aave credit pool data:

from tradeexecutor.strategy.trading_strategy_universe import load_partial_data
# TODO
pass
Parameters:

  • client (BaseClient) – Trading Strategy client instance

  • time_bucket (TimeBucket) – The candle time frame.

  • pairs (Union[Collection[Union[Tuple[ChainId, str | None, str, str, float], Tuple[ChainId, str | None, str, str]]], DataFrame]) –

    List of trading pair tickers.

    Can be

    • Human-readable descriptions, see tradingstrategy.pair.HumanReadableTradingPairDescription.

    • Direct pandas.DataFrame of pairs.

  • lending_resserves –

    Lending reserves for which you want to download the data.

    Either list of lending pool descriptions or preloaded lending universe.

  • lending_candle_types (Collection[LendingCandleType]) – What lending data columns to load

  • liquidity – Set true to load liquidity data as well

  • liquidity_time_bucket (tradingstrategy.timebucket.TimeBucket | None) –

    Granularity of loaded TVL data.

    If not given use time_bucket.

  • liquidity_query_type (OHLCVCandleType) –

    Whether to use new-style or old-style data for TVL.

    See OHLCVCandleType for details.

  • preloaded_tvl_df (pandas.core.frame.DataFrame | None) –

    Liquidity data was earlier loaded with fetch_tvl(min_tvl) when constructing the trading universe.

    We do not reload this same data, but use the preloaded DataFrame directly.

  • lending_reserves (Optional[Union[LendingReserveUniverse, Collection[Union[Tuple[ChainId, LendingProtocolType, str], Tuple[ChainId, LendingProtocolType, str, str]]]]]) – Set true to load lending reserve data as well

  • stop_loss_time_bucket (Optional[TimeBucket]) – If set load stop loss trigger data using this candle granularity.

  • execution_context (ExecutionContext) – Defines if we are live or backtesting

  • universe_options (UniverseOptions) – Override values given the strategy file. Used in testing the framework.

  • required_history_period (datetime.timedelta | None) –

    How much historical data we need to load.

    Depends on the strategy. Defaults to load all data.

  • start_at (datetime.datetime | None) –

    Load data for a specific backtesting data range.

    TODO: Going to be deprecatd. Please use universe_options.start_at instead.

  • end_at (datetime.datetime | None) –

    Load data for a specific backtesting data range.

    TODO: Going to be deprecatd. Please use universe_options.end_at instead.

  • name (str | None) – The loading operation name used in progress bars

  • candle_progress_bar_desc (str | None) – Override the default progress bar message

  • lending_candle_progress_bar_desc (str | None) – Override the default progress bar message

  • pair_extra_metadata –

    Load TokenSniffer data, buy/sell tax and other extra metadata.

    Slow and API endpoint severely limited. Use only if you are dealing with a limited number of pairs.

  • vaults (list[tuple[tradingstrategy.chain.ChainId, str]] | None) –

    List of (chain, vault address) tuples to load vault data for.

    Vault metadata loeaded from tradingstrategy data bundle.

    Currently does not load any historical data.

  • vault_bundled_price_data (bool | pathlib.Path) –

    For vaults, also load bundled static price data.

    Use minimal backtest dataset from Trading Strategy package bundle.

    Not applicable for live trading.

  • round_start_end (bool) –

    Round start and end times to the nearest time bucket.

    Avoid polluting cache space with second difference in timestamps.

    Only applicable for live trading. This is especially helpful for making unit testing/integration testing faster.

    If not given live trading tries to load the data from the server until the last second.

    Max 1d rounding. For weekly time bucket you still get daily boundaries.

Lending_candle_types:

What lending data columns to load

Returns:

Datataset containing the requested data

Return type:

Dataset