Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Read pfeed data with polars

Open in molab

Built with pfeed →, an open-source data library by PFund, which also maintains PyTrade.org.

Download data with pfeed

Download one day of 1-minute ETH perpetual bars from Bybit, and store them locally with ParquetIO().

Config file /home/runner/.pfeed/config/config.yaml is corrupted or missing, resetting to default
Copied logging.yaml to /home/runner/.pfeed/config
Running BYBIT dataflows: 100%|██████████| 1/1 [00:05<00:00,  5.33s/it]
Loading...

Read it back with polars

ParquetIO() stored the 1440 rows as a plain parquet file in a hive layout under <data_path>/parquet, where data_path is pfeed’s configured data folder:

env=BACKTEST
    └── data_layer=CLEANED
        └── data_domain=MARKET_DATA
            └── data_source=BYBIT
                └── data_origin=BYBIT
                    └── product_type=PERPETUAL
                        └── resolution=1_MINUTE
                            └── product=BYBIT_ETH_USDT_PERPETUAL
                                └── year=2026
                                    └── month=11
                                        └── day=10
                                            └── part-0.parquet

So polars can read them directly. With hive_partitioning=True, every key=value folder becomes a column, and filtering on those columns only reads the matching files.

Loading...