D4RL
Collection of reference environments, offline reinforcement learning
...It contains standardized environments, tasks and datasets (observations, actions, rewards, terminals) aimed at enabling reproducible research in offline RL. Researchers can load a dataset for a given task (e.g., maze navigation, manipulation) and apply their algorithm without the need to collect fresh transitions, which accelerates experimentation and comparison. The API is based on Gymnasium (via gym.make) and each environment also exposes a method get_dataset() that returns the offline data to learn from. The repository emphasizes open science, reproducibility, and benchmarking at scale, making it easier to compare algorithms on equal footing.