When executing load_dataset to retrieve Hugging Face repositories, a ValueError concerning invalid glob patterns may surface during the data streaming initialization. This specifically occurs with datasets version 2.20.0 when the libray attempts to resolve file paths containing recursive wildcards.

from datasets import load_dataset

training_data = load_dataset("organization/repo-name", split="train")

The complete error traceback indicates the failure point resides in fsspec/utils.py during pattern translation:

ValueError: Invalid pattern: '**' can only be an entire path component

The issue stems from compatibility constraints between datasets 2.20.0 and recent fsspec releases regarding glob syntax validation. When the resolver encounters ** characters within relative paths, the strict parsing logic in newer fsspec versions rejects these patterns unless they constitute complete path components.

Resolving this requires downgrading the filesystem specification library to a version predating these validation changes:

pip install fsspec==2023.9.2

This version constraint relaxes the glob pattern restrictions, allowing load_dataset to successfully index repository contents without triggering path resolution errors. While this approach may introduce version conflicts with other packages dependent on recent fsspec features, it effectively bypasses the immediate pattern validation failure.