All pull requests

Pull requests

• Drop validate_schema for PyArrow 15 ParquetDataset

  #821·Hashim1999164 opened Oct 3, 2026·

  ·
  1

• Put caller cwd on sys.path for generate metadata

  #820·Hashim1999164 opened Aug 22, 2026·

  ·
  1

  1

• Upgrade GitHub Actions for Node 24 compatibility

  #819·salmanmkc opened Feb 7, 2026·

  ·
  1

• Test deployment action

  #812·arushi297 opened Aug 10, 2025Collaborator·

  ·
  1

• Enable consistent ordering in Petastorm

  #811·arushi297 opened Aug 8, 2025Collaborator·

  ·
  1

• [WIP] Implement round robin and separate ventilator and result queue

  #807·arushi297 opened Jul 11, 2025Collaborator·

  ·
  2

• Add a ThreadPool which respects the order of Parquet dataset pieces.

  #796·wbeardall opened May 12, 2023·

  ·
  3

• [Test] Run CI against pyspark 3.4

  #791·WeichenXu123 opened Feb 28, 2023Collaborator·

  ·
  3

• Update CI to use latest versions of pyarrow and numpy. Drop pyarrow 4 test config.

  #778·selitvin opened Sep 15, 2022Collaborator·

  ·
  2

• Add a collate_lists_fn

  #772·selitvin opened Aug 29, 2022Collaborator·

  ·
  1

• Wrapper for Arrow Datasets & Dataset Pieces

  #754·aperiodic opened Apr 28, 2022·

  ·
  5

• Update README.rst

  #753·a-klos opened Apr 22, 2022·

  ·
  1

• Add Python3.10 to CI docker image

  #752·selitvin opened Apr 21, 2022Collaborator·

  ·
  2

• Fix type of the a batch returned by make_batch_reader when TransformSpec's function returns column with all values being None

  #750·selitvin opened Apr 8, 2022Collaborator·

  ·
  1

  3

• Enable batch fetching in parallel

  #748·jarandaf opened Mar 23, 2022·

  ·
  17

• Use spark_test_ctx fixture instead of constructing spark manually

  #711·selitvin opened Jul 30, 2021Collaborator·

  ·
  1

• Remove very old pickle compatibility code modifying old atg package names

  #702·selitvin opened Jul 26, 2021Collaborator·

  ·
  2

• Use pyarrow.fs.LocalFileSystem as per arrow 4.x

  #698·JayjeetAtGithub opened Jul 5, 2021·

  ·
  2

• fix get_dataset_path() in fs_utils.py

  #663·dongpohezui opened Apr 10, 2021·

  ·
  7

• Remove Unischema __getattr__ implementation

  #656·v01dXYZ opened Mar 18, 2021·

  ·
  2

• Use new pyarrow.fs filesystem objects

  #619·selitvin opened Nov 7, 2020Collaborator·

  ·
  1

• Reading lists of numpy arrays

  #605·selitvin opened Sep 19, 2020Collaborator·

  ·
  1

• Fix GCSFS walk() method

  #561·megaserg opened Jun 14, 2020Contributor·

  ·
  1

  13

• Parallelize encoding of a single row

  #546·selitvin opened Apr 20, 2020Collaborator·

  ·
  2

• [WIP] Auto infer schema (including fields shape) from the first row

  #512·WeichenXu123 opened Mar 23, 2020Collaborator·

  ·
  7