ak.to_arrow — Awkward Array documentation

# ak.to_arrow

Defined in awkward.operations.convert on line 1999.

ak.to_arrow(array, list_to32=False, string_to32=True, bytestring_to32=True, allow_tensor=True)
  Parameters:

    - array – Data to convert to an Apache Arrow array.

    - list_to32 (bool) – If True, convert Awkward lists into 32-bit Arrow lists if they’re small enough, even if it means an extra conversion. Otherwise, signed 32-bit `ak.layout.ListOffsetArray` maps to Arrow `ListType` and all others map to Arrow `LargeListType`.

    - string_to32 (bool) – Same as the above for Arrow `string` and `large_string`.

    - bytestring_to32 (bool) – Same as the above for Arrow `binary` and `large_binary`.

    - allow_tensor (bool) – If True, convert regular-length lists to `pyarrow.lib.Tensor`; otherwise, make `pyarrow.lib.ListArray` (generating offsets). This is used by `ak.to_parquet`, since Parquet files can’t contain regular-length tensors.

Converts an Awkward Array into an Apache Arrow array.

This produces arrays of type `pyarrow.Array`. You might need to further manipulations (using the pyarrow library) to build a `pyarrow.ChunkedArray`, a `pyarrow.RecordBatch`, or a `pyarrow.Table`.

Arrow arrays can maintain the distinction between “option-type but no elements are missing” and “not option-type” at all levels except the top level. Also, there is no distinction between `?union[X, Y, Z]]` type and `union[?X, ?Y, ?Z]` type. Be aware of these type distinctions when passing data through Arrow or Parquet.

See also `ak.from_arrow`, `ak.to_arrow_table`, `ak.to_parquet`.