apache / apache/arrow

[CI][Python][C++] Support on Power Architecture

Open
#43,817 36 comments 0 reactions 0 assignees View on GitHub
Component: C++ Component: Continuous Integration Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

**Description:**
We need to extend support for apache/arrow to the POWER/PPC64LE architecture.

**Background:**
• We have forked the apache/arrow repository and have successfully generated and tested wheels for both C++ and Python using a self-hosted CI runner on an OSU PPC64LE machine.
• The changes in the forked repository include following changes:
1. Added a job for PPC64LE in .github/workflows/cpp.yaml and .github/workflows/python.yaml, along with corresponding updates to docker-compose.yaml.
2. Created new Dockerfiles for C++ and Python: ci/docker/ppc64le-cpp.dockerfile and ci/docker/ppc64le-python.dockerfile.
3. Added build and test scripts for Python: ci/scripts/ppc64le_python_build.sh and ci/scripts/ppc64le_python_test.sh.

• We would like to upstream these changes to enable CI for ppc64le arch using GHA self-hosted runner.

**Fork Information:**
• Forked Repository: https://github.com/sandeepgupta12/arrow

**Request:**
• Support for PPC64LE: We are seeking support for the PPC64LE architecture for the apache/arrow project.
• Creation of OSU VM: To facilitate further testing and CI integration, we request the creation of an OSU VM configured for PPC64LE. Below are the details where you can create the OSU VM-
URL- https://osuosl.org/services/powerdev/request_hosting/
IBM Advocate- gerrit@us.ibm.com

**Details:**
The Open Source Lab (OSL) at Oregon State University (OSU), in partnership with IBM, provides access to IBM Power processor-based servers for developing and testing open source projects. The OSL offers following clusters:
**OpenStack (non-GPU) Cluster:**
• **Architecture**: Power little endian (LE) instances
• **Virtualization:** Kernel-based virtual machine (KVM)
• **Access:** Via Secure Shell (SSH) and/or through OpenStack's API and GUI interface
• **Capabilities:** Ideal for functional development and continuous integration (CI) work. It supports a managed Jenkins service hosted on the cluster or as a node incorporated into an external CI/CD pipeline.

**Additional Information:**
• We are prepared to provide any further details or assistance needed to support the PPC64LE architecture.
Please let us know if there are any specific requirements or steps needed to move forward with this request.

### Component(s)

C++, Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.