Support for Spark version >= 3.1 and AssertionError query
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
Hi!
I wanted to confirm if XGBoost supports Spark version 3.1.2. I have been trying to run XGBoost on the latest version of Apache Spark on a dataset > 3TB on a 28 node cluster.
Also, I have been getting the following error and haven't been able to figure out what might be causing this.
```
21/10/10 10:50:17 INFO XGBoostSpark: starting training with timeout set as 1800000 ms for waiting for resources
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: Exception in thread Thread-1:
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: Traceback (most recent call last):
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: self.run()
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: File "/usr/lib/python2.7/threading.py", line 754, in run
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: self.__target(*self.__args, **self.__kwargs)
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: File "/tmp/tracker2488390078882395804.py", line 324, in run
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: self.accept_slaves(nslave)
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: File "/tmp/tracker2488390078882395804.py", line 268, in accept_slaves
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: s = SlaveEntry(fd, s_addr)
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: File "/tmp/tracker2488390078882395804.py", line 64, in __init__
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: assert magic == kMagic, 'invalid magic number=%d from %s' % (magic, self.host)
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: AssertionError: invalid magic number=542393671 from 143.198.38.116
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger:
21/10/10 13:44:27 INFO RabitTracker$TrackerProcessLogger: Tracker Process ends with exit code 0
```
I look forward to hearing back from you. Thank you in advance for your help and time.
Contributor guide
No contributing guide indexed for this repository
Research direction
No source file, test, or entry point is identified. Start by locating the Spark integration and RabitTracker implementation, then reproduce the reported run with Spark 3.1.2 and the supplied AssertionError details. Done means establishing whether that Spark version is supported and documenting or fixing the cause of the invalid magic number.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, spark
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100