parallel performance in threads and tasks
@DanGrayson is already working on this.
Since Jun 17, 2014.
- Dominant language
- Macaulay2
- Stars
- 435
- Forks
- 297
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 11
Description
Running tasks that allocate memory in parallel results in almost no improvement of performance, compared to the time sequential execution would take. (Interestingly, compiling in debug mode causes performance to deteriorate by a factor 10-20, perhaps due to cache hits.) The following code demonstrates the problem.
```
notify = true
zeroTime = currentTime()
spin = value Core#"private dictionary"#"spin"
f = () -> {currentTime()-zeroTime, (for i to 14000000 do () ; currentTime()-zeroTime)} -- lots of garbage made
time f () -- 4 seconds on habanero
g = () -> {currentTime()-zeroTime, (spin 5000 ; currentTime()-zeroTime)} -- no garbage made
time g () -- 4 seconds on habanero
h = (n,g) -> (
s := for i from 1 to n list schedule g;
while ( sleep 1; not all(s,isReady)) do ();
netList prepend({"serial number", "start time", "end time"}, for t in s list prepend(serialNumber t, taskResult t)))
allowableThreads = 2
h(5,f)
h(5,g)
allowableThreads = 6
h(5,f)
h(5,g)
```
The relevant portion of the results is this:
```
+-------------+----------+--------+
oo11 = |serial number|start time|end time|
+-------------+----------+--------+
|1 |9 |14 |
+-------------+----------+--------+
|2 |14 |18 |
+-------------+----------+--------+
|3 |18 |23 |
+-------------+----------+--------+
|4 |23 |27 |
+-------------+----------+--------+
|5 |27 |32 |
+-------------+----------+--------+
+-------------+----------+--------+
oo14 = |serial number|start time|end time|
+-------------+----------+--------+
|11 |57 |77 |
+-------------+----------+--------+
|12 |57 |78 |
+-------------+----------+--------+
|13 |57 |78 |
+-------------+----------+--------+
|14 |57 |79 |
+-------------+----------+--------+
|15 |57 |79 |
+-------------+----------+--------+
```
The first batch shows sequential execution, and the second shows parallel execution.
The other two batches, not shown here, use the undocumented core function "spin", which
consumes CPU time without allocating memory. There the 5 processes run at the same time and complete their work in 5 seconds instead of 22.
The garbage collection algorithm of Boehm and Demers used in libgc is described in http://www.hboehm.info/gc/gcdescr.html . It includes links to useful papers, such as http://www.hboehm.info/gc/papers/pldi91.ps.Z .
Another (minor) problem is that the statement `allowableThreads = 2` seems to result in 6 threads getting ignored and not used later on when the number is increased.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.