open-input-bytes and open-output-bytes are slow

I was trying to submit an example of Racket to
https://github.com/kostya/benchmarks . I choose base64 because it was only necessary to call a few libs and call it a day. The problem is that base64-encode/base64-decode are slow.

I wrote a straightforward translation of the code to Racket (see below) and the run time was approximately 800 seconds in my slow netbook. (I guest it's about 200 seconds in a normal computer.) (Making the code idiomatic doesn't change the run time too much.)

For comparison, the benchmark that repeats 100 times the encoding in C takes 2 seconds and the benchmark in Python takes 7 seconds, so my guess is that the Racket version should take between 4 and 10 seconds, not 200 seconds.

The problem is that base64-encode uses open-input-bytes and open-output-bytes, and most of the time is used by them. I made a silly version of base64-encode (see below) to compare the time used by these ports, and removing them makes the code much faster.

Changing the input port of base64-encode to use directly the bytes is difficult without removing features, because ports are more general.

Changing the output port ofbase64-encode is possible, because I think that all the nice things of ports are not used here. But I think it would be better if it were possible to make the open-output-bytes faster in the simple use cases.

Typical result of the program (using TRIES = 1 instead of 100):

encode: 13333336, 6.54
in-port/out-port: 13333333, 4.466
in-port/out-bytes: 13333333, 2.066
in-bytes/out-bytes: 13333333, 0.8

Program:

  #lang racket/base
  (require net/base64)

  (define STR-SIZE 10000000)
  (define TRIES 1)#;(00)    ;   <-- repeat it only 1 time instead of 100

  (define str1 (make-bytes STR-SIZE (char->integer #\a)))
  (define str2 #"")
  (define t 0)

  (set! t (current-milliseconds))
  (define s1 (for/sum ([_ (in-range TRIES)])
                 (set! str2 (base64-encode str1 #""))
                 (bytes-length str2)))
  (printf "encode: ~a, ~a\n" s1 (/ (- (current-milliseconds) t) 1000.))

  (define (my-base64-encode/in-port/out-port b _)
    (define len (quotient (* (bytes-length b) 8) 6))
    (define in (open-input-bytes b))
    (define out (open-output-bytes))
    (for ([i (in-range (bytes-length b))])
      (read-byte in))
    (for ([i (in-range len)])
      (write-char #\X out))
    (get-output-bytes out))

  (define (my-base64-encode/in-port/out-bytes b _)
    (define len (quotient (* (bytes-length b) 8) 6))
    (define in (open-input-bytes b))
    (define out/byte (make-bytes len))
    (for ([i (in-range (bytes-length b))])
      (read-byte in))
    (for ([i (in-range len)])
      (bytes-set! out/byte i (char->integer #\X)))
    out/byte)

  (define (my-base64-encode/in-bytes/out-bytes b _)
    (define len (quotient (* (bytes-length b) 8) 6)) 
    (define out/byte (make-bytes len))
    (for ([i (in-range (bytes-length b))])
      (bytes-ref b i))
    (for ([i (in-range len)])
      (bytes-set! out/byte i (char->integer #\X)))
    out/byte)


  (set! t (current-milliseconds))
  (define b/0 (for/sum ([_ (in-range TRIES)])
                 (bytes-length (my-base64-encode/in-port/out-port str1 #""))))
  (printf "in-port/out-port: ~a, ~a\n" b/0 (/ (- (current-milliseconds) t) 1000.))

  (set! t (current-milliseconds))
  (define b/1 (for/sum ([_ (in-range TRIES)])
                 (bytes-length (my-base64-encode/in-port/out-bytes str1 #""))))
  (printf "in-port/out-bytes: ~a, ~a\n" b/1 (/ (- (current-milliseconds) t) 1000.))

  (set! t (current-milliseconds))
  (define b/2 (for*/sum ([_ (in-range TRIES)])
                 (bytes-length (my-base64-encode/in-bytes/out-bytes str1 #""))))
  (printf "in-bytes/out-bytes: ~a, ~a\n" b/2 (/ (- (current-milliseconds) t) 1000.))

Start by reproducing the benchmark program in the issue and compare base64-encode/base64-decode with the custom port and byte-array variants. Trace open-input-bytes and open-output-bytes to identify the overhead in these simple use cases. Done means the base64 operations retain their port-based behavior while substantially reducing the reported runtime.