PersistentEntityResource low serialization performance [DATAREST-1233]

Mikhail Kadan opened DATAREST-1233 and commented

I'm experiencing a very low serialization performance for a custom @RepositoryRestController method returning PagedResources<PersistentEntityResource>, e.g. I'm getting 15s serialization times for 2.5Mb JSON data instead of 0.5s after the workaround I made (more on it later).

Consider this:

@Entity
public class Content {

    @OneToMany
    private Set<ContentMapping> contentMappings = new HashSet<>();

    // ...
}

@RepositoryRestController
public class MyController {

    private final ContentService contentService;
    private final PagedResourcesAssembler pagedResourcesAssembler;

    public ContentRestController(
            ContentService contentService,
            PagedResourcesAssembler pagedResourcesAssembler) {
        this.contentService = contentService;
        this.pagedResourcesAssembler = pagedResourcesAssembler;
    }

    @RequestMapping(value = "/findContent", method = RequestMethod.GET)
    @ResponseBody
    public PagedResources<PersistentEntityResource> findContent(PersistentEntityResourceAssembler resourceAssembler) {
        Page<Content> page = contentService.getContent();

        @SuppressWarnings("unchecked")
        PagedResources<PersistentEntityResource> pagedResources = pagedResourcesAssembler.toResource(page, resourceAssembler);
        return pagedResources;
    }
}

A call to /findContent takes 15s to fully respond (while data start streaming immideately after it is made, so this is like 15s serialization time).

After profiling I found out that the cause of the problem are persistent collection properties on Content. During serialization of a Content a new transaction is opened for every access attempt to the contentMappings collection, even when contentMappings was properly fetched before serialization inside contentService.getContent() call.

Opening an explicit transaction on a controller method did not help (cause it was closed after the method exits and before serialization occurs), but I was able to work around this behaviour using HttpServletResponse and manually serializing the response:

@RepositoryRestController
public class MyController {

    private final ContentService contentService;
    private final PagedResourcesAssembler pagedResourcesAssembler;
    private final List<HttpMessageConverter> messageConverters;

    public ContentRestController(
            ContentService contentService,
            PagedResourcesAssembler pagedResourcesAssembler,
            List<HttpMessageConverter> messageConverters) {
        this.contentService = contentService;
        this.pagedResourcesAssembler = pagedResourcesAssembler;
        this.messageConverters = messageConverters;
    }

    @RequestMapping(value = "/findContent", method = RequestMethod.GET)
    @ResponseBody
    @Transactional(readOnly = true)
    public void findContent(PersistentEntityResourceAssembler resourceAssembler, HttpServletResponse response) throws IOException {
        Page<Content> page = contentService.getContent();

        @SuppressWarnings("unchecked")
        PagedResources<PersistentEntityResource> pagedResources = pagedResourcesAssembler.toResource(page, resourceAssembler);

        // manual response serialization
        
        MediaType mediaType = MediaType.valueOf("application/hal+json");

        ResponseEntity<String> responseEntity = messageConverters.stream()
                .filter(messageConverter -> messageConverter.canWrite(pagedResources.getClass(), mediaType))
                .findFirst()
                .map(messageConverter -> {
                    HttpOutputMessage outputMessage = new HttpOutputMessage() {

                        private final OutputStream outputStream = new ByteArrayOutputStream();
                        private final HttpHeaders httpHeaders = new HttpHeaders();

                        @Override
                        public OutputStream getBody() throws IOException {
                            return outputStream;
                        }

                        @Override
                        public HttpHeaders getHeaders() {
                            return httpHeaders;
                        }
                    };
                    try {
                        messageConverter.write(pagedResources, mediaType, outputMessage);
                        return ResponseEntity.ok()
                                .headers(outputMessage.getHeaders())
                                .body(new String(((ByteArrayOutputStream) outputMessage.getBody()).toByteArray(), StandardCharsets.UTF_8));

                    } catch (IOException e) {
                        throw new IllegalStateException("Failed to convert output to " + mediaType.toString());
                    }
                })
                .orElseThrow(() -> new IllegalStateException("Failed to convert output to " + mediaType.toString()));

        response.setContentType(mediaType.toString());
        response.setCharacterEncoding(StandardCharsets.UTF_8.name());
        response.getWriter().write(responseEntity.getBody());
        responseEntity.getHeaders().entrySet().stream()
                .flatMap(entry -> entry.getValue().stream()
                        .map(value -> Tuples.of(entry.getKey(), value)))
                .forEach(t -> response.addHeader(t.getT1(), t.getT2()));
        response.flushBuffer();
    }
}

This way response is received in 0.5s instead of 15s.

Problems I see with this workaround are e.g. completely ignoring RequestBodyAdvice / ResponseBodyAdvice processing, and the need to manually work with HttpServletResponse and HttpMessageConverters, effectively duplicating Spring code

Reference URL: https://stackoverflow.com/questions/49759434/persistententityresource-low-serialization-performance