IQSS / IQSS/dataverse

RData file not showing up as RData file, but as GZIP archive; doesn't ingest

Open
#12,558 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: Bug
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

What steps does it take to reproduce the issue?

Note, this issue was originally about RDS files, but that turned out to be inaccurate -- the files in question are actually all RData files, they just mistakenly have RDS as the file extension

We have 4 files that are RDS RData files. 3 convert successfully, the 4th does not and remains as a gzip archive file.

  • When does this issue occur?
    Upon upload

  • Which page(s) does it occurs on?
    File upload page

  • What happens?
    4 files that are supposed to be RDS RData files (compressed) are uploaded, but only three are ingested. The 4th file remains as a zip archive. I'm not sure why - if it's the file format or something going on in dataverse.

In the the log an exception is thrown:

[2026-07-23T18:16:23.359+0000] [Payara 7.2026.2] [SEVERE] [] [] [tid: _ThreadID=103 _ThreadName=http-thread-pool::jk-connector(2)] [timeMillis: 1784830583359] [levelValue: 1000] [[
  java.io.EOFException: Unexpected end of ZLIB input stream
        at java.base/java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:266)
        at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:175)
        at java.base/java.util.zip.GZIPInputStream.read(GZIPInputStream.java:149)
        at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:123)
        at edu.harvard.iq.dataverse.ingest.IngestableDataChecker.testRDAformat(IngestableDataChecker.java:593)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at edu.harvard.iq.dataverse.ingest.IngestableDataChecker.detectTabularDataFormat(IngestableDataChecker.java:669)
        at edu.harvard.iq.dataverse.ingest.IngestableDataChecker.detectTabularDataFormat(IngestableDataChecker.java:639)
        at edu.harvard.iq.dataverse.util.FileUtil.determineFileType(FileUtil.java:458)
        at edu.harvard.iq.dataverse.engine.command.impl.CreateNewDataFilesCommand.execute(CreateNewDataFilesCommand.java:218)
        at edu.harvard.iq.dataverse.engine.command.impl.CreateNewDataFilesCommand.execute(CreateNewDataFilesCommand.java:69)
        at edu.harvard.iq.dataverse.EjbDataverseEngineInner.submit(EjbDataverseEngineInner.java:36)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.glassfish.exousia.AuthorizationService.lambda$invokeBeanMethod$2(AuthorizationService.java:498)
        at org.glassfish.exousia.AuthorizationService.runInScope(AuthorizationService.java:582)
        at org.glassfish.exousia.AuthorizationService.invokeBeanMethod(AuthorizationService.java:498)
        at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:272)
        at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4716)
        at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:654)
        at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:832)
        at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:604)
        at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
        at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:868)
        at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:831)
        at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:604)
        at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:78)
        at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:868)
        at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:831)
        at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
        at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4689)
        at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4678)
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
        at jdk.proxy69/jdk.proxy69.$Proxy353.submit(Unknown Source)
        at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngineInner__Intf____Bean__.submit(Unknown Source)
        at edu.harvard.iq.dataverse.EjbDataverseEngine.submit(EjbDataverseEngine.java:300)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.glassfish.exousia.AuthorizationService.lambda$invokeBeanMethod$2(AuthorizationService.java:498)
        at org.glassfish.exousia.AuthorizationService.runInScope(AuthorizationService.java:582)
        at org.glassfish.exousia.AuthorizationService.invokeBeanMethod(AuthorizationService.java:498)
        at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:272)
        at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4716)
        at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:654)
        at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:832)
        at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:604)
        at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
        at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:868)
        at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:831)
        at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:604)
        at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:78)
        at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:868)
        at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:831)
        at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
        at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4689)
        at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4678)
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
        at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
        at jdk.proxy69/jdk.proxy69.$Proxy351.submit(Unknown Source)
        at edu.harvard.iq.dataverse.__EJB31_Generated__EjbDataverseEngine__Intf____Bean__.submit(Unknown Source)
        at edu.harvard.iq.dataverse.EditDatafilesPage.handleFileUpload(EditDatafilesPage.java:2041)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.glassfish.expressly.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:186)
        at org.glassfish.expressly.parser.AstValue.invoke(AstValue.java:253)
        at org.glassfish.expressly.MethodExpressionImpl.invoke(MethodExpressionImpl.java:248)
        at org.jboss.weld.module.web.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
        at org.jboss.weld.module.web.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
        at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:70)
        at org.primefaces.component.fileupload.FileUpload.broadcast(FileUpload.java:74)
        at jakarta.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:834)
        at jakarta.faces.component.UIViewRoot.processValidators(UIViewRoot.java:1271)
        at com.sun.faces.lifecycle.ProcessValidationsPhase.execute(ProcessValidationsPhase.java:49)
        at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:72)
        at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:131)
        at jakarta.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:689)
        at jakarta.faces.webapp.FacesServlet.service(FacesServlet.java:449)
        at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1554)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
        at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:83)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
        at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
        at edu.harvard.iq.dataverse.filter.CorsFilter.doFilter(CorsFilter.java:89)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:166)
        at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
        at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
        at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:366)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
        at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
        at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
        at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:174)
        at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:153)
        at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:196)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:178)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:118)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:96)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
        at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
        at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:101)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
        at java.base/java.lang.Thread.run(Thread.java:1583)
]]

Code line it failed on: https://github.com/IQSS/dataverse/blob/b485249231feb43c70459884089b9af7b4c3beea/src/main/java/edu/harvard/iq/dataverse/ingest/IngestableDataChecker.java#L593

This makes me think the file format may not have been complete? Or should this be relaxed? RStudio and R can read this file just fine.

Zulip Discussion: #troubleshooting > RDS looks like gzip file

File:

College_Debt_Data.zip

  • To whom does it occur (all users, curators, superusers)?
    Only these four RDS files in this record: https://doi.org/10.60600/YU/R7ILPI

  • What did you expect to happen?
    I expected all 4 files to get ingested to tab files.

Which version of Dataverse are you using?
6.10.1

Any related open or closed issues to this bug report?

Not that I'm aware of.

Screenshots:

The file that failed ingest:
Image

The three that succeeded in ingest:
Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in IngestableDataChecker.java, especially testRDAformat at line 593 and detectTabularDataFormat, then follow the upload path through FileUtil.determineFileType. Reproduce the issue with the four RData files described in the report and compare the successful and failing files. Done means the affected file is correctly recognized as RData and ingested without the reported EOF failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.