linkedin/spark-tfrecord

View on GitHub

Read and write Tensorflow TFRecord data from Apache Spark.

Stars: 300
Forks: 57
Open beginner issues: 0
Indexed issues: 17

Dominant language: Scala
License: BSD-2-Clause
Last GitHub push: Apr 22, 2024
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

17 open issues indexed

Any plans for Spark 4? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/spark-tfrecord#74 · 1 comment · Opened Aug 27, 2026
NullPointerException at org.apache.hadoop.conf.Configuration Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

linkedin/spark-tfrecord#69 · 1 comment · Opened Dec 19, 2023
java.lang.ClassNotFoundException: Failed to find data source: tfrecord. Please find packages at http://spark.apache.org/third-party-projects.html Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

linkedin/spark-tfrecord#68 · 3 comments · Opened Jul 18, 2023
Spark 3.3 support? Open

Difficulty 2/5 1-3 hours Newbie friendliness 30/100

linkedin/spark-tfrecord#58 · 4 comments · Opened Dec 25, 2022
ArrayType(ArrayType(DoubleType,true),true) Not Supported When Writing Dataframe to TFRecords Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/spark-tfrecord#56 · 2 comments · Opened Jul 29, 2022
Do we support spark dataframe straightly convert to tensorflow-java Tensor like TFloat32 TFloat64 or Operand[T <: TNumber] Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/spark-tfrecord#53 · 1 comment · Opened Jun 10, 2022
Support for Spark 3.2.0+ with scala 2.13 Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

linkedin/spark-tfrecord#49 · 2 comments · 1 reaction · Opened May 3, 2022
spark 读取 tfrecord文件，解析tf.train.Feature(bytes_list=tf.train.BytesList(np.array(feature, dtype=np.float32).tobytes())) 文件字节码异常 Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

linkedin/spark-tfrecord#48 · 2 comments · Opened Apr 26, 2022
TFRecords File is too big! 10X the size of parquet Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

linkedin/spark-tfrecord#47 · 2 comments · Opened Mar 9, 2022
tfrecord write results in no data but no error Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

linkedin/spark-tfrecord#46 · 2 comments · Opened Feb 20, 2022
Error: java.lang.ClassCastException: com.linkedin.spark.shaded.org.tensorflow.example.FeatureList cannot be cast to com.linkedin.spark.shaded.org.tensorflow.example.Feature Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

linkedin/spark-tfrecord#44 · 3 comments · Opened Jan 21, 2022
write compression none invalid Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

linkedin/spark-tfrecord#43 · 1 comment · Opened Jan 18, 2022
Dataframe can not convert to tfrecord Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

linkedin/spark-tfrecord#39 · 3 comments · Opened Dec 23, 2021
encountered a FileAlreadyExistsException when trying to save array of arrays of arrays (3d array) Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

linkedin/spark-tfrecord#35 · 1 comment · Opened Sep 29, 2021
Consume TensorFlowInferSchema from Java Spark 2.4.0 Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

linkedin/spark-tfrecord#34 · 2 comments · Opened Sep 4, 2021
java.lang.RuntimeException: unsupported type ... Open

enhancement help wanted

Difficulty 3/5 1-2 days Newbie friendliness 35/100

linkedin/spark-tfrecord#22 · 6 comments · Opened Jan 7, 2021
Support for ShortType, ByteType Open

enhancement help wanted

Difficulty 3/5 1-2 days Newbie friendliness 45/100

linkedin/spark-tfrecord#14 · 4 comments · Opened Sep 4, 2020

linkedin/spark-tfrecord

Get new issues in your inbox