Add Apache Ignite IO
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 204
Description
Hi all,
{color:#14892c}I tried to connect Apache Ignite(In-memory) via the beam's sdk:org.apache.beam.sdk.io.jdbc.JdbcIO
Here, i am not sure if the JdbcIO sdk only is provided for some specific Database: MySQL(disk), postgreSQL(disk)?{color}
my java test code is as follows:
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.io.jdbc.JdbcIO;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.Create;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PCollection;
public class BeamtoJDBC {
public static void main(String[] args) {
Integer[] value=new Integer[] {1,2,3,4,5};
List\> dataList = new ArrayList<\>();
int n=value.length;
int count=0;
for (int i=0; i\> data=p.apply("create data with time",Create.of(dataList));
data.apply(JdbcIO.\>write()
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration
.create("org.apache.ignite.IgniteJdbcDriver", "jdbc:ignite://localhost:11211/")
)
.withPreparedStatementSetter(new JdbcIO.PreparedStatementSetter\>() {
public void setParameters(KV element, PreparedStatement query)
throws SQLException {
query.setInt(1, element.getKey());
query.setInt(2, element.getValue());
}
})
);
p.run();
}
}
{color:#d04437}my error message is:
" InvocationTargetException: org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException: Cannot create PoolableConnectionFactory
(Failed to establish connection.): Failed to get future result due to waiting timed out. "{color}
{color:#14892c}I would like to know whether the connection between beam and ignite is feasible or not?{color}
Thanks
Rick
Imported from Jira [BEAM-3073](https://issues.apache.org/jira/browse/BEAM-3073). Original Jira may contain additional context.
Reported by: Ricklin.
Contributor guide
Research direction
Start with the provided Java example and the JdbcIO DataSourceConfiguration connection setup, then investigate the reported connection timeout with Apache Ignite's JDBC driver and URL. Done would require a clear project-supported path for connecting Beam's JdbcIO to Ignite, but the issue does not name Beam files or tests to update.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100