Sqoop functionality has been removed from DSE
0
I am new to cassandra. Here I am tring to transfer whole my MYSQL database to cassandra using sqoop. But after all setup, when i execute following command.
bin/dse sqoop import-all-tables -m 1 --connect jdbc:mysql://127.0.0.1:3306/ABCDatabase --username root --password root --cassandra-thrift-host localhost --cassandra-create-schema --direct
I have received following error.
Sqoop functionality has been removed from DSE.
It said that sqoop functionality is removed from datastax. can you please if it removed then is there any other way to do that?
Thanks
mysql cassandra sqoop datastax datastax-enterprise
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
add a comment |
0
I am new to cassandra. Here I am tring to transfer whole my MYSQL database to cassandra using sqoop. But after all setup, when i execute following command.
bin/dse sqoop import-all-tables -m 1 --connect jdbc:mysql://127.0.0.1:3306/ABCDatabase --username root --password root --cassandra-thrift-host localhost --cassandra-create-schema --direct
I have received following error.
Sqoop functionality has been removed from DSE.
It said that sqoop functionality is removed from datastax. can you please if it removed then is there any other way to do that?
Thanks
mysql cassandra sqoop datastax datastax-enterprise
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
add a comment |
0
0
0
I am new to cassandra. Here I am tring to transfer whole my MYSQL database to cassandra using sqoop. But after all setup, when i execute following command.
bin/dse sqoop import-all-tables -m 1 --connect jdbc:mysql://127.0.0.1:3306/ABCDatabase --username root --password root --cassandra-thrift-host localhost --cassandra-create-schema --direct
I have received following error.
Sqoop functionality has been removed from DSE.
It said that sqoop functionality is removed from datastax. can you please if it removed then is there any other way to do that?
Thanks
mysql cassandra sqoop datastax datastax-enterprise
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
I am new to cassandra. Here I am tring to transfer whole my MYSQL database to cassandra using sqoop. But after all setup, when i execute following command.
bin/dse sqoop import-all-tables -m 1 --connect jdbc:mysql://127.0.0.1:3306/ABCDatabase --username root --password root --cassandra-thrift-host localhost --cassandra-create-schema --direct
I have received following error.
Sqoop functionality has been removed from DSE.
It said that sqoop functionality is removed from datastax. can you please if it removed then is there any other way to do that?
Thanks
mysql cassandra sqoop datastax datastax-enterprise
mysql cassandra sqoop datastax datastax-enterprise
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
asked Nov 27 '18 at 14:35
Jatin PatelJatin Patel
3,36221436
3,36221436
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
2
You can use Spark to transfer data - it should be easy, something like:
val table = spark.read.jdbc(jdbcUrl, "table", connectionProperties)
table.write.format("org.apache.spark.sql.cassandra").options(
Map("table" -> "TBL", "keyspace" -> "KS")).save()
Examples of jdbc URLs, options, etc. are described in Databrick's documentation as they could be different for different databases.
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
I am new to spark and Java but little bit know python. can you please provide me blog links which help me to execute your command with help of python? Thanks
– Jatin Patel
Dec 14 '18 at 9:05
Just read first several pages of Spark SQL documentation - it has examples for python. Spark Cassandra connector documentation also has a section on python
– Alex Ott
Dec 14 '18 at 11:36
as per you documentation url, I have successfully connected to MySql and cassandra with help of spark. but now everytime i need to transfer only newly added records. how to do that? is there any api support available?
– Jatin Patel
Jan 7 at 11:44
1
That's depends on schema of your database - just use query that will select/filter only new records...
– Alex Ott
Jan 7 at 11:58
1
Look to Confluent's blog - they had several blog posts about that topic
– Alex Ott
Jan 8 at 7:02
|
show 3 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53502035%2fsqoop-functionality-has-been-removed-from-dse%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
2
You can use Spark to transfer data - it should be easy, something like:
val table = spark.read.jdbc(jdbcUrl, "table", connectionProperties)
table.write.format("org.apache.spark.sql.cassandra").options(
Map("table" -> "TBL", "keyspace" -> "KS")).save()
Examples of jdbc URLs, options, etc. are described in Databrick's documentation as they could be different for different databases.
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
I am new to spark and Java but little bit know python. can you please provide me blog links which help me to execute your command with help of python? Thanks
– Jatin Patel
Dec 14 '18 at 9:05
Just read first several pages of Spark SQL documentation - it has examples for python. Spark Cassandra connector documentation also has a section on python
– Alex Ott
Dec 14 '18 at 11:36
as per you documentation url, I have successfully connected to MySql and cassandra with help of spark. but now everytime i need to transfer only newly added records. how to do that? is there any api support available?
– Jatin Patel
Jan 7 at 11:44
1
That's depends on schema of your database - just use query that will select/filter only new records...
– Alex Ott
Jan 7 at 11:58
1
Look to Confluent's blog - they had several blog posts about that topic
– Alex Ott
Jan 8 at 7:02
|
show 3 more comments
2
You can use Spark to transfer data - it should be easy, something like:
val table = spark.read.jdbc(jdbcUrl, "table", connectionProperties)
table.write.format("org.apache.spark.sql.cassandra").options(
Map("table" -> "TBL", "keyspace" -> "KS")).save()
Examples of jdbc URLs, options, etc. are described in Databrick's documentation as they could be different for different databases.
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
I am new to spark and Java but little bit know python. can you please provide me blog links which help me to execute your command with help of python? Thanks
– Jatin Patel
Dec 14 '18 at 9:05
Just read first several pages of Spark SQL documentation - it has examples for python. Spark Cassandra connector documentation also has a section on python
– Alex Ott
Dec 14 '18 at 11:36
as per you documentation url, I have successfully connected to MySql and cassandra with help of spark. but now everytime i need to transfer only newly added records. how to do that? is there any api support available?
– Jatin Patel
Jan 7 at 11:44
1
That's depends on schema of your database - just use query that will select/filter only new records...
– Alex Ott
Jan 7 at 11:58
1
Look to Confluent's blog - they had several blog posts about that topic
– Alex Ott
Jan 8 at 7:02
|
show 3 more comments
2
2
2
You can use Spark to transfer data - it should be easy, something like:
val table = spark.read.jdbc(jdbcUrl, "table", connectionProperties)
table.write.format("org.apache.spark.sql.cassandra").options(
Map("table" -> "TBL", "keyspace" -> "KS")).save()
Examples of jdbc URLs, options, etc. are described in Databrick's documentation as they could be different for different databases.
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
You can use Spark to transfer data - it should be easy, something like:
val table = spark.read.jdbc(jdbcUrl, "table", connectionProperties)
table.write.format("org.apache.spark.sql.cassandra").options(
Map("table" -> "TBL", "keyspace" -> "KS")).save()
Examples of jdbc URLs, options, etc. are described in Databrick's documentation as they could be different for different databases.
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
answered Nov 27 '18 at 14:55
Alex OttAlex Ott
28.4k35273
28.4k35273
I am new to spark and Java but little bit know python. can you please provide me blog links which help me to execute your command with help of python? Thanks
– Jatin Patel
Dec 14 '18 at 9:05
Just read first several pages of Spark SQL documentation - it has examples for python. Spark Cassandra connector documentation also has a section on python
– Alex Ott
Dec 14 '18 at 11:36
as per you documentation url, I have successfully connected to MySql and cassandra with help of spark. but now everytime i need to transfer only newly added records. how to do that? is there any api support available?
– Jatin Patel
Jan 7 at 11:44
1
That's depends on schema of your database - just use query that will select/filter only new records...
– Alex Ott
Jan 7 at 11:58
1
Look to Confluent's blog - they had several blog posts about that topic
– Alex Ott
Jan 8 at 7:02
|
show 3 more comments
I am new to spark and Java but little bit know python. can you please provide me blog links which help me to execute your command with help of python? Thanks
– Jatin Patel
Dec 14 '18 at 9:05
Just read first several pages of Spark SQL documentation - it has examples for python. Spark Cassandra connector documentation also has a section on python
– Alex Ott
Dec 14 '18 at 11:36
as per you documentation url, I have successfully connected to MySql and cassandra with help of spark. but now everytime i need to transfer only newly added records. how to do that? is there any api support available?
– Jatin Patel
Jan 7 at 11:44
1
That's depends on schema of your database - just use query that will select/filter only new records...
– Alex Ott
Jan 7 at 11:58
1
Look to Confluent's blog - they had several blog posts about that topic
– Alex Ott
Jan 8 at 7:02
I am new to spark and Java but little bit know python. can you please provide me blog links which help me to execute your command with help of python? Thanks
– Jatin Patel
Dec 14 '18 at 9:05
I am new to spark and Java but little bit know python. can you please provide me blog links which help me to execute your command with help of python? Thanks
– Jatin Patel
Dec 14 '18 at 9:05
Just read first several pages of Spark SQL documentation - it has examples for python. Spark Cassandra connector documentation also has a section on python
– Alex Ott
Dec 14 '18 at 11:36
Just read first several pages of Spark SQL documentation - it has examples for python. Spark Cassandra connector documentation also has a section on python
– Alex Ott
Dec 14 '18 at 11:36
as per you documentation url, I have successfully connected to MySql and cassandra with help of spark. but now everytime i need to transfer only newly added records. how to do that? is there any api support available?
– Jatin Patel
Jan 7 at 11:44
as per you documentation url, I have successfully connected to MySql and cassandra with help of spark. but now everytime i need to transfer only newly added records. how to do that? is there any api support available?
– Jatin Patel
Jan 7 at 11:44
1
1
That's depends on schema of your database - just use query that will select/filter only new records...
– Alex Ott
Jan 7 at 11:58
That's depends on schema of your database - just use query that will select/filter only new records...
– Alex Ott
Jan 7 at 11:58
1
1
Look to Confluent's blog - they had several blog posts about that topic
– Alex Ott
Jan 8 at 7:02
Look to Confluent's blog - they had several blog posts about that topic
– Alex Ott
Jan 8 at 7:02
|
show 3 more comments
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53502035%2fsqoop-functionality-has-been-removed-from-dse%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
