Importing data to hbase using sqoop
When I want to import the data to hive using sqoop I can specify --hive-home <dir> and sqoop will call that specified copy of hive installed on the machine where the script is being executed. But what about hbase? How does sqoop know which hbase instance/database I want the data to be imported on?
hadoop hive hbase sqoop
asked Nov 24 '18 at 17:39
ppax1ppax1
234
add a comment |
When I want to import the data to hive using sqoop I can specify --hive-home <dir> and sqoop will call that specified copy of hive installed on the machine where the script is being executed. But what about hbase? How does sqoop know which hbase instance/database I want the data to be imported on?
hadoop hive hbase sqoop
asked Nov 24 '18 at 17:39
ppax1ppax1
234
add a comment |
When I want to import the data to hive using sqoop I can specify --hive-home <dir> and sqoop will call that specified copy of hive installed on the machine where the script is being executed. But what about hbase? How does sqoop know which hbase instance/database I want the data to be imported on?
hadoop hive hbase sqoop
asked Nov 24 '18 at 17:39
ppax1ppax1
234
When I want to import the data to hive using sqoop I can specify --hive-home <dir> and sqoop will call that specified copy of hive installed on the machine where the script is being executed. But what about hbase? How does sqoop know which hbase instance/database I want the data to be imported on?
hadoop hive hbase sqoop
hadoop hive hbase sqoop
asked Nov 24 '18 at 17:39
ppax1ppax1
234
asked Nov 24 '18 at 17:39
ppax1ppax1
234
edited Nov 24 '18 at 17:59
ppax1
asked Nov 24 '18 at 17:39
ppax1ppax1
234
asked Nov 24 '18 at 17:39
ppax1ppax1
234
asked Nov 24 '18 at 17:39
ppax1ppax1
234
234
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Maybe the documentation helps?
By specifying
--hbase-table, you instruct Sqoop to import to a table in HBase rather than a directory in HDFS
Every example I see just shows that option along with column families, and whatnot, so I assume it depends on whatever variables that might be part of the sqoop-env.sh, like what the Hortonworks docs say
When you give the hive home directory, that's not telling it any database or table information either, but rather where Hive configuration files exist on the machine you're running Sqoop on. By default, that's set to be the environment variable $HIVE_HOME
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
Thanks for the answer. But that basically means that by specifing hbase/hive path in sqoop-env.sh it tells sqoop to which database Hbase/Hive I want the data to be imported(which database instance should be called by sqoop) no? Because I can have installed 3 copies or Hbase or Hive on the machine, each with different configuration, so I can choose to which concrete hbase/hive instance I want the data to be imported on right?
– ppax1
Nov 25 '18 at 15:21
By specifying which configuration folders, it will read in the zookeeper quorums and Hive metastore and HDFS settings, etc. for different clusters, yes
– cricket_007
Nov 25 '18 at 16:20
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53460783%2fimporting-data-to-hbase-using-sqoop%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Maybe the documentation helps?
By specifying
--hbase-table, you instruct Sqoop to import to a table in HBase rather than a directory in HDFS
Every example I see just shows that option along with column families, and whatnot, so I assume it depends on whatever variables that might be part of the sqoop-env.sh, like what the Hortonworks docs say
When you give the hive home directory, that's not telling it any database or table information either, but rather where Hive configuration files exist on the machine you're running Sqoop on. By default, that's set to be the environment variable $HIVE_HOME
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
Thanks for the answer. But that basically means that by specifing hbase/hive path in sqoop-env.sh it tells sqoop to which database Hbase/Hive I want the data to be imported(which database instance should be called by sqoop) no? Because I can have installed 3 copies or Hbase or Hive on the machine, each with different configuration, so I can choose to which concrete hbase/hive instance I want the data to be imported on right?
– ppax1
Nov 25 '18 at 15:21
By specifying which configuration folders, it will read in the zookeeper quorums and Hive metastore and HDFS settings, etc. for different clusters, yes
– cricket_007
Nov 25 '18 at 16:20
add a comment |
Maybe the documentation helps?
By specifying
--hbase-table, you instruct Sqoop to import to a table in HBase rather than a directory in HDFS
Every example I see just shows that option along with column families, and whatnot, so I assume it depends on whatever variables that might be part of the sqoop-env.sh, like what the Hortonworks docs say
When you give the hive home directory, that's not telling it any database or table information either, but rather where Hive configuration files exist on the machine you're running Sqoop on. By default, that's set to be the environment variable $HIVE_HOME
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
Thanks for the answer. But that basically means that by specifing hbase/hive path in sqoop-env.sh it tells sqoop to which database Hbase/Hive I want the data to be imported(which database instance should be called by sqoop) no? Because I can have installed 3 copies or Hbase or Hive on the machine, each with different configuration, so I can choose to which concrete hbase/hive instance I want the data to be imported on right?
– ppax1
Nov 25 '18 at 15:21
By specifying which configuration folders, it will read in the zookeeper quorums and Hive metastore and HDFS settings, etc. for different clusters, yes
– cricket_007
Nov 25 '18 at 16:20
add a comment |
Maybe the documentation helps?
By specifying
--hbase-table, you instruct Sqoop to import to a table in HBase rather than a directory in HDFS
Every example I see just shows that option along with column families, and whatnot, so I assume it depends on whatever variables that might be part of the sqoop-env.sh, like what the Hortonworks docs say
When you give the hive home directory, that's not telling it any database or table information either, but rather where Hive configuration files exist on the machine you're running Sqoop on. By default, that's set to be the environment variable $HIVE_HOME
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
Maybe the documentation helps?
By specifying
--hbase-table, you instruct Sqoop to import to a table in HBase rather than a directory in HDFS
Every example I see just shows that option along with column families, and whatnot, so I assume it depends on whatever variables that might be part of the sqoop-env.sh, like what the Hortonworks docs say
When you give the hive home directory, that's not telling it any database or table information either, but rather where Hive configuration files exist on the machine you're running Sqoop on. By default, that's set to be the environment variable $HIVE_HOME
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
edited Nov 25 '18 at 1:22
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
answered Nov 25 '18 at 1:05
cricket_007cricket_007
80.7k1142110
80.7k1142110
Thanks for the answer. But that basically means that by specifing hbase/hive path in sqoop-env.sh it tells sqoop to which database Hbase/Hive I want the data to be imported(which database instance should be called by sqoop) no? Because I can have installed 3 copies or Hbase or Hive on the machine, each with different configuration, so I can choose to which concrete hbase/hive instance I want the data to be imported on right?
– ppax1
Nov 25 '18 at 15:21
By specifying which configuration folders, it will read in the zookeeper quorums and Hive metastore and HDFS settings, etc. for different clusters, yes
– cricket_007
Nov 25 '18 at 16:20
add a comment |
Thanks for the answer. But that basically means that by specifing hbase/hive path in sqoop-env.sh it tells sqoop to which database Hbase/Hive I want the data to be imported(which database instance should be called by sqoop) no? Because I can have installed 3 copies or Hbase or Hive on the machine, each with different configuration, so I can choose to which concrete hbase/hive instance I want the data to be imported on right?
– ppax1
Nov 25 '18 at 15:21
By specifying which configuration folders, it will read in the zookeeper quorums and Hive metastore and HDFS settings, etc. for different clusters, yes
– cricket_007
Nov 25 '18 at 16:20
Thanks for the answer. But that basically means that by specifing hbase/hive path in sqoop-env.sh it tells sqoop to which database Hbase/Hive I want the data to be imported(which database instance should be called by sqoop) no? Because I can have installed 3 copies or Hbase or Hive on the machine, each with different configuration, so I can choose to which concrete hbase/hive instance I want the data to be imported on right?
– ppax1
Nov 25 '18 at 15:21
Thanks for the answer. But that basically means that by specifing hbase/hive path in sqoop-env.sh it tells sqoop to which database Hbase/Hive I want the data to be imported(which database instance should be called by sqoop) no? Because I can have installed 3 copies or Hbase or Hive on the machine, each with different configuration, so I can choose to which concrete hbase/hive instance I want the data to be imported on right?
– ppax1
Nov 25 '18 at 15:21
By specifying which configuration folders, it will read in the zookeeper quorums and Hive metastore and HDFS settings, etc. for different clusters, yes
– cricket_007
Nov 25 '18 at 16:20
By specifying which configuration folders, it will read in the zookeeper quorums and Hive metastore and HDFS settings, etc. for different clusters, yes
– cricket_007
Nov 25 '18 at 16:20
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53460783%2fimporting-data-to-hbase-using-sqoop%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
