NLP ML How to know the weight of words used in text classifier?
I am building a tweet classifier where I try to train different ML models to classify tweets from 2 different tweeter accounts. So far I have train Logistic Regression model, K Neighbors Classifier and decision tree classifier.
Is there a way to know what words in the tweets those classifiers used to predict the account? like the weight of words in the classification process?? I am open to train new classifiers that can do that as well.
Already did some ngram analysis on the tweets like word frequency.
thanks in advance!
python machine-learning nlp nltk text-classification
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
add a comment |
I am building a tweet classifier where I try to train different ML models to classify tweets from 2 different tweeter accounts. So far I have train Logistic Regression model, K Neighbors Classifier and decision tree classifier.
Is there a way to know what words in the tweets those classifiers used to predict the account? like the weight of words in the classification process?? I am open to train new classifiers that can do that as well.
Already did some ngram analysis on the tweets like word frequency.
thanks in advance!
python machine-learning nlp nltk text-classification
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
I think this question is too broad for StackOverflow. Try to use CrossValidated StackExchange.
– Tomáš Přinda
Nov 24 '18 at 1:16
@Tomáš, I suppose it could be read as "how do I compute my own classifier models". But "how do I inspect my trained nltk classifier" is a reasonable question (and would have been even more so if it had named the specific classifier(s) of interest...).
– alexis
Nov 24 '18 at 21:34
add a comment |
I am building a tweet classifier where I try to train different ML models to classify tweets from 2 different tweeter accounts. So far I have train Logistic Regression model, K Neighbors Classifier and decision tree classifier.
Is there a way to know what words in the tweets those classifiers used to predict the account? like the weight of words in the classification process?? I am open to train new classifiers that can do that as well.
Already did some ngram analysis on the tweets like word frequency.
thanks in advance!
python machine-learning nlp nltk text-classification
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
I am building a tweet classifier where I try to train different ML models to classify tweets from 2 different tweeter accounts. So far I have train Logistic Regression model, K Neighbors Classifier and decision tree classifier.
Is there a way to know what words in the tweets those classifiers used to predict the account? like the weight of words in the classification process?? I am open to train new classifiers that can do that as well.
Already did some ngram analysis on the tweets like word frequency.
thanks in advance!
python machine-learning nlp nltk text-classification
python machine-learning nlp nltk text-classification
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
asked Nov 24 '18 at 0:52
mackenzie Xiemackenzie Xie
1
1
I think this question is too broad for StackOverflow. Try to use CrossValidated StackExchange.
– Tomáš Přinda
Nov 24 '18 at 1:16
@Tomáš, I suppose it could be read as "how do I compute my own classifier models". But "how do I inspect my trained nltk classifier" is a reasonable question (and would have been even more so if it had named the specific classifier(s) of interest...).
– alexis
Nov 24 '18 at 21:34
add a comment |
I think this question is too broad for StackOverflow. Try to use CrossValidated StackExchange.
– Tomáš Přinda
Nov 24 '18 at 1:16
@Tomáš, I suppose it could be read as "how do I compute my own classifier models". But "how do I inspect my trained nltk classifier" is a reasonable question (and would have been even more so if it had named the specific classifier(s) of interest...).
– alexis
Nov 24 '18 at 21:34
I think this question is too broad for StackOverflow. Try to use CrossValidated StackExchange.
– Tomáš Přinda
Nov 24 '18 at 1:16
I think this question is too broad for StackOverflow. Try to use CrossValidated StackExchange.
– Tomáš Přinda
Nov 24 '18 at 1:16
@Tomáš, I suppose it could be read as "how do I compute my own classifier models". But "how do I inspect my trained nltk classifier" is a reasonable question (and would have been even more so if it had named the specific classifier(s) of interest...).
– alexis
Nov 24 '18 at 21:34
@Tomáš, I suppose it could be read as "how do I compute my own classifier models". But "how do I inspect my trained nltk classifier" is a reasonable question (and would have been even more so if it had named the specific classifier(s) of interest...).
– alexis
Nov 24 '18 at 21:34
add a comment |
1 Answer
1
active
oldest
votes
There are plenty of ways, but they depend on the classifier. Since you tagged this nltk, I will assume that you are using the nltk's classifiers. Let's say you have trained a NaiveBayesClassifier on a classification task. The trained classifier has a method most_informative_features() that will show you the features (words) with the strongest correlation to some category. You can ask it to show you as many features as you care to.
If memory serves, several nltk classifiers have equivalent methods; but you'll have to look for the ones you use. More generally, you can examine the internal tables of any classifier, and study the code to understand what the numbers mean. (It will help greatly if you understand the algorithm before you look at the code.)
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53454284%2fnlp-ml-how-to-know-the-weight-of-words-used-in-text-classifier%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
There are plenty of ways, but they depend on the classifier. Since you tagged this nltk, I will assume that you are using the nltk's classifiers. Let's say you have trained a NaiveBayesClassifier on a classification task. The trained classifier has a method most_informative_features() that will show you the features (words) with the strongest correlation to some category. You can ask it to show you as many features as you care to.
If memory serves, several nltk classifiers have equivalent methods; but you'll have to look for the ones you use. More generally, you can examine the internal tables of any classifier, and study the code to understand what the numbers mean. (It will help greatly if you understand the algorithm before you look at the code.)
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
add a comment |
There are plenty of ways, but they depend on the classifier. Since you tagged this nltk, I will assume that you are using the nltk's classifiers. Let's say you have trained a NaiveBayesClassifier on a classification task. The trained classifier has a method most_informative_features() that will show you the features (words) with the strongest correlation to some category. You can ask it to show you as many features as you care to.
If memory serves, several nltk classifiers have equivalent methods; but you'll have to look for the ones you use. More generally, you can examine the internal tables of any classifier, and study the code to understand what the numbers mean. (It will help greatly if you understand the algorithm before you look at the code.)
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
add a comment |
There are plenty of ways, but they depend on the classifier. Since you tagged this nltk, I will assume that you are using the nltk's classifiers. Let's say you have trained a NaiveBayesClassifier on a classification task. The trained classifier has a method most_informative_features() that will show you the features (words) with the strongest correlation to some category. You can ask it to show you as many features as you care to.
If memory serves, several nltk classifiers have equivalent methods; but you'll have to look for the ones you use. More generally, you can examine the internal tables of any classifier, and study the code to understand what the numbers mean. (It will help greatly if you understand the algorithm before you look at the code.)
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
There are plenty of ways, but they depend on the classifier. Since you tagged this nltk, I will assume that you are using the nltk's classifiers. Let's say you have trained a NaiveBayesClassifier on a classification task. The trained classifier has a method most_informative_features() that will show you the features (words) with the strongest correlation to some category. You can ask it to show you as many features as you care to.
If memory serves, several nltk classifiers have equivalent methods; but you'll have to look for the ones you use. More generally, you can examine the internal tables of any classifier, and study the code to understand what the numbers mean. (It will help greatly if you understand the algorithm before you look at the code.)
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
answered Nov 24 '18 at 21:31
alexisalexis
33.6k954114
33.6k954114
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53454284%2fnlp-ml-how-to-know-the-weight-of-words-used-in-text-classifier%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I think this question is too broad for StackOverflow. Try to use CrossValidated StackExchange.
– Tomáš Přinda
Nov 24 '18 at 1:16
@Tomáš, I suppose it could be read as "how do I compute my own classifier models". But "how do I inspect my trained nltk classifier" is a reasonable question (and would have been even more so if it had named the specific classifier(s) of interest...).
– alexis
Nov 24 '18 at 21:34