pandas Add html tables values where data is int












0














I am getting html table based on day so if I search for 20 days it brings me 20 table and I want to add all 20 tables in 1 table so I can verify data within time series.
I have tried merge and add functions of pandas but it just add as string.



Table one



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


table two



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


code but it add as string.



tab_data = [[item.text for item in row_data.select("th,td")]
for row_data in tables.select("tr")]
df = pd.DataFrame(tab_data)
df2 = pd.DataFrame(tab_data)
df3 = df.add(df2,fill_value=0)
df









share|improve this question




















  • 2




    Instead of using a list comprehension, use a for loop and try to convert values in the table to numbers (float or int) in the loop. Make use of try/except to handle actual strings.
    – wwii
    Nov 22 at 22:11












  • Can you give me little hint even with pseudo code ?
    – Ashfaque Marfani
    Nov 22 at 22:15










  • In row I have string and number mix how can i solve it ?
    – Ashfaque Marfani
    Nov 22 at 22:16
















0














I am getting html table based on day so if I search for 20 days it brings me 20 table and I want to add all 20 tables in 1 table so I can verify data within time series.
I have tried merge and add functions of pandas but it just add as string.



Table one



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


table two



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


code but it add as string.



tab_data = [[item.text for item in row_data.select("th,td")]
for row_data in tables.select("tr")]
df = pd.DataFrame(tab_data)
df2 = pd.DataFrame(tab_data)
df3 = df.add(df2,fill_value=0)
df









share|improve this question




















  • 2




    Instead of using a list comprehension, use a for loop and try to convert values in the table to numbers (float or int) in the loop. Make use of try/except to handle actual strings.
    – wwii
    Nov 22 at 22:11












  • Can you give me little hint even with pseudo code ?
    – Ashfaque Marfani
    Nov 22 at 22:15










  • In row I have string and number mix how can i solve it ?
    – Ashfaque Marfani
    Nov 22 at 22:16














0












0








0


1





I am getting html table based on day so if I search for 20 days it brings me 20 table and I want to add all 20 tables in 1 table so I can verify data within time series.
I have tried merge and add functions of pandas but it just add as string.



Table one



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


table two



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


code but it add as string.



tab_data = [[item.text for item in row_data.select("th,td")]
for row_data in tables.select("tr")]
df = pd.DataFrame(tab_data)
df2 = pd.DataFrame(tab_data)
df3 = df.add(df2,fill_value=0)
df









share|improve this question















I am getting html table based on day so if I search for 20 days it brings me 20 table and I want to add all 20 tables in 1 table so I can verify data within time series.
I have tried merge and add functions of pandas but it just add as string.



Table one



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


table two



[['xa0', 'All Issues', 'Investment Grade', 'High Yield', 'Convertible'],
['Total Issues Traded', '8039', '5456', '2386', '197'],
['Advances', '3834', '2671', '1075', '88'],
['Declines', '3668', '2580', '994', '94'],
['Unchanged', '163', '54', '99', '10'],
['52 Week High', '305', '100', '193', '12'],
['52 Week Low', '152', '83', '63', '6'],
['Dollar Volume*', '27568', '17000', '9299', '1269']]


code but it add as string.



tab_data = [[item.text for item in row_data.select("th,td")]
for row_data in tables.select("tr")]
df = pd.DataFrame(tab_data)
df2 = pd.DataFrame(tab_data)
df3 = df.add(df2,fill_value=0)
df






python python-3.x pandas selenium selenium-webdriver






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 22 at 22:17

























asked Nov 22 at 22:04









Ashfaque Marfani

5712




5712








  • 2




    Instead of using a list comprehension, use a for loop and try to convert values in the table to numbers (float or int) in the loop. Make use of try/except to handle actual strings.
    – wwii
    Nov 22 at 22:11












  • Can you give me little hint even with pseudo code ?
    – Ashfaque Marfani
    Nov 22 at 22:15










  • In row I have string and number mix how can i solve it ?
    – Ashfaque Marfani
    Nov 22 at 22:16














  • 2




    Instead of using a list comprehension, use a for loop and try to convert values in the table to numbers (float or int) in the loop. Make use of try/except to handle actual strings.
    – wwii
    Nov 22 at 22:11












  • Can you give me little hint even with pseudo code ?
    – Ashfaque Marfani
    Nov 22 at 22:15










  • In row I have string and number mix how can i solve it ?
    – Ashfaque Marfani
    Nov 22 at 22:16








2




2




Instead of using a list comprehension, use a for loop and try to convert values in the table to numbers (float or int) in the loop. Make use of try/except to handle actual strings.
– wwii
Nov 22 at 22:11






Instead of using a list comprehension, use a for loop and try to convert values in the table to numbers (float or int) in the loop. Make use of try/except to handle actual strings.
– wwii
Nov 22 at 22:11














Can you give me little hint even with pseudo code ?
– Ashfaque Marfani
Nov 22 at 22:15




Can you give me little hint even with pseudo code ?
– Ashfaque Marfani
Nov 22 at 22:15












In row I have string and number mix how can i solve it ?
– Ashfaque Marfani
Nov 22 at 22:16




In row I have string and number mix how can i solve it ?
– Ashfaque Marfani
Nov 22 at 22:16












2 Answers
2






active

oldest

votes


















1














If you want to convert the numeric cells into integers, you would need to do that explicitly, as follows:



tab_data = [[int(item.text) if item.text.isdigit() else item.text 
for item in row_data.select("th,td")]
for row_data in tables.select("tr")]


Hope it helps.






share|improve this answer





















  • Thankyou so much, I did worked for int cells. :)
    – Ashfaque Marfani
    Nov 23 at 11:06



















1














The way you are converting the data frame treats all values as text.
There are two options here.




  1. Explicitly convert the strings to the data type you want using astype

  2. Use read_html to create data frames from html tables, which also tries to do the data type conversion.






share|improve this answer





















  • Thankyou so much for your answer but read_html is not working it is same as above.
    – Ashfaque Marfani
    Nov 23 at 11:05











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53438454%2fpandas-add-html-tables-values-where-data-is-int%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes









1














If you want to convert the numeric cells into integers, you would need to do that explicitly, as follows:



tab_data = [[int(item.text) if item.text.isdigit() else item.text 
for item in row_data.select("th,td")]
for row_data in tables.select("tr")]


Hope it helps.






share|improve this answer





















  • Thankyou so much, I did worked for int cells. :)
    – Ashfaque Marfani
    Nov 23 at 11:06
















1














If you want to convert the numeric cells into integers, you would need to do that explicitly, as follows:



tab_data = [[int(item.text) if item.text.isdigit() else item.text 
for item in row_data.select("th,td")]
for row_data in tables.select("tr")]


Hope it helps.






share|improve this answer





















  • Thankyou so much, I did worked for int cells. :)
    – Ashfaque Marfani
    Nov 23 at 11:06














1












1








1






If you want to convert the numeric cells into integers, you would need to do that explicitly, as follows:



tab_data = [[int(item.text) if item.text.isdigit() else item.text 
for item in row_data.select("th,td")]
for row_data in tables.select("tr")]


Hope it helps.






share|improve this answer












If you want to convert the numeric cells into integers, you would need to do that explicitly, as follows:



tab_data = [[int(item.text) if item.text.isdigit() else item.text 
for item in row_data.select("th,td")]
for row_data in tables.select("tr")]


Hope it helps.







share|improve this answer












share|improve this answer



share|improve this answer










answered Nov 22 at 23:07









TeeKea

3,19541430




3,19541430












  • Thankyou so much, I did worked for int cells. :)
    – Ashfaque Marfani
    Nov 23 at 11:06


















  • Thankyou so much, I did worked for int cells. :)
    – Ashfaque Marfani
    Nov 23 at 11:06
















Thankyou so much, I did worked for int cells. :)
– Ashfaque Marfani
Nov 23 at 11:06




Thankyou so much, I did worked for int cells. :)
– Ashfaque Marfani
Nov 23 at 11:06













1














The way you are converting the data frame treats all values as text.
There are two options here.




  1. Explicitly convert the strings to the data type you want using astype

  2. Use read_html to create data frames from html tables, which also tries to do the data type conversion.






share|improve this answer





















  • Thankyou so much for your answer but read_html is not working it is same as above.
    – Ashfaque Marfani
    Nov 23 at 11:05
















1














The way you are converting the data frame treats all values as text.
There are two options here.




  1. Explicitly convert the strings to the data type you want using astype

  2. Use read_html to create data frames from html tables, which also tries to do the data type conversion.






share|improve this answer





















  • Thankyou so much for your answer but read_html is not working it is same as above.
    – Ashfaque Marfani
    Nov 23 at 11:05














1












1








1






The way you are converting the data frame treats all values as text.
There are two options here.




  1. Explicitly convert the strings to the data type you want using astype

  2. Use read_html to create data frames from html tables, which also tries to do the data type conversion.






share|improve this answer












The way you are converting the data frame treats all values as text.
There are two options here.




  1. Explicitly convert the strings to the data type you want using astype

  2. Use read_html to create data frames from html tables, which also tries to do the data type conversion.







share|improve this answer












share|improve this answer



share|improve this answer










answered Nov 22 at 22:42









hmad

863




863












  • Thankyou so much for your answer but read_html is not working it is same as above.
    – Ashfaque Marfani
    Nov 23 at 11:05


















  • Thankyou so much for your answer but read_html is not working it is same as above.
    – Ashfaque Marfani
    Nov 23 at 11:05
















Thankyou so much for your answer but read_html is not working it is same as above.
– Ashfaque Marfani
Nov 23 at 11:05




Thankyou so much for your answer but read_html is not working it is same as above.
– Ashfaque Marfani
Nov 23 at 11:05


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53438454%2fpandas-add-html-tables-values-where-data-is-int%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

Calculate evaluation metrics using cross_val_predict sklearn

Insert data from modal to MySQL (multiple modal on website)