Selenium lagging when opening more sessions
up vote
2
down vote
favorite
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
asked Nov 22 at 9:38
Mark Letterman
161
add a comment |
up vote
2
down vote
favorite
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
asked Nov 22 at 9:38
Mark Letterman
161
Have you tried addingchrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')
– Moshe Slavin
Nov 22 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 at 11:05
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
asked Nov 22 at 9:38
Mark Letterman
161
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
python selenium web selenium-webdriver driver
asked Nov 22 at 9:38
Mark Letterman
161
asked Nov 22 at 9:38
Mark Letterman
161
asked Nov 22 at 9:38
Mark Letterman
161
asked Nov 22 at 9:38
Mark Letterman
161
asked Nov 22 at 9:38
Mark Letterman
161
161
Have you tried addingchrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')
– Moshe Slavin
Nov 22 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 at 11:05
add a comment |
Have you tried addingchrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')
– Moshe Slavin
Nov 22 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 at 11:05
Have you tried adding
chrome_options? you can use '--headless' like this chrome_options.add_argument('--headless')– Moshe Slavin
Nov 22 at 10:52
Have you tried adding
chrome_options? you can use '--headless' like this chrome_options.add_argument('--headless')– Moshe Slavin
Nov 22 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 at 10:57
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 at 10:59
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 at 11:05
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 at 11:05
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 at 15:43
Vladimir Efimov
655311
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 at 20:11
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 at 15:43
Vladimir Efimov
655311
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 at 20:11
add a comment |
up vote
1
down vote
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 at 15:43
Vladimir Efimov
655311
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 at 20:11
add a comment |
up vote
1
down vote
up vote
1
down vote
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 at 15:43
Vladimir Efimov
655311
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 at 15:43
Vladimir Efimov
655311
edited Nov 23 at 11:06
answered Nov 22 at 15:43
Vladimir Efimov
655311
answered Nov 22 at 15:43
Vladimir Efimov
655311
answered Nov 22 at 15:43
Vladimir Efimov
655311
655311
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 at 20:11
add a comment |
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 at 20:11
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 at 22:47
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 at 5:24
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 at 21:00
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 at 20:11
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 at 20:11
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53427858%2fselenium-lagging-when-opening-more-sessions%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Have you tried adding
chrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')– Moshe Slavin
Nov 22 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 at 11:05