SqlAlchemy sessions get stuck when not using them
up vote
1
down vote
favorite
I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.
My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..
I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.
This is my class:
class MySqlClient():
engine = None
Session = None
def __init__(self):
# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)
self.session = MySqlClient.Session()
def get_budget(self, budget_id):
try:
q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()
return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }
except Exception as ex:
logging.error(ex)
return None
When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:
(mysql.connector.errors.OperationalError) MySQL Connection not available.
I also tried getting a new session before executing new queries. That didn't help either.
I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.
Any help would be greatly appreciated,
Regards
python-3.x sqlalchemy
New contributor
add a comment |
up vote
1
down vote
favorite
I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.
My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..
I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.
This is my class:
class MySqlClient():
engine = None
Session = None
def __init__(self):
# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)
self.session = MySqlClient.Session()
def get_budget(self, budget_id):
try:
q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()
return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }
except Exception as ex:
logging.error(ex)
return None
When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:
(mysql.connector.errors.OperationalError) MySQL Connection not available.
I also tried getting a new session before executing new queries. That didn't help either.
I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.
Any help would be greatly appreciated,
Regards
python-3.x sqlalchemy
New contributor
Not relevant to your question but you should consider putting a lock around theif MySqlClient.engine == None:
part as there is a race condition there.
– SuperShoot
17 hours ago
Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago
I tried to replicate the issue. Ran 2 threads, each would instantiate aMySqlClient
instance (once only), callget_budget()
and sleep for 5 minutes before callingget_budget()
again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
– SuperShoot
16 hours ago
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.
My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..
I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.
This is my class:
class MySqlClient():
engine = None
Session = None
def __init__(self):
# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)
self.session = MySqlClient.Session()
def get_budget(self, budget_id):
try:
q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()
return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }
except Exception as ex:
logging.error(ex)
return None
When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:
(mysql.connector.errors.OperationalError) MySQL Connection not available.
I also tried getting a new session before executing new queries. That didn't help either.
I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.
Any help would be greatly appreciated,
Regards
python-3.x sqlalchemy
New contributor
I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.
My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..
I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.
This is my class:
class MySqlClient():
engine = None
Session = None
def __init__(self):
# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)
self.session = MySqlClient.Session()
def get_budget(self, budget_id):
try:
q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()
return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }
except Exception as ex:
logging.error(ex)
return None
When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:
(mysql.connector.errors.OperationalError) MySQL Connection not available.
I also tried getting a new session before executing new queries. That didn't help either.
I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.
Any help would be greatly appreciated,
Regards
python-3.x sqlalchemy
python-3.x sqlalchemy
New contributor
New contributor
New contributor
asked 20 hours ago
Alexander Cornillie
61
61
New contributor
New contributor
Not relevant to your question but you should consider putting a lock around theif MySqlClient.engine == None:
part as there is a race condition there.
– SuperShoot
17 hours ago
Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago
I tried to replicate the issue. Ran 2 threads, each would instantiate aMySqlClient
instance (once only), callget_budget()
and sleep for 5 minutes before callingget_budget()
again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
– SuperShoot
16 hours ago
add a comment |
Not relevant to your question but you should consider putting a lock around theif MySqlClient.engine == None:
part as there is a race condition there.
– SuperShoot
17 hours ago
Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago
I tried to replicate the issue. Ran 2 threads, each would instantiate aMySqlClient
instance (once only), callget_budget()
and sleep for 5 minutes before callingget_budget()
again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
– SuperShoot
16 hours ago
Not relevant to your question but you should consider putting a lock around the
if MySqlClient.engine == None:
part as there is a race condition there.– SuperShoot
17 hours ago
Not relevant to your question but you should consider putting a lock around the
if MySqlClient.engine == None:
part as there is a race condition there.– SuperShoot
17 hours ago
Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago
Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago
I tried to replicate the issue. Ran 2 threads, each would instantiate a
MySqlClient
instance (once only), call get_budget()
and sleep for 5 minutes before calling get_budget()
again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.– SuperShoot
16 hours ago
I tried to replicate the issue. Ran 2 threads, each would instantiate a
MySqlClient
instance (once only), call get_budget()
and sleep for 5 minutes before calling get_budget()
again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.– SuperShoot
16 hours ago
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.
Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.
Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.
Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53407729%2fsqlalchemy-sessions-get-stuck-when-not-using-them%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Not relevant to your question but you should consider putting a lock around the
if MySqlClient.engine == None:
part as there is a race condition there.– SuperShoot
17 hours ago
Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago
I tried to replicate the issue. Ran 2 threads, each would instantiate a
MySqlClient
instance (once only), callget_budget()
and sleep for 5 minutes before callingget_budget()
again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.– SuperShoot
16 hours ago