SqlAlchemy sessions get stuck when not using them











up vote
1
down vote

favorite












I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.



My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..



I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.



This is my class:



class MySqlClient():

engine = None
Session = None

def __init__(self):

# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)

self.session = MySqlClient.Session()

def get_budget(self, budget_id):
try:

q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()

return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }

except Exception as ex:
logging.error(ex)

return None


When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:



(mysql.connector.errors.OperationalError) MySQL Connection not available.


I also tried getting a new session before executing new queries. That didn't help either.



I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.



Any help would be greatly appreciated,



Regards










share|improve this question







New contributor




Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • Not relevant to your question but you should consider putting a lock around the if MySqlClient.engine == None: part as there is a race condition there.
    – SuperShoot
    17 hours ago










  • Does it fail when a single thread running? Or only when multiple threads running?
    – SuperShoot
    16 hours ago










  • I tried to replicate the issue. Ran 2 threads, each would instantiate a MySqlClient instance (once only), call get_budget() and sleep for 5 minutes before calling get_budget() again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
    – SuperShoot
    16 hours ago















up vote
1
down vote

favorite












I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.



My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..



I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.



This is my class:



class MySqlClient():

engine = None
Session = None

def __init__(self):

# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)

self.session = MySqlClient.Session()

def get_budget(self, budget_id):
try:

q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()

return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }

except Exception as ex:
logging.error(ex)

return None


When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:



(mysql.connector.errors.OperationalError) MySQL Connection not available.


I also tried getting a new session before executing new queries. That didn't help either.



I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.



Any help would be greatly appreciated,



Regards










share|improve this question







New contributor




Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • Not relevant to your question but you should consider putting a lock around the if MySqlClient.engine == None: part as there is a race condition there.
    – SuperShoot
    17 hours ago










  • Does it fail when a single thread running? Or only when multiple threads running?
    – SuperShoot
    16 hours ago










  • I tried to replicate the issue. Ran 2 threads, each would instantiate a MySqlClient instance (once only), call get_budget() and sleep for 5 minutes before calling get_budget() again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
    – SuperShoot
    16 hours ago













up vote
1
down vote

favorite









up vote
1
down vote

favorite











I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.



My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..



I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.



This is my class:



class MySqlClient():

engine = None
Session = None

def __init__(self):

# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)

self.session = MySqlClient.Session()

def get_budget(self, budget_id):
try:

q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()

return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }

except Exception as ex:
logging.error(ex)

return None


When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:



(mysql.connector.errors.OperationalError) MySQL Connection not available.


I also tried getting a new session before executing new queries. That didn't help either.



I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.



Any help would be greatly appreciated,



Regards










share|improve this question







New contributor




Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I'm having a hard time implementing a "MySqlClient" class for my application. My application consists of several modules which have to make use of my database & some of the modules are running on other threads.



My intention is to make an instance for every module that needs to communicate with my MySql database. For example: every client connecting to a websocket server creates his own instance, a telegram bot client has its own instance, ..



I've been searching for days now, I've read the docs, searched the forums .. but somehow I'm missing something or I'm not implementing it the right way.



This is my class:



class MySqlClient():

engine = None
Session = None

def __init__(self):

# create engine
if MySqlClient.engine == None:
MySqlClient.engine = sqlalchemy.create_engine("mysql+mysqlconnector://{0}:{1}@{2}/{3}".format(
state.config["mysql"]["user"],
state.config["mysql"]["password"],
state.config["mysql"]["host"],
state.config["mysql"]["database"]
))
MySqlClient.Session = scoped_session(sessionmaker(bind=MySqlClient.engine))
Base.metadata.create_all(MySqlClient.engine)

self.session = MySqlClient.Session()

def get_budget(self, budget_id):
try:

q = self.session.query(
(Budget.amount).label("budgetAmount"),
func.sum(BudgetRecord.amount).label("total")
).all().filter(Budget.id == budget_id).join(BudgetRecord).filter(extract("month", BudgetRecord.ts) == datetime.datetime.now().month)
self.session.close()

return { "budgetAmount": q[0].budgetAmount, "total": 0.0 if q[0].total == None else q[0].total }

except Exception as ex:
logging.error(ex)

return None


When I start my application everything runs fine, I can execute the method "get_budget" returning the data. However, if after this I wait for 5 minutes, the method won't run again (if I don't wait, it still works). After about 15 minutes after I made the call, the query finally fails saying the MySql connection has dropped:



(mysql.connector.errors.OperationalError) MySQL Connection not available.


I also tried getting a new session before executing new queries. That didn't help either.



I've done things like this before but it's the first time I'm using an ORM & I'd like to keep the benefits of using ORM.



Any help would be greatly appreciated,



Regards







python-3.x sqlalchemy






share|improve this question







New contributor




Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question







New contributor




Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question






New contributor




Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 20 hours ago









Alexander Cornillie

61




61




New contributor




Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Alexander Cornillie is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.












  • Not relevant to your question but you should consider putting a lock around the if MySqlClient.engine == None: part as there is a race condition there.
    – SuperShoot
    17 hours ago










  • Does it fail when a single thread running? Or only when multiple threads running?
    – SuperShoot
    16 hours ago










  • I tried to replicate the issue. Ran 2 threads, each would instantiate a MySqlClient instance (once only), call get_budget() and sleep for 5 minutes before calling get_budget() again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
    – SuperShoot
    16 hours ago


















  • Not relevant to your question but you should consider putting a lock around the if MySqlClient.engine == None: part as there is a race condition there.
    – SuperShoot
    17 hours ago










  • Does it fail when a single thread running? Or only when multiple threads running?
    – SuperShoot
    16 hours ago










  • I tried to replicate the issue. Ran 2 threads, each would instantiate a MySqlClient instance (once only), call get_budget() and sleep for 5 minutes before calling get_budget() again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
    – SuperShoot
    16 hours ago
















Not relevant to your question but you should consider putting a lock around the if MySqlClient.engine == None: part as there is a race condition there.
– SuperShoot
17 hours ago




Not relevant to your question but you should consider putting a lock around the if MySqlClient.engine == None: part as there is a race condition there.
– SuperShoot
17 hours ago












Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago




Does it fail when a single thread running? Or only when multiple threads running?
– SuperShoot
16 hours ago












I tried to replicate the issue. Ran 2 threads, each would instantiate a MySqlClient instance (once only), call get_budget() and sleep for 5 minutes before calling get_budget() again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
– SuperShoot
16 hours ago




I tried to replicate the issue. Ran 2 threads, each would instantiate a MySqlClient instance (once only), call get_budget() and sleep for 5 minutes before calling get_budget() again. Both threads successfully returned a result from the query 5 times before I killed it. Why are you so sure that it is the session causing the problem? If you can provide a copy/paste example that reproduces the issue, I'll have another go at it.
– SuperShoot
16 hours ago

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});






Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.










 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53407729%2fsqlalchemy-sessions-get-stuck-when-not-using-them%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes








Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.










 

draft saved


draft discarded


















Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.













Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.












Alexander Cornillie is a new contributor. Be nice, and check out our Code of Conduct.















 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53407729%2fsqlalchemy-sessions-get-stuck-when-not-using-them%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

Calculate evaluation metrics using cross_val_predict sklearn

Insert data from modal to MySQL (multiple modal on website)