Jersey: Close I/O resources after HTTP response











up vote
1
down vote

favorite












My Setup: I have created a REST service (Jersey/Dropwizard) that streams large content from a database. During a GET operation, the service taps into the database through a connection pool, wraps the data into a stream and performs some on-the-fly transformation to render the requested data in various encodings (CSV, JSON, ...). The life time of the database connection is tied to the life time of the stream and only when the stream is closed, the database connection is released.



The stream transformation is performed by an Encoder class that returns a StreamingOutput which is then passed to the Response object. The Encoder currently handles resource closing when the stream is fully consumed.



My Problem: Since StreamingOutput does not implement AutoCloseable, connection leaks may occur when the output is only partially consumed.



I sometimes observe that stale active connections are piling up in the connection pool, and I suspect that they arise from aborted HTTP connections. As you can see below, the current code handles exceptions that occur in the try block. What I cannot handle are Exceptions that occur after the return statement and I don't know how to attach any instructions for resource closing to the Response object.



My Question: How can I inform the Response object to close particular resources after the request has terminated (regularly or due to an error)? Or: Is there a better way to safely close any associated resources when the request context ends?



 @GET
//@Produces(...)
public Response streamData(
@PathParam("key") String key,
// ... other params
) {

//decode and validate params
Stream<Pojo> ps = null;
try {
// connect to db and obtain data stream for <key>
ps = loadData(db, key);
// apply detailed encoding instrunctions and create a StreamingOutput
final StreamingOutput stream = Encoder.encodeData(ps, encodingArgs);
return Response.ok(stream).build();
} catch (Exception e) {
closeOnException(ps); // wrapper for ps.close();
throw e;
}
}









share|improve this question






















  • Just thinking, from my understanding of your code, the real problem seems the reference to StreamingOutput object inside the Encode class (I don't know this Encode class, perhaps you wrote it). An alternative solution, you can extend StreamingOutput and create a class that also contains the loaded data as well, and the Encode is only a builder that does not maintain references to created streams.
    – Fabiano Tarlao
    Nov 22 at 17:05










  • In this way, when something goes wrong (everywhere), the GC is able to free the object cause the only reference is in the Response obj, that is going to be freed by GC as well. Is this feasible in your case? In that case I'll convert comment into an answer.
    – Fabiano Tarlao
    Nov 22 at 17:06










  • @FabianoTarlao: True, and this is probably the most robust solution. However, that way forces me to construct a StreamingOutput around every block of application logic that streams from the DB and encodes.
    – Matthias
    Nov 22 at 18:39










  • @FabianoTarlao: You really want to close I/O resources eagerly and don't wait for the GC. There's simply no guarantee that the GC will clean up the response objects in the near future.
    – Matthias
    Nov 22 at 18:45










  • Got it. The other idea is to return an object that implements StreamingOutput interface, perhaps extending your previous class, but that also updates an internal long "last_used_time" variable that can be periodically checked by Encode class. But I don't understand your constraints, can you change/rewrite the internals of StreamingOutput and Encode class, or not? I think that this is my last useful tip, I have not other ideas :-) and all these solutions look too much convoluted ;-)
    – Fabiano Tarlao
    Nov 22 at 22:18















up vote
1
down vote

favorite












My Setup: I have created a REST service (Jersey/Dropwizard) that streams large content from a database. During a GET operation, the service taps into the database through a connection pool, wraps the data into a stream and performs some on-the-fly transformation to render the requested data in various encodings (CSV, JSON, ...). The life time of the database connection is tied to the life time of the stream and only when the stream is closed, the database connection is released.



The stream transformation is performed by an Encoder class that returns a StreamingOutput which is then passed to the Response object. The Encoder currently handles resource closing when the stream is fully consumed.



My Problem: Since StreamingOutput does not implement AutoCloseable, connection leaks may occur when the output is only partially consumed.



I sometimes observe that stale active connections are piling up in the connection pool, and I suspect that they arise from aborted HTTP connections. As you can see below, the current code handles exceptions that occur in the try block. What I cannot handle are Exceptions that occur after the return statement and I don't know how to attach any instructions for resource closing to the Response object.



My Question: How can I inform the Response object to close particular resources after the request has terminated (regularly or due to an error)? Or: Is there a better way to safely close any associated resources when the request context ends?



 @GET
//@Produces(...)
public Response streamData(
@PathParam("key") String key,
// ... other params
) {

//decode and validate params
Stream<Pojo> ps = null;
try {
// connect to db and obtain data stream for <key>
ps = loadData(db, key);
// apply detailed encoding instrunctions and create a StreamingOutput
final StreamingOutput stream = Encoder.encodeData(ps, encodingArgs);
return Response.ok(stream).build();
} catch (Exception e) {
closeOnException(ps); // wrapper for ps.close();
throw e;
}
}









share|improve this question






















  • Just thinking, from my understanding of your code, the real problem seems the reference to StreamingOutput object inside the Encode class (I don't know this Encode class, perhaps you wrote it). An alternative solution, you can extend StreamingOutput and create a class that also contains the loaded data as well, and the Encode is only a builder that does not maintain references to created streams.
    – Fabiano Tarlao
    Nov 22 at 17:05










  • In this way, when something goes wrong (everywhere), the GC is able to free the object cause the only reference is in the Response obj, that is going to be freed by GC as well. Is this feasible in your case? In that case I'll convert comment into an answer.
    – Fabiano Tarlao
    Nov 22 at 17:06










  • @FabianoTarlao: True, and this is probably the most robust solution. However, that way forces me to construct a StreamingOutput around every block of application logic that streams from the DB and encodes.
    – Matthias
    Nov 22 at 18:39










  • @FabianoTarlao: You really want to close I/O resources eagerly and don't wait for the GC. There's simply no guarantee that the GC will clean up the response objects in the near future.
    – Matthias
    Nov 22 at 18:45










  • Got it. The other idea is to return an object that implements StreamingOutput interface, perhaps extending your previous class, but that also updates an internal long "last_used_time" variable that can be periodically checked by Encode class. But I don't understand your constraints, can you change/rewrite the internals of StreamingOutput and Encode class, or not? I think that this is my last useful tip, I have not other ideas :-) and all these solutions look too much convoluted ;-)
    – Fabiano Tarlao
    Nov 22 at 22:18













up vote
1
down vote

favorite









up vote
1
down vote

favorite











My Setup: I have created a REST service (Jersey/Dropwizard) that streams large content from a database. During a GET operation, the service taps into the database through a connection pool, wraps the data into a stream and performs some on-the-fly transformation to render the requested data in various encodings (CSV, JSON, ...). The life time of the database connection is tied to the life time of the stream and only when the stream is closed, the database connection is released.



The stream transformation is performed by an Encoder class that returns a StreamingOutput which is then passed to the Response object. The Encoder currently handles resource closing when the stream is fully consumed.



My Problem: Since StreamingOutput does not implement AutoCloseable, connection leaks may occur when the output is only partially consumed.



I sometimes observe that stale active connections are piling up in the connection pool, and I suspect that they arise from aborted HTTP connections. As you can see below, the current code handles exceptions that occur in the try block. What I cannot handle are Exceptions that occur after the return statement and I don't know how to attach any instructions for resource closing to the Response object.



My Question: How can I inform the Response object to close particular resources after the request has terminated (regularly or due to an error)? Or: Is there a better way to safely close any associated resources when the request context ends?



 @GET
//@Produces(...)
public Response streamData(
@PathParam("key") String key,
// ... other params
) {

//decode and validate params
Stream<Pojo> ps = null;
try {
// connect to db and obtain data stream for <key>
ps = loadData(db, key);
// apply detailed encoding instrunctions and create a StreamingOutput
final StreamingOutput stream = Encoder.encodeData(ps, encodingArgs);
return Response.ok(stream).build();
} catch (Exception e) {
closeOnException(ps); // wrapper for ps.close();
throw e;
}
}









share|improve this question













My Setup: I have created a REST service (Jersey/Dropwizard) that streams large content from a database. During a GET operation, the service taps into the database through a connection pool, wraps the data into a stream and performs some on-the-fly transformation to render the requested data in various encodings (CSV, JSON, ...). The life time of the database connection is tied to the life time of the stream and only when the stream is closed, the database connection is released.



The stream transformation is performed by an Encoder class that returns a StreamingOutput which is then passed to the Response object. The Encoder currently handles resource closing when the stream is fully consumed.



My Problem: Since StreamingOutput does not implement AutoCloseable, connection leaks may occur when the output is only partially consumed.



I sometimes observe that stale active connections are piling up in the connection pool, and I suspect that they arise from aborted HTTP connections. As you can see below, the current code handles exceptions that occur in the try block. What I cannot handle are Exceptions that occur after the return statement and I don't know how to attach any instructions for resource closing to the Response object.



My Question: How can I inform the Response object to close particular resources after the request has terminated (regularly or due to an error)? Or: Is there a better way to safely close any associated resources when the request context ends?



 @GET
//@Produces(...)
public Response streamData(
@PathParam("key") String key,
// ... other params
) {

//decode and validate params
Stream<Pojo> ps = null;
try {
// connect to db and obtain data stream for <key>
ps = loadData(db, key);
// apply detailed encoding instrunctions and create a StreamingOutput
final StreamingOutput stream = Encoder.encodeData(ps, encodingArgs);
return Response.ok(stream).build();
} catch (Exception e) {
closeOnException(ps); // wrapper for ps.close();
throw e;
}
}






java jersey response autocloseable






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 22 at 14:32









Matthias

948




948












  • Just thinking, from my understanding of your code, the real problem seems the reference to StreamingOutput object inside the Encode class (I don't know this Encode class, perhaps you wrote it). An alternative solution, you can extend StreamingOutput and create a class that also contains the loaded data as well, and the Encode is only a builder that does not maintain references to created streams.
    – Fabiano Tarlao
    Nov 22 at 17:05










  • In this way, when something goes wrong (everywhere), the GC is able to free the object cause the only reference is in the Response obj, that is going to be freed by GC as well. Is this feasible in your case? In that case I'll convert comment into an answer.
    – Fabiano Tarlao
    Nov 22 at 17:06










  • @FabianoTarlao: True, and this is probably the most robust solution. However, that way forces me to construct a StreamingOutput around every block of application logic that streams from the DB and encodes.
    – Matthias
    Nov 22 at 18:39










  • @FabianoTarlao: You really want to close I/O resources eagerly and don't wait for the GC. There's simply no guarantee that the GC will clean up the response objects in the near future.
    – Matthias
    Nov 22 at 18:45










  • Got it. The other idea is to return an object that implements StreamingOutput interface, perhaps extending your previous class, but that also updates an internal long "last_used_time" variable that can be periodically checked by Encode class. But I don't understand your constraints, can you change/rewrite the internals of StreamingOutput and Encode class, or not? I think that this is my last useful tip, I have not other ideas :-) and all these solutions look too much convoluted ;-)
    – Fabiano Tarlao
    Nov 22 at 22:18


















  • Just thinking, from my understanding of your code, the real problem seems the reference to StreamingOutput object inside the Encode class (I don't know this Encode class, perhaps you wrote it). An alternative solution, you can extend StreamingOutput and create a class that also contains the loaded data as well, and the Encode is only a builder that does not maintain references to created streams.
    – Fabiano Tarlao
    Nov 22 at 17:05










  • In this way, when something goes wrong (everywhere), the GC is able to free the object cause the only reference is in the Response obj, that is going to be freed by GC as well. Is this feasible in your case? In that case I'll convert comment into an answer.
    – Fabiano Tarlao
    Nov 22 at 17:06










  • @FabianoTarlao: True, and this is probably the most robust solution. However, that way forces me to construct a StreamingOutput around every block of application logic that streams from the DB and encodes.
    – Matthias
    Nov 22 at 18:39










  • @FabianoTarlao: You really want to close I/O resources eagerly and don't wait for the GC. There's simply no guarantee that the GC will clean up the response objects in the near future.
    – Matthias
    Nov 22 at 18:45










  • Got it. The other idea is to return an object that implements StreamingOutput interface, perhaps extending your previous class, but that also updates an internal long "last_used_time" variable that can be periodically checked by Encode class. But I don't understand your constraints, can you change/rewrite the internals of StreamingOutput and Encode class, or not? I think that this is my last useful tip, I have not other ideas :-) and all these solutions look too much convoluted ;-)
    – Fabiano Tarlao
    Nov 22 at 22:18
















Just thinking, from my understanding of your code, the real problem seems the reference to StreamingOutput object inside the Encode class (I don't know this Encode class, perhaps you wrote it). An alternative solution, you can extend StreamingOutput and create a class that also contains the loaded data as well, and the Encode is only a builder that does not maintain references to created streams.
– Fabiano Tarlao
Nov 22 at 17:05




Just thinking, from my understanding of your code, the real problem seems the reference to StreamingOutput object inside the Encode class (I don't know this Encode class, perhaps you wrote it). An alternative solution, you can extend StreamingOutput and create a class that also contains the loaded data as well, and the Encode is only a builder that does not maintain references to created streams.
– Fabiano Tarlao
Nov 22 at 17:05












In this way, when something goes wrong (everywhere), the GC is able to free the object cause the only reference is in the Response obj, that is going to be freed by GC as well. Is this feasible in your case? In that case I'll convert comment into an answer.
– Fabiano Tarlao
Nov 22 at 17:06




In this way, when something goes wrong (everywhere), the GC is able to free the object cause the only reference is in the Response obj, that is going to be freed by GC as well. Is this feasible in your case? In that case I'll convert comment into an answer.
– Fabiano Tarlao
Nov 22 at 17:06












@FabianoTarlao: True, and this is probably the most robust solution. However, that way forces me to construct a StreamingOutput around every block of application logic that streams from the DB and encodes.
– Matthias
Nov 22 at 18:39




@FabianoTarlao: True, and this is probably the most robust solution. However, that way forces me to construct a StreamingOutput around every block of application logic that streams from the DB and encodes.
– Matthias
Nov 22 at 18:39












@FabianoTarlao: You really want to close I/O resources eagerly and don't wait for the GC. There's simply no guarantee that the GC will clean up the response objects in the near future.
– Matthias
Nov 22 at 18:45




@FabianoTarlao: You really want to close I/O resources eagerly and don't wait for the GC. There's simply no guarantee that the GC will clean up the response objects in the near future.
– Matthias
Nov 22 at 18:45












Got it. The other idea is to return an object that implements StreamingOutput interface, perhaps extending your previous class, but that also updates an internal long "last_used_time" variable that can be periodically checked by Encode class. But I don't understand your constraints, can you change/rewrite the internals of StreamingOutput and Encode class, or not? I think that this is my last useful tip, I have not other ideas :-) and all these solutions look too much convoluted ;-)
– Fabiano Tarlao
Nov 22 at 22:18




Got it. The other idea is to return an object that implements StreamingOutput interface, perhaps extending your previous class, but that also updates an internal long "last_used_time" variable that can be periodically checked by Encode class. But I don't understand your constraints, can you change/rewrite the internals of StreamingOutput and Encode class, or not? I think that this is my last useful tip, I have not other ideas :-) and all these solutions look too much convoluted ;-)
– Fabiano Tarlao
Nov 22 at 22:18












1 Answer
1






active

oldest

votes

















up vote
0
down vote













You can add to your method HttpServletResponse:



@Produces(MediaType.APPLICATION_OCTET_STREAM)
public Object streamData(
@PathParam("key") String key,
@Context HttpServletResponse response,
// ... other params
) {
...
response.getOutputStream().write(....)
response.flushBuffer();
response.getOutputStream().close();
return null;
}





share|improve this answer





















  • Hmm, I was hoping for some convenient hook (similar to Stream.onClose(Autocloseable)) that I could use to avoid boilerplate code. Do you know how Jersey treats the retuned object from the method signature? Is it just discarded?
    – Matthias
    Nov 22 at 14:59












  • may be this will be helpfull stackoverflow.com/questions/39572872/…
    – xyz
    Nov 22 at 15:24











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433174%2fjersey-close-i-o-resources-after-http-response%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
0
down vote













You can add to your method HttpServletResponse:



@Produces(MediaType.APPLICATION_OCTET_STREAM)
public Object streamData(
@PathParam("key") String key,
@Context HttpServletResponse response,
// ... other params
) {
...
response.getOutputStream().write(....)
response.flushBuffer();
response.getOutputStream().close();
return null;
}





share|improve this answer





















  • Hmm, I was hoping for some convenient hook (similar to Stream.onClose(Autocloseable)) that I could use to avoid boilerplate code. Do you know how Jersey treats the retuned object from the method signature? Is it just discarded?
    – Matthias
    Nov 22 at 14:59












  • may be this will be helpfull stackoverflow.com/questions/39572872/…
    – xyz
    Nov 22 at 15:24















up vote
0
down vote













You can add to your method HttpServletResponse:



@Produces(MediaType.APPLICATION_OCTET_STREAM)
public Object streamData(
@PathParam("key") String key,
@Context HttpServletResponse response,
// ... other params
) {
...
response.getOutputStream().write(....)
response.flushBuffer();
response.getOutputStream().close();
return null;
}





share|improve this answer





















  • Hmm, I was hoping for some convenient hook (similar to Stream.onClose(Autocloseable)) that I could use to avoid boilerplate code. Do you know how Jersey treats the retuned object from the method signature? Is it just discarded?
    – Matthias
    Nov 22 at 14:59












  • may be this will be helpfull stackoverflow.com/questions/39572872/…
    – xyz
    Nov 22 at 15:24













up vote
0
down vote










up vote
0
down vote









You can add to your method HttpServletResponse:



@Produces(MediaType.APPLICATION_OCTET_STREAM)
public Object streamData(
@PathParam("key") String key,
@Context HttpServletResponse response,
// ... other params
) {
...
response.getOutputStream().write(....)
response.flushBuffer();
response.getOutputStream().close();
return null;
}





share|improve this answer












You can add to your method HttpServletResponse:



@Produces(MediaType.APPLICATION_OCTET_STREAM)
public Object streamData(
@PathParam("key") String key,
@Context HttpServletResponse response,
// ... other params
) {
...
response.getOutputStream().write(....)
response.flushBuffer();
response.getOutputStream().close();
return null;
}






share|improve this answer












share|improve this answer



share|improve this answer










answered Nov 22 at 14:44









xyz

133113




133113












  • Hmm, I was hoping for some convenient hook (similar to Stream.onClose(Autocloseable)) that I could use to avoid boilerplate code. Do you know how Jersey treats the retuned object from the method signature? Is it just discarded?
    – Matthias
    Nov 22 at 14:59












  • may be this will be helpfull stackoverflow.com/questions/39572872/…
    – xyz
    Nov 22 at 15:24


















  • Hmm, I was hoping for some convenient hook (similar to Stream.onClose(Autocloseable)) that I could use to avoid boilerplate code. Do you know how Jersey treats the retuned object from the method signature? Is it just discarded?
    – Matthias
    Nov 22 at 14:59












  • may be this will be helpfull stackoverflow.com/questions/39572872/…
    – xyz
    Nov 22 at 15:24
















Hmm, I was hoping for some convenient hook (similar to Stream.onClose(Autocloseable)) that I could use to avoid boilerplate code. Do you know how Jersey treats the retuned object from the method signature? Is it just discarded?
– Matthias
Nov 22 at 14:59






Hmm, I was hoping for some convenient hook (similar to Stream.onClose(Autocloseable)) that I could use to avoid boilerplate code. Do you know how Jersey treats the retuned object from the method signature? Is it just discarded?
– Matthias
Nov 22 at 14:59














may be this will be helpfull stackoverflow.com/questions/39572872/…
– xyz
Nov 22 at 15:24




may be this will be helpfull stackoverflow.com/questions/39572872/…
– xyz
Nov 22 at 15:24


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433174%2fjersey-close-i-o-resources-after-http-response%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Futebolista

Lallio

Jornalista