Does Java HTTP Client handle compression
I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip
or deflate
compression?
I would expect to have a BodyHandler
for e.g. something like this:
HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())
but I don't see any. I don't see any configuration in HttpClient
either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?
java java-11 java-http-client
add a comment |
I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip
or deflate
compression?
I would expect to have a BodyHandler
for e.g. something like this:
HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())
but I don't see any. I don't see any configuration in HttpClient
either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?
java java-11 java-http-client
Have you tryed looking at the network log? If the client attaches the headerAccept-Encoding: gzip
it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.
– patrickf
Nov 27 '18 at 16:02
1
Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.
– nullpointer
Nov 27 '18 at 16:22
@patrickf actually I added such header and was surprised that I got uncompressed content
– Krzysztof Krasoń
Nov 27 '18 at 20:30
1
@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.
– Krzysztof Krasoń
Nov 27 '18 at 20:34
add a comment |
I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip
or deflate
compression?
I would expect to have a BodyHandler
for e.g. something like this:
HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())
but I don't see any. I don't see any configuration in HttpClient
either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?
java java-11 java-http-client
I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip
or deflate
compression?
I would expect to have a BodyHandler
for e.g. something like this:
HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())
but I don't see any. I don't see any configuration in HttpClient
either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?
java java-11 java-http-client
java java-11 java-http-client
edited Feb 17 at 11:53
Bobulous
9,86142851
9,86142851
asked Nov 27 '18 at 15:11
Krzysztof KrasońKrzysztof Krasoń
15.8k106183
15.8k106183
Have you tryed looking at the network log? If the client attaches the headerAccept-Encoding: gzip
it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.
– patrickf
Nov 27 '18 at 16:02
1
Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.
– nullpointer
Nov 27 '18 at 16:22
@patrickf actually I added such header and was surprised that I got uncompressed content
– Krzysztof Krasoń
Nov 27 '18 at 20:30
1
@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.
– Krzysztof Krasoń
Nov 27 '18 at 20:34
add a comment |
Have you tryed looking at the network log? If the client attaches the headerAccept-Encoding: gzip
it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.
– patrickf
Nov 27 '18 at 16:02
1
Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.
– nullpointer
Nov 27 '18 at 16:22
@patrickf actually I added such header and was surprised that I got uncompressed content
– Krzysztof Krasoń
Nov 27 '18 at 20:30
1
@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.
– Krzysztof Krasoń
Nov 27 '18 at 20:34
Have you tryed looking at the network log? If the client attaches the header
Accept-Encoding: gzip
it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.– patrickf
Nov 27 '18 at 16:02
Have you tryed looking at the network log? If the client attaches the header
Accept-Encoding: gzip
it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.– patrickf
Nov 27 '18 at 16:02
1
1
Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.
– nullpointer
Nov 27 '18 at 16:22
Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.
– nullpointer
Nov 27 '18 at 16:22
@patrickf actually I added such header and was surprised that I got uncompressed content
– Krzysztof Krasoń
Nov 27 '18 at 20:30
@patrickf actually I added such header and was surprised that I got uncompressed content
– Krzysztof Krasoń
Nov 27 '18 at 20:30
1
1
@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.
– Krzysztof Krasoń
Nov 27 '18 at 20:34
@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.
– Krzysztof Krasoń
Nov 27 '18 at 20:34
add a comment |
2 Answers
2
active
oldest
votes
No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber
to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber(Flow.Subscriber<? super List<ByteBuffer>> subscriber)
or BodyHandlers.ofPublisher()
methods.
It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.
– Krzysztof Krasoń
Nov 29 '18 at 15:56
1
Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could useBodyPublishers.ofInputStream(..)
andBodySubscribers.ofInputStream()
with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.
– daniel
Nov 29 '18 at 17:16
I did try theBodySubscriber
approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.
– Bobulous
Jan 11 at 17:13
add a comment |
I was also surprised that the new java.net.http
framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream
and are either uncompressed or compressed with gzip:
public static InputStream getDecodedInputStream(
HttpResponse<InputStream> httpResponse) {
String encoding = determineContentEncoding(httpResponse);
try {
switch (encoding) {
case "":
return httpResponse.body();
case "gzip":
return new GZIPInputStream(httpResponse.body());
default:
throw new UnsupportedOperationException(
"Unexpected Content-Encoding: " + encoding);
}
} catch (IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
public static String determineContentEncoding(
HttpResponse<?> httpResponse) {
return httpResponse.headers().firstValue("Content-Encoding").orElse("");
}
Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body()
in an InflaterInputStream
.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53502626%2fdoes-java-http-client-handle-compression%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber
to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber(Flow.Subscriber<? super List<ByteBuffer>> subscriber)
or BodyHandlers.ofPublisher()
methods.
It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.
– Krzysztof Krasoń
Nov 29 '18 at 15:56
1
Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could useBodyPublishers.ofInputStream(..)
andBodySubscribers.ofInputStream()
with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.
– daniel
Nov 29 '18 at 17:16
I did try theBodySubscriber
approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.
– Bobulous
Jan 11 at 17:13
add a comment |
No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber
to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber(Flow.Subscriber<? super List<ByteBuffer>> subscriber)
or BodyHandlers.ofPublisher()
methods.
It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.
– Krzysztof Krasoń
Nov 29 '18 at 15:56
1
Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could useBodyPublishers.ofInputStream(..)
andBodySubscribers.ofInputStream()
with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.
– daniel
Nov 29 '18 at 17:16
I did try theBodySubscriber
approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.
– Bobulous
Jan 11 at 17:13
add a comment |
No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber
to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber(Flow.Subscriber<? super List<ByteBuffer>> subscriber)
or BodyHandlers.ofPublisher()
methods.
No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber
to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber(Flow.Subscriber<? super List<ByteBuffer>> subscriber)
or BodyHandlers.ofPublisher()
methods.
edited Nov 29 '18 at 18:46
answered Nov 29 '18 at 14:25
danieldaniel
3039
3039
It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.
– Krzysztof Krasoń
Nov 29 '18 at 15:56
1
Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could useBodyPublishers.ofInputStream(..)
andBodySubscribers.ofInputStream()
with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.
– daniel
Nov 29 '18 at 17:16
I did try theBodySubscriber
approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.
– Bobulous
Jan 11 at 17:13
add a comment |
It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.
– Krzysztof Krasoń
Nov 29 '18 at 15:56
1
Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could useBodyPublishers.ofInputStream(..)
andBodySubscribers.ofInputStream()
with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.
– daniel
Nov 29 '18 at 17:16
I did try theBodySubscriber
approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.
– Bobulous
Jan 11 at 17:13
It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.
– Krzysztof Krasoń
Nov 29 '18 at 15:56
It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.
– Krzysztof Krasoń
Nov 29 '18 at 15:56
1
1
Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use
BodyPublishers.ofInputStream(..)
and BodySubscribers.ofInputStream()
with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.– daniel
Nov 29 '18 at 17:16
Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use
BodyPublishers.ofInputStream(..)
and BodySubscribers.ofInputStream()
with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.– daniel
Nov 29 '18 at 17:16
I did try the
BodySubscriber
approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.– Bobulous
Jan 11 at 17:13
I did try the
BodySubscriber
approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.– Bobulous
Jan 11 at 17:13
add a comment |
I was also surprised that the new java.net.http
framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream
and are either uncompressed or compressed with gzip:
public static InputStream getDecodedInputStream(
HttpResponse<InputStream> httpResponse) {
String encoding = determineContentEncoding(httpResponse);
try {
switch (encoding) {
case "":
return httpResponse.body();
case "gzip":
return new GZIPInputStream(httpResponse.body());
default:
throw new UnsupportedOperationException(
"Unexpected Content-Encoding: " + encoding);
}
} catch (IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
public static String determineContentEncoding(
HttpResponse<?> httpResponse) {
return httpResponse.headers().firstValue("Content-Encoding").orElse("");
}
Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body()
in an InflaterInputStream
.
add a comment |
I was also surprised that the new java.net.http
framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream
and are either uncompressed or compressed with gzip:
public static InputStream getDecodedInputStream(
HttpResponse<InputStream> httpResponse) {
String encoding = determineContentEncoding(httpResponse);
try {
switch (encoding) {
case "":
return httpResponse.body();
case "gzip":
return new GZIPInputStream(httpResponse.body());
default:
throw new UnsupportedOperationException(
"Unexpected Content-Encoding: " + encoding);
}
} catch (IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
public static String determineContentEncoding(
HttpResponse<?> httpResponse) {
return httpResponse.headers().firstValue("Content-Encoding").orElse("");
}
Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body()
in an InflaterInputStream
.
add a comment |
I was also surprised that the new java.net.http
framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream
and are either uncompressed or compressed with gzip:
public static InputStream getDecodedInputStream(
HttpResponse<InputStream> httpResponse) {
String encoding = determineContentEncoding(httpResponse);
try {
switch (encoding) {
case "":
return httpResponse.body();
case "gzip":
return new GZIPInputStream(httpResponse.body());
default:
throw new UnsupportedOperationException(
"Unexpected Content-Encoding: " + encoding);
}
} catch (IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
public static String determineContentEncoding(
HttpResponse<?> httpResponse) {
return httpResponse.headers().firstValue("Content-Encoding").orElse("");
}
Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body()
in an InflaterInputStream
.
I was also surprised that the new java.net.http
framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream
and are either uncompressed or compressed with gzip:
public static InputStream getDecodedInputStream(
HttpResponse<InputStream> httpResponse) {
String encoding = determineContentEncoding(httpResponse);
try {
switch (encoding) {
case "":
return httpResponse.body();
case "gzip":
return new GZIPInputStream(httpResponse.body());
default:
throw new UnsupportedOperationException(
"Unexpected Content-Encoding: " + encoding);
}
} catch (IOException ioe) {
throw new UncheckedIOException(ioe);
}
}
public static String determineContentEncoding(
HttpResponse<?> httpResponse) {
return httpResponse.headers().firstValue("Content-Encoding").orElse("");
}
Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body()
in an InflaterInputStream
.
answered Jan 6 at 17:37
BobulousBobulous
9,86142851
9,86142851
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53502626%2fdoes-java-http-client-handle-compression%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Have you tryed looking at the network log? If the client attaches the header
Accept-Encoding: gzip
it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.– patrickf
Nov 27 '18 at 16:02
1
Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.
– nullpointer
Nov 27 '18 at 16:22
@patrickf actually I added such header and was surprised that I got uncompressed content
– Krzysztof Krasoń
Nov 27 '18 at 20:30
1
@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.
– Krzysztof Krasoń
Nov 27 '18 at 20:34