Does Java HTTP Client handle compression












4















I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip or deflate compression?



I would expect to have a BodyHandler for e.g. something like this:



HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())


but I don't see any. I don't see any configuration in HttpClient either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?










share|improve this question

























  • Have you tryed looking at the network log? If the client attaches the header Accept-Encoding: gzip it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.

    – patrickf
    Nov 27 '18 at 16:02






  • 1





    Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.

    – nullpointer
    Nov 27 '18 at 16:22













  • @patrickf actually I added such header and was surprised that I got uncompressed content

    – Krzysztof Krasoń
    Nov 27 '18 at 20:30






  • 1





    @RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.

    – Krzysztof Krasoń
    Nov 27 '18 at 20:34
















4















I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip or deflate compression?



I would expect to have a BodyHandler for e.g. something like this:



HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())


but I don't see any. I don't see any configuration in HttpClient either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?










share|improve this question

























  • Have you tryed looking at the network log? If the client attaches the header Accept-Encoding: gzip it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.

    – patrickf
    Nov 27 '18 at 16:02






  • 1





    Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.

    – nullpointer
    Nov 27 '18 at 16:22













  • @patrickf actually I added such header and was surprised that I got uncompressed content

    – Krzysztof Krasoń
    Nov 27 '18 at 20:30






  • 1





    @RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.

    – Krzysztof Krasoń
    Nov 27 '18 at 20:34














4












4








4








I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip or deflate compression?



I would expect to have a BodyHandler for e.g. something like this:



HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())


but I don't see any. I don't see any configuration in HttpClient either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?










share|improve this question
















I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip or deflate compression?



I would expect to have a BodyHandler for e.g. something like this:



HttpResponse.BodyHandlers.ofGzipped(HttpResponse.BodyHandlers.ofString())


but I don't see any. I don't see any configuration in HttpClient either. Am I looking in the wrong place or was this intentionally not implemented and deferred to support libraries?







java java-11 java-http-client






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Feb 17 at 11:53









Bobulous

9,86142851




9,86142851










asked Nov 27 '18 at 15:11









Krzysztof KrasońKrzysztof Krasoń

15.8k106183




15.8k106183













  • Have you tryed looking at the network log? If the client attaches the header Accept-Encoding: gzip it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.

    – patrickf
    Nov 27 '18 at 16:02






  • 1





    Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.

    – nullpointer
    Nov 27 '18 at 16:22













  • @patrickf actually I added such header and was surprised that I got uncompressed content

    – Krzysztof Krasoń
    Nov 27 '18 at 20:30






  • 1





    @RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.

    – Krzysztof Krasoń
    Nov 27 '18 at 20:34



















  • Have you tryed looking at the network log? If the client attaches the header Accept-Encoding: gzip it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.

    – patrickf
    Nov 27 '18 at 16:02






  • 1





    Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.

    – nullpointer
    Nov 27 '18 at 16:22













  • @patrickf actually I added such header and was surprised that I got uncompressed content

    – Krzysztof Krasoń
    Nov 27 '18 at 20:30






  • 1





    @RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.

    – Krzysztof Krasoń
    Nov 27 '18 at 20:34

















Have you tryed looking at the network log? If the client attaches the header Accept-Encoding: gzip it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.

– patrickf
Nov 27 '18 at 16:02





Have you tryed looking at the network log? If the client attaches the header Accept-Encoding: gzip it supports it. Note that there is often a difference of headers you get on the application side and on the network side of the http client.

– patrickf
Nov 27 '18 at 16:02




1




1





Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.

– nullpointer
Nov 27 '18 at 16:22







Going through some documentation felt this might be related to the question HPACK (Header Compression for HTTP/2) implementation. Details over Indexing Tables used in compression does mention both your sample compressions header in the Appendix.

– nullpointer
Nov 27 '18 at 16:22















@patrickf actually I added such header and was surprised that I got uncompressed content

– Krzysztof Krasoń
Nov 27 '18 at 20:30





@patrickf actually I added such header and was surprised that I got uncompressed content

– Krzysztof Krasoń
Nov 27 '18 at 20:30




1




1





@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.

– Krzysztof Krasoń
Nov 27 '18 at 20:34





@RomainHippeau how is it a duplicate? The question you linked is about apache http client and mine is about Java http client (the one embedded in Java 11), notice the tags.

– Krzysztof Krasoń
Nov 27 '18 at 20:34












2 Answers
2






active

oldest

votes


















1














No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber​(Flow.Subscriber<? super List<ByteBuffer>> subscriber) or BodyHandlers.ofPublisher() methods.






share|improve this answer


























  • It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.

    – Krzysztof Krasoń
    Nov 29 '18 at 15:56






  • 1





    Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use BodyPublishers.ofInputStream(..) and BodySubscribers.ofInputStream() with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.

    – daniel
    Nov 29 '18 at 17:16











  • I did try the BodySubscriber approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.

    – Bobulous
    Jan 11 at 17:13



















2














I was also surprised that the new java.net.http framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream and are either uncompressed or compressed with gzip:



public static InputStream getDecodedInputStream(
HttpResponse<InputStream> httpResponse) {
String encoding = determineContentEncoding(httpResponse);
try {
switch (encoding) {
case "":
return httpResponse.body();
case "gzip":
return new GZIPInputStream(httpResponse.body());
default:
throw new UnsupportedOperationException(
"Unexpected Content-Encoding: " + encoding);
}
} catch (IOException ioe) {
throw new UncheckedIOException(ioe);
}
}

public static String determineContentEncoding(
HttpResponse<?> httpResponse) {
return httpResponse.headers().firstValue("Content-Encoding").orElse("");
}


Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body() in an InflaterInputStream.






share|improve this answer























    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53502626%2fdoes-java-http-client-handle-compression%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    1














    No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber​(Flow.Subscriber<? super List<ByteBuffer>> subscriber) or BodyHandlers.ofPublisher() methods.






    share|improve this answer


























    • It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.

      – Krzysztof Krasoń
      Nov 29 '18 at 15:56






    • 1





      Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use BodyPublishers.ofInputStream(..) and BodySubscribers.ofInputStream() with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.

      – daniel
      Nov 29 '18 at 17:16











    • I did try the BodySubscriber approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.

      – Bobulous
      Jan 11 at 17:13
















    1














    No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber​(Flow.Subscriber<? super List<ByteBuffer>> subscriber) or BodyHandlers.ofPublisher() methods.






    share|improve this answer


























    • It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.

      – Krzysztof Krasoń
      Nov 29 '18 at 15:56






    • 1





      Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use BodyPublishers.ofInputStream(..) and BodySubscribers.ofInputStream() with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.

      – daniel
      Nov 29 '18 at 17:16











    • I did try the BodySubscriber approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.

      – Bobulous
      Jan 11 at 17:13














    1












    1








    1







    No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber​(Flow.Subscriber<? super List<ByteBuffer>> subscriber) or BodyHandlers.ofPublisher() methods.






    share|improve this answer















    No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber​(Flow.Subscriber<? super List<ByteBuffer>> subscriber) or BodyHandlers.ofPublisher() methods.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Nov 29 '18 at 18:46

























    answered Nov 29 '18 at 14:25









    danieldaniel

    3039




    3039













    • It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.

      – Krzysztof Krasoń
      Nov 29 '18 at 15:56






    • 1





      Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use BodyPublishers.ofInputStream(..) and BodySubscribers.ofInputStream() with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.

      – daniel
      Nov 29 '18 at 17:16











    • I did try the BodySubscriber approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.

      – Bobulous
      Jan 11 at 17:13



















    • It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.

      – Krzysztof Krasoń
      Nov 29 '18 at 15:56






    • 1





      Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use BodyPublishers.ofInputStream(..) and BodySubscribers.ofInputStream() with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.

      – daniel
      Nov 29 '18 at 17:16











    • I did try the BodySubscriber approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.

      – Bobulous
      Jan 11 at 17:13

















    It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.

    – Krzysztof Krasoń
    Nov 29 '18 at 15:56





    It's a pity, considering that there is already a GzipInput/OutputStream in the standard library.

    – Krzysztof Krasoń
    Nov 29 '18 at 15:56




    1




    1





    Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use BodyPublishers.ofInputStream(..) and BodySubscribers.ofInputStream() with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.

    – daniel
    Nov 29 '18 at 17:16





    Right. Though using Input/OutputStream would force you back to synchronous mode when pulling the request bytes. Maybe you could use BodyPublishers.ofInputStream(..) and BodySubscribers.ofInputStream() with some combination of the PipedInput/OutputStream and GzipInput/OutputStream too - but then you'd still have to pull the request bytes.

    – daniel
    Nov 29 '18 at 17:16













    I did try the BodySubscriber approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.

    – Bobulous
    Jan 11 at 17:13





    I did try the BodySubscriber approach (see this question) but it led to a total hang. So instead I went with the less glamorous approach which I've described in my answer to @KrzysztofKrasoń and that works fine. Frustrating, though.

    – Bobulous
    Jan 11 at 17:13













    2














    I was also surprised that the new java.net.http framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream and are either uncompressed or compressed with gzip:



    public static InputStream getDecodedInputStream(
    HttpResponse<InputStream> httpResponse) {
    String encoding = determineContentEncoding(httpResponse);
    try {
    switch (encoding) {
    case "":
    return httpResponse.body();
    case "gzip":
    return new GZIPInputStream(httpResponse.body());
    default:
    throw new UnsupportedOperationException(
    "Unexpected Content-Encoding: " + encoding);
    }
    } catch (IOException ioe) {
    throw new UncheckedIOException(ioe);
    }
    }

    public static String determineContentEncoding(
    HttpResponse<?> httpResponse) {
    return httpResponse.headers().firstValue("Content-Encoding").orElse("");
    }


    Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body() in an InflaterInputStream.






    share|improve this answer




























      2














      I was also surprised that the new java.net.http framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream and are either uncompressed or compressed with gzip:



      public static InputStream getDecodedInputStream(
      HttpResponse<InputStream> httpResponse) {
      String encoding = determineContentEncoding(httpResponse);
      try {
      switch (encoding) {
      case "":
      return httpResponse.body();
      case "gzip":
      return new GZIPInputStream(httpResponse.body());
      default:
      throw new UnsupportedOperationException(
      "Unexpected Content-Encoding: " + encoding);
      }
      } catch (IOException ioe) {
      throw new UncheckedIOException(ioe);
      }
      }

      public static String determineContentEncoding(
      HttpResponse<?> httpResponse) {
      return httpResponse.headers().firstValue("Content-Encoding").orElse("");
      }


      Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body() in an InflaterInputStream.






      share|improve this answer


























        2












        2








        2







        I was also surprised that the new java.net.http framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream and are either uncompressed or compressed with gzip:



        public static InputStream getDecodedInputStream(
        HttpResponse<InputStream> httpResponse) {
        String encoding = determineContentEncoding(httpResponse);
        try {
        switch (encoding) {
        case "":
        return httpResponse.body();
        case "gzip":
        return new GZIPInputStream(httpResponse.body());
        default:
        throw new UnsupportedOperationException(
        "Unexpected Content-Encoding: " + encoding);
        }
        } catch (IOException ioe) {
        throw new UncheckedIOException(ioe);
        }
        }

        public static String determineContentEncoding(
        HttpResponse<?> httpResponse) {
        return httpResponse.headers().firstValue("Content-Encoding").orElse("");
        }


        Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body() in an InflaterInputStream.






        share|improve this answer













        I was also surprised that the new java.net.http framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream and are either uncompressed or compressed with gzip:



        public static InputStream getDecodedInputStream(
        HttpResponse<InputStream> httpResponse) {
        String encoding = determineContentEncoding(httpResponse);
        try {
        switch (encoding) {
        case "":
        return httpResponse.body();
        case "gzip":
        return new GZIPInputStream(httpResponse.body());
        default:
        throw new UnsupportedOperationException(
        "Unexpected Content-Encoding: " + encoding);
        }
        } catch (IOException ioe) {
        throw new UncheckedIOException(ioe);
        }
        }

        public static String determineContentEncoding(
        HttpResponse<?> httpResponse) {
        return httpResponse.headers().firstValue("Content-Encoding").orElse("");
        }


        Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body() in an InflaterInputStream.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Jan 6 at 17:37









        BobulousBobulous

        9,86142851




        9,86142851






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53502626%2fdoes-java-http-client-handle-compression%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

            Calculate evaluation metrics using cross_val_predict sklearn

            Insert data from modal to MySQL (multiple modal on website)