Btukfyl

My results look like...

POST /post HTTP/1.1

Host: localhost:3003

User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0)    

Gecko/20100101 Firefox/62.0

Accept: */*

Accept-Language: en-US,en;q=0.5

Accept-Encoding: gzip, deflate

Referer: http://localhost:3003/profile

Content-type: multipart/form-data

Content-Length: 14708

Cookie: mycookie=7bdbed41954cd4133a172acb92988e58

Connection: keep-alive



-----------------------------4636945214860352321751082034

...

binary characters...

...

-----------------------------4636945214860352321751082034

Acquired from

boost::asio::async_read(

    socket_,

    strmbuffer_,

    boost::asio::transfer_exactly(bytes_to_transfer),

    strand_.wrap(

    [this, self](boost::system::error_code ec, std::size_t bytes_transferred)

    {

        std::stringstream ss;

        ss << buffer_data;  // from socket_.async_read_some()

        ss << &strmbuffer_; // now stringstream contains everything



        // the character routine which writes the above

        // and which i use for output...



        std::string output_file = "../../upload/test.png";

        std::ofstream outfile(output_file);

        char c;

        unsigned bl = boundary.length();

        bool endfile = false;

        unsigned bufsize = 512;

        if(outfile){

            char buffer[bufsize];

            while(!endfile){

                // here looks like below

                // to find and pass the first boundary

            }   // then stream continues...

            while(!endfile){

                unsigned j = 0;

                unsigned k;

                memset(buffer, 0, bufsize); // redundant

                while(j < bufsize && ss.get(c) && !endfile){

                    buffer[j] = c;

                    k = 0;

                    while(boundary[bl - 1 - k] == buffer[j - k]){

                        if(k >= bl - 1){

                            endfile = true;

                            break;

                        }

                        k++;

                     }

                     j++;

                 }

                 outfile.write(buffer, j);

                 j = 0;

             }

         }

    }

);

...essentially. Thus, the receiving

socket_.async_read_some()

gives me a

boost::array<char, 8192> buffer_;

which gives me the http request info. But in the case of multipart/form-data, it reads passed the first boundary, which means the next read() doesn't see it. Argh! (So does async_read_until().) So, in

boost::asio::async_read()

i convert the

boost::asio::streambuf strmbuffer_;

to a stringstream and add them together for the std::cout results above.

I'm not convinced that I should be using stringstream, at all. But the above routine (using stringstream) works fine in Boost::Beast. It does NOT in Asio. Unfortunately, the string_body type, which receives the http request in Beast, has a restrictive size limit, 1 meg, i believe. No idea how to change that.

I can't find much on this topic, anywhere. Perhaps, the information is too dangerous. If they told me, they'd have to kill me. What should I be using in Asio to write binary data to disk??

By default, the HTTP request parser starts with a 1 megabyte limit. This is to prevent resource exhaustion attacks where the client sends very large or endless amounts of body data. You can easily change this limit by calling parser::body_limit with the desired maximum. This is described in the documentation:

https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/ref/boost__beast__http__parser/body_limit.html
https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/using_http/buffer_oriented_parsing.html

In order to adjust parser parameters such as the body limit (or header limit) you will need to use the "parser stream operation" interface. This is explained here:

https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/using_http/parser_stream_operations.html

I will post my own partial solution. It works without error on a 14.2kb png. Above this can result in a segfault, unless the 'magic' number, as the following shows, is adjusted.

I'm using the Boost Asio HTTP Server example C++11.

In connection.hpp, change...

//boost::array<char, 8192> buffer_;

boost::array<char, 512> buffer_;

Also, add...

boost::asio::streambuf strmbuffer_;

In connection.cpp, here is the connection::handle_read()...

void connection::handle_read(const boost::system::error_code& e, std::size_t bytes_transferred)

{

    if (!e)

    {

        //*** buffer_.data() for this file ***

        //

        //POST /post HTTP/1.1

        //Host: localhost:3003

        //User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0

        //Accept: */*

        //Accept-Language: en-US,en;q=0.5

        //Accept-Encoding: gzip, deflate

        //Referer: http://localhost:3003/profile

        //Content-type: multipart/form-data

        //Content-Length: 14710

        //Cookie: knowstoryadmin=7bdbed41954cd4133a172acb92988e58

        //Connection: keep-alive

        //

        //-----------------------------14071968205478138611648202646

        //Content-Disposition: form-data; name="admin_profile_image_load"; filename="tlc-logo.png"

        //Content-Type: image/png

        //

        //�PNG

        //▒



        std::stringstream strm1;

        std::string buffer_data = buffer_.data();

        strm1 << buffer_data;



        std::string method;

        std::smatch match_method;

        std::regex regex_method ("\b([^ ]*)( )([^ ]*)( HTTP/1.1)([^ ]*)");



        std::string content_type;

        std::smatch match_content_type;

        std::regex regex_content_type ("\b(Content-type: )([^ ]*)");



        std::string line;



        while (std::getline(strm1, line)) {

            if (std::regex_search(line, match_method, regex_method)) {

                method = match_method[0];

                method = method.substr(0, method.find(' '));

                boost::trim(method);

                //std::cout << method << std::endl;

            }

            if (std::regex_search(line, match_content_type, regex_content_type))     {

                content_type = match_content_type[0];

                boost::erase_all(content_type, "Content-type:");

                boost::trim(content_type);

                //std::cout << content_type << std::endl;

            }

        }



        if (method == "POST") {

            if (content_type == "multipart/form-data") {



                std::string content_length;

                std::smatch match_content_length;

                std::regex regex_content_length ("\b(Content-Length: )([^ ]*)");



                std::string filename;

                std::smatch match_filename;

                std::regex regex_filename ("\b(filename)([^ ]*)");



                std::string action;

                std::smatch match_action;

                std::regex regex_action ("\b(name)([^ ]*)");



                std::string boundary;

                std::smatch match_boundary;

                std::regex regex_boundary ("([-]{10,}[0-9]{10,})");



                std::string line;



                strm1.clear();

                strm1 << buffer_data;

                while (std::getline(strm1, line)) {

                    if (std::regex_search(line, match_content_length, regex_content_length)) {

                        //Content-Length: 14710

                        content_length = match_content_length[0];

                        boost::erase_all(content_length, "Content-Length:");

                        boost::trim(content_length);

                        //std::cout << content_length << std::endl;

                    }

                    if (std::regex_search(line, match_filename, regex_filename)) {

                        filename = match_filename[0];

                        boost::erase_all(filename, """);

                        boost::erase_all(action, ";");

                        boost::erase_all(filename, "filename=");

                        std::size_t found = filename.find_last_of(".");

                        std::size_t len = filename.length();

                        std::string mime = filename.substr(found, len);

                        boost::trim(filename);

                        //std::cout << filename << std::endl;

                        //std::cout << mime << std::endl;

                    }

                    if (std::regex_search(line, match_action, regex_action)) {

                        action = match_action[0];

                        boost::erase_all(action, """);

                        boost::erase_all(action, ";");

                        boost::erase_all(action, "name=");

                        boost::trim(action);

                        //std::cout << action << std::endl;

                    }

                    if (std::regex_search(line, match_boundary, regex_boundary)) {

                        boundary = match_boundary[0];

                        boost::trim(boundary);

                        //std::cout << boundary << std::endl;

                    }

                }



                //pubseekpos works as expected, but useless here

                //strmbuffer_.pubseekpos(bytes_transferred);



                //content length minus bytes_transfered does NOT yield

                //the right result. The number, 392, is the 'magic' number

                //adjustment for this file size, approx 14.2kb, that i found

                //by trial and error.

                //Adjusting the magic number is necessary for every image size

                //in order to avoid a segfault.

                //bytes_transferred, for each read(), is the only 'reliable'

                //number with which to work, as far as i know.

                //If there is a brainier way of calculating this,

                //i don't care, anymore.

                int n_content_length = std::stoi(content_length);

                int transfer = n_content_length - bytes_transferred + 392;

                auto self(shared_from_this());

                boost::asio::async_read(

                    socket_,

                    strmbuffer_,

                    boost::asio::transfer_exactly(transfer),

                    strand_.wrap(

                        [this, self, boundary](boost::system::error_code ec, std::size_t bytes_transferred)

                        {

                            std::stringstream strm2;

                            strm2 << &strmbuffer_;

                            std::string line;

                            unsigned bufsize = 512;

                            while (std::getline(strm2, line))

                            {

                                if(line.length() == 1){

                                    std::string output_file = "../../upload/test.png";

                                    std::ofstream outfile(output_file);

                                    char c;

                                    unsigned bl = boundary.length();

                                    bool endfile = false;

                                    if(outfile){

                                        char buffer[bufsize];

                                        while(!endfile){

                                            unsigned j = 0;

                                            unsigned k;

                                            while(j < bufsize && strm2.get(c) && !endfile){

                                                buffer[j] = c;

                                                k = 0;

                                                while(boundary[bl - 1 - k] == buffer[j - k]){

                                                    if(k >= bl - 1){

                                                        endfile = true;

                                                        break;

                                                    }

                                                    k++;

                                                }

                                                j++;

                                            }

                                            outfile.write(buffer, j);

                                            j = 0;

                                        };

                                        outfile.close();

                                        std::cout << "outfile close" << std::endl;

                                        break;

                                    }

                                }

                            }

                        }

                    )

                );

            }

            else {

                // POST AJAX

                std::cout << "connection " << method << std::endl;

            }

        }

        else {

            boost::tribool result;

            boost::tie(result, boost::tuples::ignore) = request_parser_.parse(

                request_, buffer_.data(), buffer_.data() + bytes_transferred);



            if (result)

            {

                request_handler_.handle_request(

                    request_,

                    reply_);



                boost::asio::async_write(

                    socket_,

                    reply_.to_buffers(),

                    strand_.wrap(

                        boost::bind(

                            &connection::handle_write,

                            shared_from_this(),

                            boost::asio::placeholders::error)

                    ));

            }

            else if (!result)

            {

                reply_ = reply::stock_reply(reply::bad_request);

                boost::asio::async_write(

                    socket_,

                    reply_.to_buffers(),

                    strand_.wrap(

                        boost::bind(&connection::handle_write, shared_from_this(),

                        boost::asio::placeholders::error)));

            }

            else

            {

                socket_.async_read_some(

                    boost::asio::buffer(buffer_),

                    strand_.wrap(

                        boost::bind(

                            &connection::handle_read,

                            shared_from_this(),

                            boost::asio::placeholders::error,

                            boost::asio::placeholders::bytes_transferred)));

            }

        }

    }

}

This partial solution is rather 'unobtrusive' for this Asio example. The request_handler class is unchanged and serves GET.

In connection::start(), i tried async_read. But the results were even less predictable. It meant that i had to append two strings, one made from strmbuffer1_, the other from strmbuffer2_, in order to construct (hack) together the stream in the file output loop. No matter how precise my preparations appeared in cout<<, there were roughly 500 or more bytes missing, writing an incomplete image file.

Somewhere in memory there exists, and must exist, the complete information that i need for an uploaded binary and a handle to it. But extracting it from the Asio classes seems unnecessarily tricky.

This is how far I've come with Boost Asio. And this is as far as i go. For what i want, my answers are not here.