I am working on a C project where i listen on sockets and generate HTTP responses based on the requests i get from my clients on my fds, in short i use my browser to send a request, i end up getting the raw request, i parse it and generate the corresponding http response.
However in the case of large POST requests, usually what happens is that i get partial requests, so in the first part i will usually only find the first line (version/method/uri), some headers but no body, and i guess am supposed to get the rest of the body somehow, however i am unable to figure out two things,
first of all how do i know if the request i am getting is partial or completed from just the first part ? i am not getting any information relating to range, here's the first part i get when my client sends me a POST request.
POST / HTTP/1.1
Host: localhost:8081
Connection: keep-alive
Content-Length: 8535833
Cache-Control: max-age=0
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Origin: http://127.0.0.1:8081
Upgrade-Insecure-Requests: 1
DNT: 1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryOs6fsdbaegBIumqh
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36
Accept: text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Referer: http://127.0.0.1:8081/
Accept-Encoding: gzip, deflate, br
Accept-Language: fr,en-US;q=0.9,en;q=0.8
how can i figure out just from this whether or not am getting a partial request or just a faulty request (I need to generate a 400 error in the case of a request that says it has X content-length but the body size is different)
second question is, suppose i already know whether or not its partial, how do i proceed with storing the entire request in a buffer before sending it to my parser and generating a response ? here's my reception function (i already know the client's fd, so i just recv on it
void Client::receive_request(void)
{
char buffer[2024];
int ret;
ret = recv(_fd, buffer, 2024, 0);
buffer[ret] = 0;
_received_request = buffer;
_bytes_request = ret;
std::cout << "Raw Request:\n" << _received_request << std::endl;
if (buffer[ret-1] == '\n')
{
_ready_request = true;
_request.parse(_received_request, _server->get_config());
}
}
and here's the code that checks whether or not a client is attempting to send a request, parse and generate a response
int Connections::check_clients() {
int fd;
for (std::vector<Client*>::iterator client = clients.begin();
client != clients.end() && ready_fd != 0 ; client )
{
fd = (*client)->get_fd();
if (FD_ISSET(fd, &ready_rset))
{
ready_fd--;
(*client)->receive_request();
if ((*client)->request_is_ready())
{
(*client)->wait_response();
close(fd);
FD_CLR(fd, &active_set);
fd_list.remove(fd);
max_fd = *std::max_element(fd_list.begin(), fd_list.end());
free(*client);
client = clients.erase(client);
}
}
}
return 0;
}
as you can see am coding everything in C (98) and would rather not get answers that just dismiss my questions and refer me to different technologies or libraries, unless it will help me understand what am doing wrong and how to handle partial requests.
for info, am only handling HTTP 1.1(GET/POST/DELETE only) and i usually only get this issue when am getting a large chunked file or a file upload that has a very large body. thank you
PS : if needed i can link up the github repo of the current project if you wanna look further into the code
CodePudding user response：
how can i figure out just from this whether or not am getting a partial request or just a faulty request (I need to generate a 400 error in the case of a request that says it has X content-length but the body size is different)
The body size is, by definition, the size of the Content-Length field. Any bytes that you receive afterwards belong to the next HTTP request (see HTTP pipelining). If you do not receive Content-Length bytes within a reasonable time period, then you can make the server issue a 408 Request Timeout error.
second question is, suppose i already know whether or not its partial, how do i proceed with storing the entire request in a buffer before sending it to my parser and generating a response ? here's my reception function (i already know the client's fd, so i just recv on it
Your posted code has at least the following problems:
- You should check the return value of
recvto determine whether the function succeeded or failed, and if it failed, you should handle the error appropriately. In your current code, ifrecvfails with the return value-1, then you will write to the arraybufferout of bounds, causing undefined behavior. - It does not seem appropriate to use the line
if (buffer[ret-1] == '\n'). The HTTP request header will be over when you encounter a"\r\n\r\n", and the HTTP request body will be over when you have readContent-Lengthbytes of the body. The ends of the header and body will not necessarily occur at the end of the data read byrecv, but can also occur in the middle. If you want to support HTTP pipelining, then the additional data should be handled by the handler for the next HTTP request. If you don't want to support HTTP pipelining, then you can simply discard the additional data and useConnection: closein the HTTP response header. - You seem to be using a null terminating character to mark the end of the data read by
recv. However, this will not work if a byte with the value0is part of the HTTP request. It is probably safe to assume that such a byte should not be part of the HTTP request header, but it is probably not safe to assume that such a byte won't be part of the HTTP request body (for example when using POST with binary data).