Code for streaming video over tcp socket in python; need help understanding parts of it-CodePudding

I got this code for streaming a video from a client to a server:

Client:

import cv2, imutils
import mss
import numpy
from win32api import GetSystemMetrics
import pickle
import socket, struct

client_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host_ip = "IPADRESS"
port = 9999
client_socket.connect((host_ip,port))

with mss.mss() as sct:
    monitor = {"top": 0, "left": 0, "width": GetSystemMetrics(0), "height": GetSystemMetrics(1)}
    while True:
        img = numpy.array(sct.grab(monitor))
        frame = imutils.resize(img, width=1400)
        a = pickle.dumps(frame)
        message = struct.pack("Q",len(a)) a
        client_socket.send(message)

Server:

import cv2, imutils
import numpy as np
import pickle, struct
import socket
import threading

server_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host_ip = "IP_ADRESS"
port = 9999
socket_address = (host_ip,port)
server_socket.bind(socket_address)
server_socket.listen()
print("Listening at",socket_address)

def show_client(addr,client_socket):
    try:
        print('CLIENT {} CONNECTED!'.format(addr))
        if client_socket: # if a client socket exists
            data = b""
            payload_size = struct.calcsize("Q")
            while True:
                while len(data) < payload_size:
                    packet = client_socket.recv(4*1024)
                    if not packet: 
                        break
                    data =packet
                packed_msg_size = data[:payload_size]
                data = data[payload_size:]
                msg_size = struct.unpack("Q",packed_msg_size)[0]
                while len(data) < msg_size:
                    data  = client_socket.recv(4*1024)
                frame_data = data[:msg_size]
                data  = data[msg_size:]
                frame = pickle.loads(frame_data)
                cv2.imshow("Screen", frame)
                key = cv2.waitKey(1) & 0xFF
                if key  == ord('q'):
                    break
            client_socket.close()
    except Exception as e:
        print(e)
        print(f"CLINET {addr} DISCONNECTED")
        pass

while True:
    client_socket,addr = server_socket.accept()
    thread = threading.Thread(target=show_client, args=(addr,client_socket))
    thread.start()
    print("TOTAL CLIENTS ",threading.activeCount() - 1)

A lot of this code is from a youtuber called "pyshine", and everything is working just fine, but I don't understand, what a specific part of this code is really doing. These are the parts:

First of all in the client-code:

message = struct.pack("Q",len(a)) a

I know that it does something with the length of the pickle and, that it appends the pickle to it, but not more.

Second of all in the server-code:

            data = b""
            payload_size = struct.calcsize("Q")
            while True:
                while len(data) < payload_size:
                    packet = client_socket.recv(4*1024)
                    if not packet: 
                        break
                    data =packet
                packed_msg_size = data[:payload_size]
                data = data[payload_size:]
                msg_size = struct.unpack("Q",packed_msg_size)[0]
                while len(data) < msg_size:
                    data  = client_socket.recv(4*1024)
                frame_data = data[:msg_size]

With printing out some values, I definitely understood it a bit better, but the whole process, how it gets the final "frame_data", is still a mystery to me. So I would really appreciate, if someone could explain me the process that is going there.

CodePudding user response：

socket is primitive object and it doesn't care what data you send. You can send two frames and client can get it as single package or it may get it as many small packages - socket doesn't care where is end of first frame. To resolve this problem this code first sends len(data) and next data. It uses struct("Q") so this value len(data) has always 8 bytes. This way receiver knows how much data it has to receive to have complete frame - first it gets 8 bytes to get len(data) and later it use this value to get all data. And this is what second code does - it repeats recv() until it gets all data. It also checks if it doesn't get data from next frame - and keep this part as data[payload_size:] to use it with next frame/

If you will use the same rule(s) on both sides - sender first sends 8 bytes with sizeand next data, receiver first gets 8 bytes with size and next get data (using size) - then you have defined protocol. (similar to other protocols: HTTP (HyperText Transfer Protocol), FTP (File Transfer Protocol), SMTP (Send Mail Transfer Protocol), etc.)