How could I handle mouse input while streaming MJPEG video to browser?


Let’s suppose I’m streaming a video to a browser, for example using one of the solutions reported here:

Streaming openCV C++ video to the browser

Let’s suppose I would like to select a region of the streamed video for example with a rectangle drawn with the mouse and then I want to send the coordinates of the vertexes of the rectangle to the server.

I could connect server and client with websockets, for example, but I do not know how to handle the fact that I have a streaming loop in the server that cannot be interrupted, while at the same time I have to check for mouse input on the client and then send the result of the mouse position to the server that could decide to modify the streamed output depending on the mouse position (for example I would like to apply a specific image filter to the selected rectangle).

How could I achieve this?

Source: Windows Questions C++