This is a great project to learn and you are off to a good start. In case anyone would find it helpful I will share a bit about my learning process. One way reverse-engineer enthusiasts decode full industry NES and GBA game devices is by firing bits into memory locations and observing the results. It’s easier than simply observing the image stream data due to the sheer complexity and entropy of states the stream bits could be in(Was very very fun). Initially while attempting to decode I applied the same to single complete frames of the byte stream and observed how image changes are made on the Wifi UAV app. Then I collected my data
1.Changing certain bits alters brightness or darkness(sometimes completely dark other times completely white)
2. Changing other bits alters the intensity of the green or pink component(however a lot rarer and few and even further between)
3. Changing other bits seems to delete a ‘pixel’ and cause a tearing shift in subsequent rendered parts of the image
4. If the ‘pixel’ didn’t delete the ‘pixel’ itself appears to be distorted
5. These ‘pixels’ are not in fact pixels but blocks
6. The FF bit would break the Wifi UAV App on IOS and force an exit on the app
Eventually, you discover this is statistically similar to how JPEG images work. The usual Huffman, Quantization and Chrominance tables are not transmitted in the UDP stream. To read the data on your desktop you’d have to create your headers. That was why I had to write the program in my github. It simply puts the stream in a .jpg readable format, not by manipulating the raw data but by prepending the appropriate headers and mathematical values to the top. As you observed “it works” but it still glitches with lines when playing back videos on some frames. Whether it’s ‘Bad Frames’ which should be dropped or my algorithm needs some tweaking on the mathematical tables it uses or some odd combination of both, the final solution could still further be improved.
The flying and drone manipulation part is orders of magnitude easier however, you have maybe already realized that