Why GIFs get large in the first place
GIF compresses with LZW, scanning each row for repeated byte sequences and replacing them with shorter codes. It works brilliantly on flat colour and poorly on noise, and — this is the important part — it has no concept of motion. Every frame is compressed on its own terms.
Modern video codecs do the opposite: they store one full frame and then describe how the following frames differ from it. That is why a video of a mostly static screen recording is tiny and the same recording as a GIF is not. GIF does support skipping unchanged regions between frames, but it is a much weaker mechanism than motion compensation, and any noise, dithering or camera movement defeats it.