I don’t know what a .webp file is but I don’t like it. They’re like a filthy prank version of the image/gif you’re looking for. They make you jump through all these hoops to find the original versions of the files that you can actually do anything with.
Edit: honestly I assumed it had something to do with Google protecting themselves from image piracy shit
5mb to 100kb is not a typical result, so I would imagine that you are comparing apples to oranges (e.g. a very high quality jpeg vs a low quality webp)