Having experienced the above when the changes rolled out on .com years ago this is deja-vu for me so I'll add my two cents in hope it helps someone else.
No, google doesn't when you download image its from your servers, not google's.
This is partially incorrect. Google does keep a lower resolution version of all images which it actually displays instantly when you expand an image on their site. The design simultaneously is downloading the image from your site and if/when successful your image is swapped into place on the Google page seamlessly, without reload. If you block Google from downloading the image, or your site is down, that attempt fails and Google still shows their copy.
This broke webmasters being able to redirect visitors to their sites whenever Google tried to load the image, "if image is not on my domain then redirect visitor to my domain" no longer works. I thought it was because my page couldn't be loaded as a preview behind the image for a while but when I dug in and tried to force the same on images it was clear they simply abort the load attempt if you try to use this type of code.
The reason for the drop, however, is that any incentive to come visit your site is lost when the image is already displayed in Google search and can be enlarged directly on google's page. Google did add a couple of links that help a visitor reach your site but, in my experience, those were rarely used. Again, the person looking up images already saw what they wanted to see. Three popular "fixes" were used by many webmasters
#1 - Do nothing, it is what it is. I think most webmasters left things as is and moved on.
#2 - Create a "click on image to see full size" semi transparent watermark that gets loaded over the image in Google search image results. This involved some code to detect the referrer (which google later removed causing a cat and mouse game with webmasters) but has long since been made into a plugin and extensively described online. This option encourages clicks from people who want to see the image but the bounce rate is extremely high.
#3 - The approach I ultimately ended up using, a robots.txt entry prohibiting google's image bot from indexing images. Unlike with web pages where Google will still index a page with a "the contents can not be shown due to robots.txt" as description, Google image simply does not index the image at all.
Why I went with #3. In the new layout I found that not only was traffic reduced by 90% the quality of that traffic also took a nosedive. I was left with 10% of my original image traffic which was also twice as likely to bounce back to Google to see more images. Not only that but I was still left with 100% of the scrapers of my images which had the effect of making them all duplicate anyway, and so wrongly credited to others. Scraping became easy and I couldn't block it because the scraper didn't even need to visit my site anymore leaving me no way to stop them.
My metrics in the Google dashboard, then known as webmaster tools, instantly and very dramatically changed when I blocked Google from my images. CTR skyrocketed because all data was now web based and not skewed by image data which had low CTR. All of the data in my account also became extremely useful and easy to understand because it was 100% web data and not an image/web mix. The settings for "web only" or "image only" did not filter 100% of the other, that also became obvious after the full image block.
In short I lost no benefits, 10% of the original traffic with a high bounce rate wasn't worth it, but I gained a lot in terms of improved metrics across the board. For sites who relied heavily on image traffic, and have a site that does well with a visual crowd, social sites like Pinterest replaced the lost traffic over the next year or so.
I now 100% block Google's image search and, unless they change their layout again to send more traffic, doubt I'll ever enble it again. I sincerely hoped they would revist their new layout when the effects were clear for webmasters, and I even joined the discussion to ask them for exactly that, but nothing has changed since. I guess that since it's been rolled out to other countries just recently it's not likely to change soon either. Maybe some day...