"sign-in with" buttons mostly implement something like OAuth2 or OpenID protocol communicating with the authority (i.e. Google) server. Signing up/in the ordinary email way (when you enter your e-mail address directly into the 1-st party website without getting redirected to GMail/Facebook/whatever) does not, even if the email address is hosted on GMail. It either doesn't communicate to Google at all or just sends a simple email message to your address using bare SMTP. This is the nature of the difference and it is huge from the technical point of view.

There are downsides to both. Arguably those of the second are more annoying but less harmful.

The first downside I found (the moment I stopped use "Sign-in with GMail") - Google was passing additional non-essential privacy-compromising information about me besides my ID to the websites I signed-in to..