How to make cookie auth scheme generate relative login url instead of absolute one?

You can handle the OnRedirectToLogin event to provide your own logic for the redirection process. Here's an example implementation:

services.AddAuthentication(CookieAuthenticationDefaults.AuthenticationScheme)
    .AddCookie(o =>
    {
        o.Events = new CookieAuthenticationEvents
        {
            OnRedirectToLogin = ctx =>
            {
               var relativeRedirectUri = new Uri(ctx.RedirectUri).PathAndQuery;

               context.Response.Headers["Location"] = relativeRedirectUri;
               context.Response.StatusCode = 401;

               return Task.CompletedTask;
           }
       };
   });

The ctx.RedirectUri property passed in is absolute, so the code above makes a relative copy and uses that instead.

The default implementation that this replaces is a little more involved, as it supports both AJAX requests that return a 401 and non-AJAX requests that perform a redirect. If you need to support both, have a look at the source and modify it accordingly.