ASP.NET Core - Cache problems with SPA route in Web API

Ok, I'm not really sure what the exact problem with my caching is but I'm leaning towards my SPA route fallback in my WebAPI. Sorry for a rather messy post and I know that cache busting is a hot topic here on SO and on blogs but I just can't find the solution to my problem.

First a short overview of the solution. I'm using webpack to bundle my front end code into more or less one css, one js and a index.html. I'm 100% sure that this index.html references these unique css and js files and that the hashes are unique per build/release. Webpack copies these files into my ASP.NET Core project and the wwwroot directory. In my ASP.NET Core code I setup my MVC routes like this:

app.UseMvc(routes =>
{
  routes.MapRoute("api", "api/{controller=Controller}/{action=Index}/{id?}");
  routes.MapRoute("default", "{*url}", new {controller = "Home", action = "RedirectIndex"});
});

The home controller and index action does this:

public ActionResult RedirectIndex()
{
  return new PhysicalFileResult(Path.Combine(_hostingEnvironment.WebRootPath, "index.html"), new MediaTypeHeaderValue("text/html"));
}

I also use app.UseDefaultFiles(); and app.UseStaticFiles(); in my Startup.cs but no cache control.

This is hosted in a web app on Azure.

When I navigate to my app like so: https://my-app.com/ then the default route kicks in and I get served my index.html. Everything works as expected. Except that I get a cached index.html that references my old js and old css files.

NOTE! I do NOT delete my old js and css files from the Azure hosted site so when the cached index.html references the old files - they get served without problems.

Since the index.html is cached my users get an old version of the js logic but the server is updated so my web api sends updated stuff that the js can't handle - I get crashes. I have to tell my users to ctrl + shift + r to do a hard refresh and then it all works.

And one final piece to the puzzle. When I logout I redirect the users to auth0 (an authentication service) to perform the logout. This service (auth0) then redirects the users back to my site (https://my-app.com/) and guess what - they get the old cached index.html again, even though they just ctrl+shift+r refreshed the very same site a couple of minutes earlier.

I'm so confused by this - is it the browser (Chrome) that performs this cache? Is it my magical SPA route redirect using PhysicalFileResult()? Is it Azure? Should I just rename my index.html each build and reference that unique index.html? (That would be an obvious solution but then my webpack config needs to manipulate my C# server code and that creeps me out)