How to change the x-axis ticks in a facet-wrapt stacked bar chart--ggplot R

I'm trying to replace the x-axis tick in a faceted stacked bar chart by another variable I have in my dataframe. I am comparing the corresponding week for two different years, so I am using the "year" variable on x-axis, the "total" variable on y-axis and wrapping by the week's number in the years (i.e the first week of January gets number 1, the second is 2 etc). I've tried a lot of things already including scale_x_discrete; scale_x_date; axi.ticks.x, geom_text etc....

Here is a reproducible example:

dat <- data.frame(date = as.Date(c("2021-01-03", "2020-01-05", "2021-01-03", "2020-01-05", "2021-01-03", "2021-01-03", "2021-01-10","2020-01-12", "2021-01-10", "2020-01-12", "2021-01-10", "2021-01-10", "2021-01-17", "2020-01-19", "2021-01-17", "2020-01-19", "2021-01-17", "2021-01-17")),
                 total = c(109334, 150052, 36546, 51476, 2961, 19988, 106719, 126748, 31893, 
                           29576, 5401, 19336, 91021, 108892, 23030, 26115, 3861, 15663),
                 product = c("A", "A","B", "B", "C", "D", "A", "A", "B", "B", "C", "D", "A", 
                              "A","B", "B", "C", "D"),
                 week = c(1,1,1,1,1,1,2,2,2,2,2,2,3,3,3,3,3,3),
                 year = c(2021,2020,2021,2020,2021,2021,2021,2020,2021,2020,2021,2021,2021,2020,2021,2020,2021,2021))  

dat %>% ggplot(aes(year, total, color = year == 2021)) +
  geom_bar(position = "stack", stat = "identity", size =0.6)+
  #scale_x_date(expand = c(0,0), labels = date_format("%m/%d/%y"))+
  scale_y_continuous(expand = c(0,0), breaks = seq(0, 350000, by = 50000), limits=c(0, 350000))+
  #scale_x_discrete(breaks = dat$year, labels = dat$date)+
  scale_color_manual(values = c(NA, 'red'), guide=F)+
  facet_wrap(~week, nrow = 1)+
  #geom_text(aes(y= total, label = date), position = position_dodge(width = 1), angle = 90, vjust = -1, size = 2)+
  theme(axis.text.x=element_text(angle = 90, vjust =0, hjust =0),
        #axis.ticks.x = ,
        axis.title.x = element_blank(),
        axis.title.y = element_blank(),
        panel.background = element_blank(),
        panel.grid.major = element_blank(), 
        panel.grid.minor = element_blank(),
        panel.spacing.x = unit(0.1, "lines"))

The final result would be having the x-axis showning the week's date (Sunday's date for that week). desired x-axis ticks (please ignore the * on it)

Thanks!