adding polygon over map in R, CRS issues

I'm trying to plot a map of Sweden, then plot a fake shape that I created over it. I have a script that plots municipalities of Sweden:

library(ggplot2)
library(sf)
library(smoothr)
library(jsonlite)


tmp <- tempfile()
download.file("http://api.thenmap.net/v2/se-7/geo/2020-06-06", destfile = tmp)

mun_name <- fromJSON("http://api.thenmap.net/v2/se-7/data/2020-06-06?language=en&data_props=name|shapeid|is_in") %>% 
  unnest(is_in) %>% 
  rename(county = is_in)

mun <- read_sf(tmp) %>% 
  left_join(mun_name, by = c("id" = "shapeid"))

ggplot(mun) +
  geom_sf(color="grey20",fill="grey95",size=.3) +
  theme_bw()

then I have a script where I make a polygon with st_polygon :

# make a polygon that is mostly inside sweden
sweden_polygon <-
  # create list of matrices and the first point same as last point
  list(
    matrix(
      c(14, 62, 
        12, 63, 
        14, 64, 
        16, 66, 
        20, 68,  
        20, 66,
        19, 65, 
        18, 63, 
        14, 62),
      ncol=2, byrow=T
    )
  ) 

# Create an sf polygon
sweden_polygon <-  sf::st_polygon(sweden_polygon)
# smooth the polygon
smooth_sweden_polygon <- smooth(sweden_polygon, method = "chaikin")

I can plot them both separately with seemingly the same coordinates, but when I plot them together, it doesn't work because the polygon doesn't have a CRS that matches Sweden's.

# this works:
ggplot() +
  geom_sf(data=mun,color="grey20",fill="grey95",size=.3) +
  theme_bw()

# this works:
ggplot() +
  geom_sf(data=smooth_sweden_polygon) +
  theme_bw()

# this don't work:
ggplot() +
  geom_sf(data=mun,color="grey20",fill="grey95",size=.3) +
  geom_sf(data=smooth_sweden_polygon) +
  theme_bw()

I know from st_crs(mun) that the coordinate system for Sweden is WGS 84 but I can't figure out how to assign that to my polygon.