Plotly (R) - Pie chart: How to fixate the color assignment color per group?

We are building a Shiny app in which we use a dynamic plotly pie chart. Users can select variables to include in the plot.

Adding custom color codes allow me to add our custom colors, but they won't stick to the grouping variable as they do in plotly bar charts (using the colors command).

Example:

library(plotly)

dat <- data.frame(dis = c(1,2,3), value = c(20,30,24))

plot_ly(dat, labels = ~dis, values = ~value, sort = F, 
        marker = list(colors = c("1" = "#B76C9E", 
                                 "2" = "#4285F4",
                                 "3" = "#EA4335"))) %>%
  add_pie(hole = 0.3) %>%
  layout(legend = list(orientation = 'h'), margin = list(l = 0 , r = 0, t = 0, b = 100, pad = 1),
         yaxis = list(showgrid = FALSE, zeroline = FALSE, showticklabels = FALSE))

But when I remove dis == 1 from the data.frame (like my users will do in the Shiny app) the colors shift and won't remain their assigned color:

dat2 <- data.frame(dis = c(2,3), value = c(30,24))

plot_ly(dat2, labels = ~dis, values = ~value, sort = F, 
        marker = list(colors = c("1" = "#B76C9E", 
                                 "2" = "#4285F4",
                                 "3" = "#EA4335"))) %>%
  add_pie(hole = 0.3) %>%
  layout(legend = list(orientation = 'h'), margin = list(l = 0 , r = 0, t = 0, b = 100, pad = 1),
         yaxis = list(showgrid = FALSE, zeroline = FALSE, showticklabels = FALSE))

Is there a way to lock the color to the categorical variable so "2" will always get the same color?