Using Entity Framework Code First to have two Foreign Keys from same parent table without having to specify the collections on the parent entity

So I'm following this answer in trying to get two foreign keys to get to a single table and it works.

public class Team
{
    public int TeamId { get; set;} 
    public string Name { get; set; }

public virtual ICollection<Match> HomeMatches { get; set; }
    public virtual ICollection<Match> AwayMatches { get; set; }
}

public class Match
{
    public int MatchId { get; set; }

public int HomeTeamId { get; set; }
    public int GuestTeamId { get; set; }

public float HomePoints { get; set; }
    public float GuestPoints { get; set; }
    public DateTime Date { get; set; }

public virtual Team HomeTeam { get; set; }
    public virtual Team GuestTeam { get; set; }
}

public class Context : DbContext
{
    ...

protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Match>()
                    .HasRequired(m => m.HomeTeam)
                    .WithMany(t => t.HomeMatches)
                    .HasForeignKey(m => m.HomeTeamId)
                    .WillCascadeOnDelete(false);

modelBuilder.Entity<Match>()
                    .HasRequired(m => m.GuestTeam)
                    .WithMany(t => t.AwayMatches)
                    .HasForeignKey(m => m.GuestTeamId)
                    .WillCascadeOnDelete(false);
    }
}

However, in my solution I don't want Team to have HomeMatches and AwayMatches collections as it doesn't make sense to be able to navigate to Match from that entity.

Is it possible to have two foreign keys pointing to the same table when the entity for the parent table doesn't have collections for the child tables.

I would like my Team entity to be like below.

public class Team
{
    public int TeamId { get; set;} 
    public string Name { get; set; }
    // HomeMatches and AwayMatches collection is no longer here
}

How could I use the modelBuilder to articulate to EntityFramework that I want to HomeTeamID and GuestTeamID to be foreign keys of Team?