remove background or mark filled areas from imfill

I'm doing some image segmentation in matlab of grayscale images taken from a drone using a thermo sensitive camera. The idea is that you should be able to put in a video whereafter it analyzes every frame and give a new video as output, now where each person is marked, clustered and a total count in the frame is given. So far what I am doing to remove the background is first imtophat and then some threshold on top of this I build some analysis to identify the people from e.g. fences, houses etc. However this threshold is way to static, so once there is a shift in outdoor temperature or the layer changes e.g. from grass to tarmac then I either get to many things in the picture or I remove some of the people. So what I am ultimately looking for is a way to get rid of the background. So what I have left is buildings, cars, people etc..

This is the ultimate goal and a solution to this would be highly appreciated.

What I tried to do was to first use the following code on the first picture (where pic1 is the original picture):

%Make it double
pic2 = double(pic1);
%Remove some noise
pic2 = wiener2(pic2);
%Make the pedestrians larger
pic2 = imdilate(pic2,strel('disk',5));
%In case of shadows take these to some minimum
pic3 = pic2.*(pic2>mean(mean(pic2))) + mean(mean(pic2))*(pic2<mean(mean(pic2)));

%Remove some of the background
pic4 = imtophat(pic3,strel('disk',10));

%Make the edges stand out.
hy = fspecial('sobel');
hx = hy';
Iy = imfilter(gaussian, hy, 'replicate');
Ix = imfilter(gaussian, hx, 'replicate');
gradmag = sqrt(Ix.^2 + Iy.^2);

%Threshold the edges
BW = gradmag>100;

%Close the circles
BW2 = imclose(BW1,strel('disk',5))

Now I have a binary image of the edges of the objects in the picture. And I want to fill out the pedestrians, such that I have an initial guess of where they are and how they look. So I apply imfill.

[BW3] = imfill(BW2);

Then what I want is the coordinates of all the pixels that matlab have turned white for me. How do I get that? I have tried with [BW3,locations] = infill(BW2), but this does not work (as I want it to.)

As testing you can use the attached picture. Also if you are trying the solve the ultimate problem at the top, then I have no problem of getting the house, the cars and the pedestrians out - the house and the cars I can perfectly fine sort out if they appear whole.