# Face Recognition Oddities



## Linwood Ferguson (Jun 6, 2020)

As I digitize old photos I've been reviewing thousands and doing facial recognition.

There are a couple of things that keep happening to me, and wondering if they are all real, or in some cases if I may have some oddity in preferences or keyboard or such. 

Oddity #1:   Sort of suggested names

The sort order of suggested names seems more suggestion than rule.  Here is a folder I just scanned, and have entered nothing.  The setting (see bottom) is to sort by suggested names, but look at the names.  Not only is the order wrong, but they are not grouped together.  I think they are in image order.  Now if I do something (exactly what seems to depend on the weather) it may snap to the right order, then later depart again.







Oddity #2: Search-as-Type Restarts after space

This is perhaps the most annoying of the features.  Let's say I have a bunch of Tom's in my keywords, and I start typing (it doesn't matter if this on the loup-with-face-regions or the suggested name area: 





The natural thing for me is to type a couple more characters, say it was Snyder I wanted, so I would hit space and S.  But if I do, here's what happens: 





It restarts the lookup again entirely.  So instead of Tom Snyder, I create a new entry for Tom Sara Duke if I'm not really careful. 


Oddity #3: Approving with corrections

So consider this one.    The two shown are not the same person but pretend.   If I click on the big check mark, what should happen?   Note both faces are selected.   I'll give you a hint -- it accepts them as two separate names as shown.   Now however.....





Note I was unable to get back to the exact same situation even by doing a complete find again -- apparently there's a lot of randomness involved. 

Here's a similar situation, except instead of checking the X, I just clicked on the name.  I did not change the name, just clicked on it.  Now if I hit enter both of these faces become Jack Hilliard.

Now I GET what it is doing, it is assuming any keystroke is corrective.  I can argue it's even not a bad thing.  But to me it's just too similar a process with completely opposite results.  I've trained myself to work through it.  But the problem comes in combination with oddity #1 -- the sorting.  On a large gallery I will see 2 or 3 rows all of one person, the current person.  I'll highlight them all and click the check box. But.... wonder of wonders there's a different name in the middle, becuase it's not in order.  Suddenly my cousin becomes my unclue, or becomes a cheerleader from basketball photos. 







Is it just me with these oddities?   Is anyone using face recognition enough you have hit them?   Perhaps the most important question is if it is not doing these weird things for you, I wonder if I can fix mine and how? 

PS. Virtually none of the names are actually right, so hopefully no one minds me posting random person X's face with random name Y.


----------



## PhilBurton (Jun 8, 2020)

Ferguson,

I haven't enough experience with Facial Recognition to actually answer your questions.  But in my limited experience so far, Lightroom Facial Recognition seems woefully inadequate.  It identifies many different "strangers" as family members, even though I have lots of photos already identified as family members.  It often identifies men as women, and women and men.  In a word, it is a very labor-intensive process to get right.  

Is anyone aware of a plug-in or even an external program that can do a better job than Lightroom?


----------



## Linwood Ferguson (Jun 8, 2020)

Phil, you are quite correct, it is not very good at recognition.  Though I have no experience really to know what the scale of "good" is.

To me it's biggest failure is in false negatives -- it misses faces entirely.  The normal mode of operation is to let it find faces, then work to identify them.  It is quite tedious to go through and check that it found all the faces.

I've not got thousands and thousands of photos and faces done, and while its suggestions may be weak, its biggest strength is organization and automation.  At something like a party, I can take a hundred photos and it will distill them down into maybe 100 faces.  Even if I ignore the guesses entirely, it is fairly quick and easy to click on each "Sally" in the list, and then enter "Sally".  Much, much easier than doing it by photos as opposed to faces.  I can also usually quickly see people I do not recognize or need to code, and en-mass remove their face regions entirely.

So I find it a very useful tool.  But it seriously needs some TLC from Adobe to clean up the quirks, and to me the big failing of missing faces entirely. 

I would also like to know if there are better alternatives.


----------



## PhilBurton (Jun 9, 2020)

Ferguson

Thanks for confirming that it's a Lightroom issue, and not an issue of the people I photo.

It sounds like you are using an "alternate" way to do Facial Recognition.  I always use the keyword "current" to isolate just the folder or collection that I'm currently processing, and I select All Photos in the smart collection (Thanks John B!) that selects for keyword "current."  Is that what you are doing, or something different.

I wasn't aware that Facial Recognition actually misses faces, but that is a big fail.

From many threads I have read on the Adobe Lightroom forum, the entire LIBRARY module needs some TLC.  Or maybe a lot of TLC.

Phil


----------



## Linwood Ferguson (Jun 9, 2020)

PhilBurton said:


> It sounds like you are using an "alternate" way to do Facial Recognition.  I always use the keyword "current" to isolate just the folder or collection that I'm currently processing, and I select All Photos in the smart collection (Thanks John B!) that selects for keyword "current."  Is that what you are doing, or something different.



I'm confused, that seems complicated. 

I keep face recognition paused (top left). 

That way the only times it runs is if I click on the face emblem in the toolbar (whatever you call that thing  icon that is beside grid/loup/side-by-side/survey).  

So if I have a bunch of photos to folder 20200609, and display those in the grid and click on that, it runs face recognition right then on that folder (only).

So when I go out and shoot basketball or baseball I just never click that icon.

If I shoot a family birthday party, after import at some point I click it, and do the face recognition thing.

 I can do the same with any grouping - published collection, collection, folder, etc.    I can even just use the metadata search at the top and select by date or keyword of any sort.  once the grid shows that subset, if I click the face emblem it does just those photos. 

Is your "current' the same idea?   I'm unclear why you bother with a keyword for it. 

But yes, missed face regions is really my biggest complaint.   Here's a simple example.   In this photo it found the guy, but missed the ladey entirely.  It seems weakest in profiles, and also in faces that are very large.  If you have a real closeup of a face and people out of focus way in the background, it will often pick up the background and miss the great big face. 





Here's an even better example, two photos almost the same, it found the lady in the first, missed her in the second, but found her also in the third which surprises me considering the partial face occlusion. 









To me it's just weird that it can find this third one, and not the second.  Note I am not saying it guessed wrong, it just did not think there was a face at all in the second shot. 

I just ran through this entire model shoot.  It missed maybe 10 of 80 shots (eyeball estimate), and with no apparent rhyme or reason.  I have a lot of her riding in profile it found.  I have a couple with her completely blacked out with the sun behind her and it found those. 

If you REALLY want to make sure you get all the faces, there is no alternative to, once you are done, going through in Loupe and seeing who isn't tagged. 

So while I'm getting thousands of faces tagged, I expect that I am missing hundreds if not thousands as well.


----------



## PhilBurton (Jun 15, 2020)

Ferguson,

Sorry for the late reply.  

Just wondering if Facial Recognition  picks up the horse.    It is distressing to read that sometimes Facial Recognition doesn't recognize a face.  That makes me very open to a non-LR solution for F/R, as long as it can be made compatible with LR keywording.

About my use of the workflow word current.  Sometimes I have to stop work on one folder/subfolder/subcollection, and switch over to another group of photos.  The keyword current is part of all my workflow smart collection rules, so that the range coverage is confined to just my working set.  It works for me, and my current usage of LR.  Here is an example:


----------



## tspear (Jun 15, 2020)

@PhilBurton

I have tried to replace Lr or integrate alternatives either with or without Lr. As much as I dislike Lr facial recognition and a few other aspects of the application, the reality is even though I take a lot of family pics with lots of people in them, nothing else comes close to pulling together in a single workflow all the parts needed.  Therefore, I am pretty much forced into reviewing each image and manually doing the facial tagging; Lr really does not save me time in this manor.
I keep hoping though....


----------



## Linwood Ferguson (Jun 15, 2020)

PhilBurton said:


> Just wondering if Facial Recognition  picks up the horse.



No, and so far almost never birds or wildlife, but my wife took a bazillion shots while in Italy and I think it found every tiny statue's face no matter how far in the background. And it didn't correctly identify any of them.  

While we are on the subject, I found something I knew about, but found to be quite helpful -- if you drill to all the images of a person, underneath is a "similar" grouping, which I found much more strongly tied to finding that person than the suggested names on the main page.  I got a lot more matches going to each person individually then looking for "find similar" for the first few rows. 

The biggest problem I'm struggling with is I am sort of face blind.  I tried doing tennis last night, and despite having a photographic roster, I found I could identify only 2 of the people reliably -- not LR, me.   And women's sports -- over 4 years, I swear half of them changed hair color not just style. 

Which brings up something that I really wish existed... I am sure for some sports I could get the school to do the ID's for me.  Except -- how?  There's just no good mechanism, even pre-Covid.  I really wish Adobe had an associated cloud just for face tagging -- upload a bunch of shots, let people interact with it to tag (password protected or whatever), and be able to pull that data back down.  Heck, I bet the schools would pay actual money for it, they must waste a lot of time as they get requests like "hey, Joe Smith graduated in 2013 and just signed with Spain -- got any photos?)".


----------



## tspear (Jun 15, 2020)

@Ferguson

Expect for manually pulling the data back down. There is a cloud service for that. Two of them. FaceBook for older, and Sanpchat for the younger generations.


----------



## Linwood Ferguson (Jun 15, 2020)

tspear said:


> @Ferguson
> 
> Expect for manually pulling the data back down. There is a cloud service for that. Two of them. FaceBook for older, and Sanpchat for the younger generations.



Well, that's for public or mostly public use, where they benefit from it, and the people tagged (I assume) know you tag them.  

I'm talking about as a service specifically to photographers, like sports or photojournalists, event photographers.


----------



## PhilBurton (Jun 17, 2020)

tspear said:


> @PhilBurton
> 
> I have tried to replace Lr or integrate alternatives either with or without Lr. As much as I dislike Lr facial recognition and a few other aspects of the application, the reality is even though I take a lot of family pics with lots of people in them, nothing else comes close to pulling together in a single workflow all the parts needed.  Therefore, I am pretty much forced into reviewing each image and manually doing the facial tagging; Lr really does not save me time in this manor.
> I keep hoping though....


Tom,

I wish some someone would write a plug-in that integrates (Google/Facebook/other service/???????) into Lightroom's overall workflow.  That said, (1) I don't expect that such a plug-in will be available any time soon, and (2) I'm willing to use something else as a "pre-processor" that adds keywords to RAW images prior to Lightroom import.  Specifically I sometimes use Fast Raw Viewer, which has a so-so interface, but does allow me to star-rate/cull photos before they are imported by Lightroom.  That is useful "pre-processing" functionality.

As much as I don't like the Facial Recognition now, it just annoys me to know that I really should be reviewing all photos manually.  Is there any way to view all photos in Grid View to see which faces were missed by Lightroom?

Phil Burton


----------



## Linwood Ferguson (Jun 17, 2020)

PhilBurton said:


> As much as I don't like the Facial Recognition now, it just annoys me to know that I really should be reviewing all photos manually.  Is there any way to view all photos in Grid View to see which faces were missed by Lightroom?


Only indirectly in that you can look at the keyword panel and go photo by photo and see what names appear.  It's quicker to go into loupe and go through that way and look at the actual regions, plus then you can just draw in any missing.

It's not horrible to do if you come back with 60-80 shots.  It's truly terrible when you realize you have 80,000 or so and have not been doing it and probably should have.  

I'm sure I'm the only one in that situation though.


----------



## LRList001 (Jun 18, 2020)

Ferguson said:


> Well, that's for public or mostly public use, where they benefit from it, and the people tagged (I assume) know you tag them.
> 
> I'm talking about as a service specifically to photographers, like sports or photojournalists, event photographers.



Would a NAS photo app service work?  Looking at Synology, it might.  I have no experience with this.  The following suggests that both public access and commenting is possible with their photo app

https://www.synology.com/en-global/knowledgebase/DSM/help/PhotoStation/sharing


----------



## Linwood Ferguson (Jun 18, 2020)

I would never, ever enable public access to a NAS drive in my home.  Sorry, I do computing for a living not photography.  Never open ports on your firewall to the outside.

That said, I didn't see anything about face recognition.


----------



## Dale Strumpell (Jun 18, 2020)

PhilBurton said:


> Tom,
> 
> I wish some someone would write a plug-in that integrates (Google/Facebook/other service/???????) into Lightroom's overall workflow.  That said, (1) I don't expect that such a plug-in will be available any time soon, and (2) I'm willing to use something else as a "pre-processor" that adds keywords to RAW images prior to Lightroom import.  Specifically I sometimes use Fast Raw Viewer, which has a so-so interface, but does allow me to star-rate/cull photos before they are imported by Lightroom.  That is useful "pre-processing" functionality.
> 
> ...


John Rellis has made the Any Vision plugin: Any Vision Lightroom Plugin

"Any Vision uses Google Cloud Vision, the state-of-the-art machine-learning technology underlying Google image search. Any Vision automatically tags your photos with labels, landmarks, logos, face expressions, and dominant colors and extracts embedded text (OCR). You can search these tags in Lightroom, making it much easier to find photos in large catalogs. You can export the tags in photo metadata as keywords and GPS locations or in comma-separated text files."


----------



## PhilBurton (Jun 19, 2020)

Ferguson said:


> Only indirectly in that you can look at the keyword panel and go photo by photo and see what names appear.  It's quicker to go into loupe and go through that way and look at the actual regions, plus then you can just draw in any missing.
> 
> It's not horrible to do if you come back with 60-80 shots.  It's truly terrible when you realize you have 80,000 or so and have not been doing it and probably should have.
> 
> I'm sure I'm the only one in that situation though.


No you are not.  This should be a high priority for "Mother Adobe" to fix.


----------

