# Moving files?



## frostbytes (Dec 5, 2018)

Because of ongoing problems with Lightroom's folder view (hello Adobe?) I've decided to reorganize the physical folder structure for my hundreds of thousands of photos.

My current folder structure looks like this:

YYYY-MM-DD Shoot Name
x many shoots

I'll be reorganizing to this format:

[2017]
   - YYYY-MM-DD Shoot Name
   - YYYY-MM-DD Shoot Name
   - etc.
[2018]
   - YYYY-MM-DD Shoot Name
   - YYYY-MM-DD Shoot Name
   - etc.
[2019]
   - YYYY-MM-DD Shoot Name
   - YYYY-MM-DD Shoot Name
   - etc.

What's the best way or reorganizing this so that I minimize any downtime with Lightroom?  Do I reorganize the files with Windows Explorer and then click a million times in Lightroom to find missing files or is there a better way?


----------



## Philippe Coudé du Foresto (Dec 5, 2018)

I woul do it inside Lightroom folders panel
- Create the year folders (2016, 2017, 208, etc.)
- Select all the shoot folders of one year, then, with the mouse move them to the corresponding year folder. 

The trick to avoid a lot of manipulation, is to select several folders at a time using the <Shift> key.
It should be very fast, since it won't be a real "move" of the files by the OS, just a change in the disk directory to reflect the new path.


----------



## frostbytes (Dec 5, 2018)

Philippe Coudé du Foresto said:


> I woul do it inside Lightroom folders panel
> - Create the year folders (2016, 2017, 208, etc.)
> - Select all the shoot folders of one year, then, with the mouse move them to the corresponding year folder.
> 
> ...



Thanks for the suggestion.  That would help avoid the tedious task of clicking "find missing folder" hundreds of times.

Is there any downside to move the folders inside of Lightroom?  I have a healthy level of paranoia about the potential for data corruption.


----------



## LouieSherwin (Dec 5, 2018)

frostbytes said:


> Is there any downside to move the folders inside of Lightroom? I have a healthy level of paranoia about the potential for data corruption.



You do not want to move from one hard drive to another hard drive. This is very slow as it requires a copy and delete if each individual image and sidecar if present. Works ok for limited reorganization but will will potentially cause confusion if it is interrupted by say a power failure.

On the other hand when moving folders and/or files in the Folders panel that stay on the same hard drive it is very fast and efficient. Because only the file system folder structure is updated. The individual files are not touched. 

In either case it will cause a new backup of every file that is moved. The backup software does not have the smarts to recognize that only the path has changed. 

-louie


----------



## frostbytes (Dec 5, 2018)

LouieSherwin said:


> You do not want to move from one hard drive to another hard drive. This is very slow as it requires a copy and delete if each individual image and sidecar if present. Works ok for limited reorganization but will will potentially cause confusion if it is interrupted by say a power failure.
> 
> On the other hand when moving folders and/or files in the Folders panel that stay on the same hard drive it is very fast and efficient. Because only the file system folder structure is updated. The individual files are not touched.
> 
> ...



As part of my reorganization, I do want to switch drives (from D to E).   Maybe I should reorganize everything outside of Lightroom first?


----------



## Bristolian (Dec 5, 2018)

I have moved my whole catalogue from one volume, which consisted of two physical hard drives, to another (larger) hard drive from within LR. My catalogue contains around 120,000 images (plus associated xmp files) and although it was a slow process I don't think  any of the files got corrupted - at least none of the ones I have opened so far anyway


----------



## LouieSherwin (Dec 5, 2018)

First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up. 

In case there is any confusion I mean a complete backup of your entire filesystem.

I would organize them into the date folders first. That will go pretty quickly. Then once in the new year folders. Use Explorer or another copy tool to move the selected years to the other hard drive. All you will have to reconnect is each year that you move to the other hard drive.

You may want to consider a copy utility that will validate each file as it is copied. 

-louie


----------



## clee01l (Dec 5, 2018)

frostbytes said:


> As part of my reorganization, I do want to switch drives (from D to E).   Maybe I should reorganize everything outside of Lightroom first?


DON'T reorganize outside of LR.  When moving folders around inside of LR the file path in the catalog is maintained and updated.   If you want to (eventually) move to a different drive.  Do your reorganization in LR first on the current drive, then move whole folder structures either in LR or in Windows Explorer.  If using Windows Explorer ,you of course need to update the file path using the Find missing file/folder.


----------



## Zenon (Dec 5, 2018)

I like that advice and is what I do. I always move folders around within LR unless it is a big move like the parent folder structure. I think when moving stuff around using the OS people have good intentions to re-establish the link but forget which causes problems down the road.


----------



## frostbytes (Dec 5, 2018)

Bristolian said:


> I have moved my whole catalogue from one volume, which consisted of two physical hard drives, to another (larger) hard drive from within LR. My catalogue contains around 120,000 images (plus associated xmp files) and although it was a slow process I don't think  any of the files got corrupted - at least none of the ones I have opened so far anyway



That's promising.  My catalog is substantially larger -- almost 700,000 images.  Unless I know for certain that Lightroom's folder moving functionality contains robust error-checking I'm hesitant to move that much data within Lightroom.

Does anyone here know for sure about error checking?  I read elsewhere that moving large numbers of files inside of Lightroom has been known to cause corruptions.


----------



## frostbytes (Dec 5, 2018)

LouieSherwin said:


> First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.
> 
> In case there is any confusion I mean a complete backup of your entire filesystem.
> 
> ...



I have multiple levels of backups (Drobo devices, cloud, etc.).  I'm too paranoid not to.  

And yes, I do use a third-party utility (Teracopy) to move/copy files, which allows for a verification step.

It sounds like Lightroom would let me using the "find missing folder" on, for instance, the 2017 folder so I wouldn't have to find folders for each folder?  If so, that should be a lot less onerous than clicking hundreds of folders.


----------



## frostbytes (Dec 5, 2018)

clee01l said:


> DON'T reorganize outside of LR.  When moving folders around inside of LR the file path in the catalog is maintained and updated.   If you want to (eventually) move to a different drive.  Do your reorganization in LR first on the current drive, then move whole folder structures either in LR or in Windows Explorer.  If using Windows Explorer ,you of course need to update the file path using the Find missing file/folder.



It sounds like I should break this into two steps:  1) reorganize the folders on drive D, 2) copy the data to drive E and use "find missing folder" on that?


----------



## frostbytes (Dec 5, 2018)

Zenon said:


> I like that advice and is what I do. I always move folders around within LR unless it is a big move like the parent folder structure. I think when moving stuff around using the OS people have good intentions to re-establish the link but forget which causes problems down the road.



This is a big move.  Almost 700,000 photos.


----------



## clee01l (Dec 6, 2018)

frostbytes said:


> It sounds like I should break this into two steps:  1) reorganize the folders on drive D, 2) copy the data to drive E and use "find missing folder" on that?


Yes, That is how I would approach it.  Re-arranging folders on the same drive simply updates pointers in the Master File Table (MFT).  No file blocks get moved.  The physical move from one drive to the other is going to take time (over night might be best) because the filesystem uses a Copy function to the target destination , followed by a data integrity check followed by a Delete operation on the source drive.   If you do this in the LR Folder panel, LR will automatically update to the new path. If you use Windows Explorer, you will need to use the find missing folder (only on the top level folders). Lightroom will use the same filesystem "Move" function as does Windows Explorer.


----------



## PhilBurton (Dec 6, 2018)

LouieSherwin said:


> First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.



What Louie said.



> You may want to consider a copy utility that will validate each file as it is copied.
> 
> -louie


Doesn't a copy utility operate OUTSIDE of Lightroom?  Isn't it a NO-NO to move files outside of Lightroom?

Phil


----------



## PhilBurton (Dec 6, 2018)

frostbytes said:


> Because of ongoing problems with Lightroom's folder view (hello Adobe?) I've decided to reorganize the physical folder structure for my hundreds of thousands of photos.
> 
> My current folder structure looks like this:
> 
> ...


Probably a dumb question.

If you have top level folders by year, then why do you have second-level folders whose names include YYYY?  Why not just MM-DD Shoot Name?

My original file structure was:  YYYY/YYYY-MM/YYYY-MM-DD.  Pretty soon I realized how cumbersome that was, and it added unneeded characters to the file path.  So I simplified down to YYYY/MM/DD.  Never looked back.  Never any urges to go back to the original schema.

Phil


----------



## clee01l (Dec 6, 2018)

PhilBurton said:


> If you have top level folders by year, then why do you have second-level folders whose names include YYYY? Why not just MM-DD Shoot Name?


Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA.   The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).


----------



## PhilBurton (Dec 6, 2018)

clee01l said:


> Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA.   The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).


That would also be me, but there is the countervailing force called "common sense."  Which sometimes actually gets me to change my ideas.  Sometimes.


----------



## msmack (Dec 6, 2018)

I have to agree that the YYYY is not necessary as the Parent in that group is the year.  I feel the less numbers I have to look at, the easier it is to see something.


----------



## LouieSherwin (Dec 6, 2018)

PhilBurton said:


> Doesn't a copy utility operate OUTSIDE of Lightroom?  Isn't it a NO-NO to move files outside of Lightroom?
> 
> Phil



Sometimes it makes sense to do the copy outside of Lightroom even though you have to go back into Lightroom to tell it what you did. The case in point is that of moving a large collection of images from one hard drive to another.  While Lightroom will do this and keep track of where the images are now stored it is faster and I suggest safer to make a large move using an external utility and preferably one that will verify the files as they are copied. 

What is being recommended here is to first reorganize the shoot folders in to top level years within Lightroom. Then selective copy some of these top level year folders to the new external hard drive. Once the copy is complete use the "Update Folder Location" to point Lightroom to the folder on the new hard drive.

Where people get into trouble is by going into Finder or Explorer and deciding that they want to move things around. Then when going back to Lightroom  it is a big mess and be a real challenge to reconstruct  what was done externally. 

-louie


----------



## frostbytes (Dec 6, 2018)

PhilBurton said:


> Doesn't a copy utility operate OUTSIDE of Lightroom?  Isn't it a NO-NO to move files outside of Lightroom?



It sounds like a split card between people who think you shouldn't and those who think it's okay.


----------



## frostbytes (Dec 6, 2018)

PhilBurton said:


> Probably a dumb question.
> 
> If you have top level folders by year, then why do you have second-level folders whose names include YYYY?  Why not just MM-DD Shoot Name?
> 
> My original file structure was:  YYYY/YYYY-MM/YYYY-MM-DD.  Pretty soon I realized how cumbersome that was, and it added unneeded characters to the file path.  So I simplified down to YYYY/MM/DD.  Never looked back.  Never any urges to go back to the original schema.



I like having the folders containing photos complete with year.  That's how I've been doing it for years and that's worked well for me -- at least until the "bug" with Lightroom.  It might be redundant to have YYYY > YYYY-MM-DD but that also makes the folder name independent of high-level folders and more flexible in the event I switch away from Lightroom.


----------



## frostbytes (Dec 6, 2018)

clee01l said:


> Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA.   The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).



It wouldn't be too onerous to rename all the folders using a file naming utility but I'd rather not drop the YYYY from the folder name.  I'm anal like that too.


----------



## Hal P Anderson (Dec 6, 2018)

frostbytes said:


> It wouldn't be too onerous to rename all the folders using a file naming utility



Renaming in bulk is incredibly hard to recover from unless you do it within Lightroom. Moving a whole directory structure outside of Lightroom can be recovered from with a couple of clicks


----------



## clee01l (Dec 7, 2018)

frostbytes said:


> It wouldn't be too onerous to rename all the folders using a file naming utility but I'd rather not drop the YYYY from the folder name.  I'm anal like that too.


As soon as you rename outside of LR, you then lose the internal reference to the path and have to manually find these folders again in LR.   There is no automated renaming utility that works inside Lightroom


----------



## frostbytes (Dec 5, 2018)

Because of ongoing problems with Lightroom's folder view (hello Adobe?) I've decided to reorganize the physical folder structure for my hundreds of thousands of photos.

My current folder structure looks like this:

YYYY-MM-DD Shoot Name
x many shoots

I'll be reorganizing to this format:

[2017]
   - YYYY-MM-DD Shoot Name
   - YYYY-MM-DD Shoot Name
   - etc.
[2018]
   - YYYY-MM-DD Shoot Name
   - YYYY-MM-DD Shoot Name
   - etc.
[2019]
   - YYYY-MM-DD Shoot Name
   - YYYY-MM-DD Shoot Name
   - etc.

What's the best way or reorganizing this so that I minimize any downtime with Lightroom?  Do I reorganize the files with Windows Explorer and then click a million times in Lightroom to find missing files or is there a better way?


----------



## PhilBurton (Dec 7, 2018)

Hal P Anderson said:


> Renaming in bulk is incredibly hard to recover from unless you do it within Lightroom. Moving a whole directory structure outside of Lightroom can be recovered from with a couple of clicks


Amen brother.


----------



## PhilBurton (Dec 7, 2018)

clee01l said:


> As soon as you rename outside of LR, you then lose the internal reference to the path and have to manually find these folders again in LR.   There is no automated renaming utility that works inside Lightroom


yes.  I've discovered that if I want to rename files in bulk, in my case including YYYYMMDD and a sequence number in the file name, it's best to do that rename before *importing *into Lightroom.  I currently use Downloader Pro, but it doesn't handle XMP files, which means I can't use it after I cull photos with Fast RAW Previewer.

Phil Burton


----------



## Philippe Coudé du Foresto (Dec 7, 2018)

> There is no automated renaming utility that works inside Lightroom


More precisely "_There is no automated *folder* renaming utility that works inside Lightroom_" . Such utility exists in LR for filename.


----------



## PhilBurton (Dec 8, 2018)

Philippe Coudé du Foresto said:


> More precisely "_There is no automated *folder* renaming utility that works inside Lightroom_" . Such utility exists in LR for filename.


Phillippe,

What is the name of the utility for file renaming inside LR?

Phil Burton


----------



## Johan Elzenga (Dec 8, 2018)

PhilBurton said:


> Phillippe,
> 
> What is the name of the utility for file renaming inside LR?
> 
> Phil Burton


The name is ‘Lightroom Classic CC’. 
Seriously; it’s not a ‘utility’ (which suggests something separate), it’s a Lightroom menu.


----------



## Philippe Coudé du Foresto (Dec 8, 2018)

More precisely, it's in the menu Library>Rename photos. It can also be called by pressing the key <F2>. 
There, you can define a rule for renaming, using text as well as metadata (like capture date) and/or counters. You can save the rule and apply it to a group of selected photos.


----------



## PhilBurton (Dec 8, 2018)

Philippe Coudé du Foresto said:


> More precisely, it's in the menu Library>Rename photos. It can also be called by pressing the key <F2>.
> There, you can define a rule for renaming, using text as well as metadata (like capture date) and/or counters. You can save the rule and apply it to a group of selected photos.


I am aware of that feature.  However, there is no automatic "reset to 1 on new date" for the counter name option.  That's why I was hoping there was a plug-in that had that option.

Phil


----------



## Philippe Coudé du Foresto (Dec 9, 2018)

Just out of curiosity, why do you need to start the counter at 1 for each day ?


----------



## PhilBurton (Dec 10, 2018)

Philippe Coudé du Foresto said:


> Just out of curiosity, why do you need to start the counter at 1 for each day ?


Good question that deserves a good answer.  I started to use this naming scheme:  YYYYMMDD-seqn, with the Adobe-created seqn.  Then I got some comments about how weird it was to have something like 20180915-4824.  As in, "What happened to the first 4823? "  The request was for a seqn that started with 1 each day.  Unfortunately, Adobe's naming scheme doesn't have a "reset to 0001 on date change" option, unlike something like Downloader Pro - What's new in version 2 that will populate a folder with new imports before starting Lightroom.  

One implication of this Adobe lack is that if I delete some images after import, and I want to renumber to eliminate gaps, I have to manually reset the seqn to 1 each time.


----------



## Tom75 (Dec 10, 2018)

Hi frostbytes / all,

I am following this thread because I am also about to move and reorganize "some" images soon.

[2017]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.

I have used the above structure you are intending to use for quite many years and it worked technically very well for me.
I am generally a fan or chronological systems  such as the above however I am planning to re-organize everything due to the fact that after years it is difficult to find things because if you are looking for something specific you always needc to know the year.

I admit I should have listened to Scott Kelby some years ago but I thought a chronological system is perfect but its not I realized.

Therefore I am planning to Change to a category based system such as the below.

[Family]
- YYYY-MM-DD Shoot Name

[Landscape]
- YYYY-MM-DD Shoot Name

[Travel]
- YYYY-MM-DD Shoot Name

[Sports]
- YYYY-MM-DD Shoot Name

etc.

In the main category folders such as family there will also be sub category folders such as for family members

I think this will make it much easier to find stuff because you always know the category where one shoot should be located.
Any opinions about this system?

Regards,
Tom


----------



## Philippe Coudé du Foresto (Dec 10, 2018)

Since you're using LR, you have all the search tools you need to find your images. The OS folder system is there to manage computer files, whereas LR is there to manage images. 
Imagine you have a picture of one of your family's member taken during a travel. You then think it's also a good portrait and decide to make a virtual copy to have a specific developpement (crop, for exemple) as a portrait. So you have only one file, but two images. With your folder system, how will you find this portrait ? Within LR, it's simple to have 2 collections, one for the travel and one for the portrait. You don't mind if the images comes from the same or from different computer files.
Another reason to use date organisation, is that it allows a totally automtized (via import preset) import of the files, which garantee to always have the same organisation. Having a manual intervention on import might sooner or later make a mistake which will lead to files being imported in the wrong place.
However, I recognize that having a comment in the folder name might sometimes be helpful, espiecially when looking at them outside of LR (which I don't do very often !). Hence my organization :

- The files are imported in a folder named YYYY\YYYY-MM-DD (automated via import preset, so no risk of error)
- I then rename inside LR the folder to add a comment while keeping the date structure, something like "YYYY\YYYY-MM-DD Aunt anniversary"
- If the event is spread over several days (like a travel), I move all the photos to the first day folder (which I've renamed using the above convention) and suppress all the other folders.
This takes only one or 2 mouse clicks, but garantee (since automated) a coherent and easy import.

I then use keywords, IPTC and collections to organize, comment and make searchable all the images, which is very fast and easy in LR. One of the advantage of the keywords is that they can be exported with the final image, and are compatible whith any immage magement tool. This means that when I send the photo to someone else, he wil get the keywords and IPTC and can use them to organize and search in his tool.


----------



## johnbeardy (Dec 10, 2018)

Assuming you don't live in Paris, Tom, into which folder will you put a wide angle photo of the city which includes a member of the family, though not very obviously? Into Travel, Landscape, or Family? It could fit any three, so do you want to spend time moving stuff around? What if you only notice the family member later, after deciding the picture belongs in Travel? What if you later decide to add Street or Urban as top level folders? So you move stuff around again, meaning that your backup no longer reflects where files are?

These are downfalls of using folders to subjectively categorize your pictures, and they really aren't flexible enough for all the ways we might think of or categorize pictures. So:

Folders for physical storage - YYYY-MM-DD Shoot Name  is fine - date is objective. Leave the photos where they are.
Keywords and other metadata for categorization - can be as fluid as you want
Collections for grouping and gathering - arbitrary and useful
So I encourage you not to put your time into moving stuff around - put it into adding keywords and other metadata.

John


----------



## Tom75 (Dec 10, 2018)

Hi and thanks for your coomments.

Philippe: I fully agree with you, I have my folder structure like this 
[2017]
- YYYY-MM-DD Shoot Name 

...and I intend to keep it like that for the physical location of my pictures on the disc. I am sorry that I forgot to mention that my re-organzing plan with categories is what I want to do with my collections structure.

@John: This Paris picture would for me go into travel since it was obviously a trip or a holiday there. BUT it might well be that I take a a nice portrait of one person on this trip so these I could put as virtual copies also under this persons folder., meaning there are 2 possiblilities to find this when I am looking for it and I will find it in both of them.

Basically the reason why I want to do this job to re-organize my collections with this system is because it happened quite many times that I was looking for something I had to go through many of the year folders to find the shoot I was looking for.

Regards,
Tom


----------



## johnbeardy (Dec 10, 2018)

How can you put the VC into two folders, Tom?


----------



## Tom75 (Dec 10, 2018)

not into 2 folders but into 2 collections


----------



## johnbeardy (Dec 10, 2018)

Which is why one should not put effort into folder structures that categorize your photos. Save that time and put it into keywords and other metadata, and then into collections which make use of these tags. You don't even need VCs for this .


----------



## Tom75 (Dec 10, 2018)

hmm, good point, what kind of collections structure do you have?


----------



## johnbeardy (Dec 10, 2018)

Not very different from the folder system you described in your post 35, Tom. It varies over time, so this year I added a top level collection set "London Walks " when I started doing long daily walks, and this contains a mix of smart collections (eg City Contains Dulwich) to gather everything from the area, and dumb collections such as Dulwich Park, which is a "best of" for one of the projects I have been working on. 

But the key approach is folders for storage, keywords and other metadata to describe and categorise, and then collections to collect or gather pictures for however I want to use them.


----------

