Tracking imported files across moves/renames

Channels DVR Server Feature Requests
#1

The current scheme for tracking labels and collections seems to be based on absolute paths. This is just horrifying!!!
Twice now I have lost an extensively curated set of labels and collections because I either renamed a folder or needed to move files to a larger drive.

Can we not make this more robust?
NTFS, APFS and ext4 all provide extended attrobutes, and it seems like Channels should be able to store a GUID in this metadata that will then persist even across file copying to a new drive. This GUID can be accessed every time a new file is discovered, then looked up in a Channels database, from which all the rest of the persistence (like Labels and Collections) can be derived.
(There are alternative solutions like storing the label and a collection ID in the xattr's, but storing a GUID seems like the most robust solution in terms of only requiring a small amount of xattr storage, and being most extensible/controllable on the Channels DVR side.)

Even if a solution cannot be found to work for ALL file systems (maybe not some of the networked ones?) at least a solution that covers most users is better than nothing.

Because right now all the fancy UI ordering/presentation features are useless to me :frowning:
Why would I want to create labels and collections YET AGAIN, when I know that it's just a matter of time before some minor mistake (renaming a folder because I forgot that's not allowed, or being forced to a larger hard drive) means I lose all that work?

1 Like
#2

This is incorrect.

You didn't lose your edits, you lost the entire entry for these video files, because you changed their locations on your file system.

What probably happened is, you edited their locations, Channels DVR Server saw them as NEW files, imported them as new entries in the library, and then pruned out the old entries in its library because the files were gone.

Channels DVR Server use it's own database to track your files. You can't just move them and expect it to know. I understand your proposed solution, but it's not something we'll be implementing.

Just know if you mess around with the files in your filesystem after you've imported them into Channels DVR Server, you'll have problems.

1 Like
#3

Hi John,

But it boils down to the same thing, doesn't it?
Meaning is: I cannot move files after I have "placed" them in my Library? Whether I want to change the directory layout, or move the files to a larger drive, I am simply SOL?

#4

Essentially. This is a limitation of how Channels DVR Server works. The same thing would happen across any library based file indexing app.

If you moved your photos around, a photo indexing app would lose track of them as well.

#5

Sure. But doesn't that simply remind us that "The Easy Way is Rarely the Best Way"?
There ARE mechanisms to solve this sort of thing, mechanisms that are even used by at least some apps.
Channels seems to want to have it both ways, providing the flexibility of a multi-location library and the implication that you can move your data wherever you want, split it up however you want; but not doing the work required to adhere to that promise.

I understand that you want more flexibility than just say iTunes or iPhoto, with their opaque single Library. But those apps are very clear on the user contract:

  • they control that single library
  • user does not modify it in any way
  • there is ONE way to movie/grow it, by copying the opaque contents to a larger drive, then changing the Library path in the app preferences.

Where Channels goes wrong (IMHO) is it does not make that contract clear. Everything about how user libraries is presented suggested (at least to me) that this was a mix-and-match, order the files as you like deal, NOT an "opaque library that you never touch" deal. For example, movies are imported by modifying those folders in the file system, NOT by using some sort of in-app "Import movie" command. iPhoto or iTunes do not operate on that model of having the user modify the file system.

Now don't get me wrong. I appreciate the value of providing multiple "sub-library" locations; and I appreciate that it makes many things very easy and fast if one can import via the file system. (In-app alternatives can be tremendously clumsy if implemented poorly; for example there is no way I know of to bulk select a whole lot of movies and mark them all as Kids, or to give them all a particular label...)
I think the solution is not to make the user libraries opaque (like iTunes or iPhoto) but to make them ROBUST (via the sort of mechanism I suggested).

#6

You can move Channels DVR recorded video, movies or TV, into additional DVR Database storage paths. Just add the new storage path drive on the Web UI. All Channels requires is that the file name is not modified. I have 16 disk drives listed under additional stage paths.

For content that I hope to keep for an extended period of time, I add a new drive, copy the content on the that drive. Channels DVR places the newly added drive at the top of the list and will look for the video first on the recording drive and then next on that newest added drive and so on until the last addition storage drive. The important part is the recorded file name for movies cannot change and the recorded name and folder (director structure) for TV shows must match.

I even remove older drives and keep them on the list of additional storage paths. If a newer drive fails, I can mount the older drive and all is well. The database stores the information about the video including any edited metadata. I never use a prune command. If I want to delete a TV show season or movie, I do it one movie at a time, individual show episode, or the entire show series using the iOS app.

The developers have created a remarkably resilience application. I sometimes wish for more documentation. The Channels Community is great at helping others. If you outline your plans first on the Community, you can save more than a few grey hairs.

2 Likes
#7

You have simply repeated what I already said.
I am not denying that the system works IF you use it the way you describe.

But the way you describe is NOT ROBUST. It relies on the user KNOWING that certain manipulations of the file system are allowed while different manipulations of the file system are not allowed. I'm happy that you found those limitations natural, never transgressed them, and never felt a desire to re-arrange your file system; but that doesn't change the fact that other people do not find those limitations natural, and have (repeatedly) transgressed them.

#8

I apologize for my reply. I had hoped to help. Channels works like many applications. Even my Mac OS can be crippled by a user renaming operating system files.

#9

I found it informative.

#10

While it would be nice if Channels DVR could do this, I understand why it doesn't.

I'm curious why someone would import local content knowing it depends on folder and file names, then rename or move the local source files after they're imported to the Channels DVR database.

Does Plex support that?

What happens when you move or rename a folder or file while the imported content is being played?

#11

Why would I assume that Channels depends on folder and file names? Iā€™m a SW engineer; I know alternative, robust ways to design the system. Iā€™m familiar with concepts lik GUID and inode number.
So would I assume that the system was designed fragile?

#12

I'm not trying to be cantankerous or difficult here. A HUGE problem with software is ensuring that the developer and the user have the same mental model of how the system works. Even Apple occasionally gets this wrong. It's extremely easy, as a developer, or as a user from a different community, to just assume that of course everyone envisages the world the same way they do.

This is why I file reports like this and try to push the issue ā€“ precisely because a fish doesn't know about water until a bird explains it to him (and even then, multiple explanations may be required). You imagine it's obvious that file paths should never be modified; I, on the other hand, having spent my professional life making hard things look easy, ask why that should be a limitation, when it's not even a very difficult problem to remove the limitation.

#13

Actually this is not true.
Firstly I do not believe that it is now possible to cripple your mac by moving files. The essential files are part of the sealed system volume and you simply cannot touch that. Similarly (FWIW) you'll have great difficulty modifying files on a Time Machine Volume.

Secondly, Apple's recommendation for how you to store a persistent reference to a file is to use bookmarks, as in:

A bookmark will continue to track a file that is moved on a disk, though it's not perfect because it uses something like an inode ID so that tracking can break when copied to a different volume. That's why I suggest an app like Channels attach a GUID to the file via an xattr; that will persist across all copyings (even to alternative file systems as long as they are reasonably modern file systems), and track that as well as a bookmark.

Such a GUID solution is (IMHO) feasible even for the problem Apple tries to solve using bookmarks/aliases, and should be implemented at some point. (Basically along with the GUID for each file [and perhaps also folder] you want a per-volume database matching each on-volume GUID to an inode, so that an alias lookup that has failed by path, then by inode, can query each volume [including possibly Time Machine...] by GUID.
Still not a total solution because of the possibility of duplicating files and then modifying the duplicates in different ways! But able to solve 90% of the problems that remain with the current bookmark/alias solution.

#14

I understand what you're saying, just can't see the use case for someone moving/renaming folders/files after importing them.

Maybe because I used Plex for a year and was used to using filebot to structure the paths and filenames before importing. I then started using the Channels DVR local content imports when they were introduced and read the documentation/topics on how it worked.

I did some folder moves for some imports, but used samba mounts so Channels DVR always sees the same paths. I know others have used symlinks.

#15

Thanks for the feedback. Moving your files after you have imported them will break the record in the library. There is no plans to make this work differently.

1 Like
#16