Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add native metadata storage for macOS hosts. #1813

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

NJRoadfan
Copy link
Contributor

-Store all extended attributes without the user. prefix on macOS. This will orphan Netatalk metadata created on past versions, but is needed for native extended attributes to work properly on shares. Unlike Linux, macOS doesn't prefix user created extended attributes with user..

-Store resource forks natively by using the /..namedfork/rsrc path instead of creating AppleDouble files. No need for separate files since the file system supports it natively.

-Synchronize native FinderInfo (stored in com.apple.FinderInfo) with Netatalk. Any changes made to FinderInfo by Netatalk clients or by the host OS are now visible to each other. Note that the contents of com.apple.FinderInfo take priority of that over what is stored in Netatalk's extended attributes.

@NJRoadfan
Copy link
Contributor Author

Marked draft because this will need some testing. To see the power of this integration, download some Stuff-It archives on your macOS host and then extract them using The Unarchiver to your Netatalk share. Open up the folder in your share, and you should see your extracted files complete with their resource forks and proper file types/creator.

@rdmark
Copy link
Member

rdmark commented Dec 11, 2024

Would it be feasible to automatically convert the orphaned meta data to the new format, akin to how AppleDouble v2 is converted to EA when detected?

@NJRoadfan
Copy link
Contributor Author

Netatalk re-creates a new org.netatalk.Metadata EA from the CNID database. Any Finder Info would auto populate from com.apple.FinderInfo at that point.

@rdmark
Copy link
Member

rdmark commented Dec 11, 2024

Aha, so the orphaning process just means that there will be junk metadata files floating around on the host file system, correct?

If so, we should provide some cleanup facility. If not built into afpd, then as a remark in the release notes, and perhaps a simple shell script.

@NJRoadfan
Copy link
Contributor Author

The command xattr -d -r user.org.netatalk.Metadata * run at the root of a share should work.

@NJRoadfan NJRoadfan force-pushed the njroadfan-native-macos-metadata branch 4 times, most recently from 07f87de to 0109bab Compare December 15, 2024 20:48
@NJRoadfan
Copy link
Contributor Author

Test suite has been modified to reflect changes in Netatalk under macOS. I haven't tested it, but it should run without issue.

-Store all extended attributes without the `user.` prefix on macOS. This will orphan Netatalk metadata created on past versions, but is needed for native extended attributes to work properly on shares. Unlike Linux, macOS doesn't prefix user created extended attributes with `user.`.

-Store resource forks natively by using the `/..namedfork/rsrc` path instead of creating AppleDouble files. No need for separate files since the file system supports it natively.

-Synchronize native FinderInfo (stored in `com.apple.FinderInfo`) with Netatalk. Any changes made to FinderInfo by Netatalk clients or by the host OS are now visible to each other. Note that the contents of `com.apple.FinderInfo` take priority of that over what is stored in Netatalk's extended attributes.

-Fix testsuite for native resource fork handling.
@NJRoadfan NJRoadfan force-pushed the njroadfan-native-macos-metadata branch from 0109bab to 5363af3 Compare December 16, 2024 02:36
@NJRoadfan
Copy link
Contributor Author

Extended attributes should be working for macOS clients now. There is a subtle difference in how the getxattr() functions work under macOS. They will not return the size of an EA unless you point to a null buffer when calling. Otherwise the function fails if the buffer passed is too small. Linux doesn't care in these cases and will always return the EA size if it exists.

@rdmark
Copy link
Member

rdmark commented Dec 20, 2024

I added instructions to the wiki how to run the spectests locally.

https://github.com/Netatalk/netatalk/wiki/Testing#run-the-spectest-test-suite

These are written for Debian, but if you modify the user/group steps it should apply to macOS too. If you want to try to run the testsuite. :)

@rdmark
Copy link
Member

rdmark commented Dec 25, 2024

@NJRoadfan I got around to setting up and running the testsuite on my macOS Sequoia host. For the most part, it works well with these changes. However, exactly one test goes from passing to failing after this PR.

  • FPOpenFork:test431: check AppleDouble conversion from v2 to ea - FAILED

As a side note, 8 other tests are failing on the macOS host, both before and after this PR.

  • FPGetFileDirParms:test420: FPGetFileDirParms an open file is renamed with local fs - FAILED
  • FPMoveAndRename:test302: file renamed by someone else, cnid not updated - FAILED
  • FPResolveID:test331: Resolve ID file modified with local fs - FAILED
  • FPResolveID:test360: Resolve ID file modified with local fs and a file is opened - FAILED
  • FPResolveID:test412: Resolve ID file modified with local fs - FAILED
  • FPResolveID:test413: Resolve ID file modified with local fs - FAILED
  • Error:test102: access error but not cname - FAILED
  • Error:test103: did access error - FAILED

@NJRoadfan
Copy link
Contributor Author

At least nothing broke with the change to the resource fork storage. I haven't tested adouble_v2 conversions yet, so not surprised.

There are still some issues that need to be dealt with in terms of using the native FinderInfo and files that were added to the share from outside of Netatalk (ie: you copy files from your downloads folder into a share). Right now, the calls only read the native FinderInfo if there is already a Netatalk AppleDouble EA for the file.

MacOS9 and OSX both only call FPGetFileDirParms when opening folders (OS8 also calls FPSetFileParams, creating the Netatalk EAs). Reading com.apple.FinderInfo will be needed during the calls, not just in the adouble library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants