Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testsuite: File ID change after local file system operation on macOS hosts #1825

Open
rdmark opened this issue Dec 25, 2024 · 9 comments · May be fixed by #1826
Open

testsuite: File ID change after local file system operation on macOS hosts #1825

rdmark opened this issue Dec 25, 2024 · 9 comments · May be fixed by #1826

Comments

@rdmark
Copy link
Member

rdmark commented Dec 25, 2024 •
edited
Loading

Six testsuite test cases that deal with local file system file IDs fail when running on macOS Sequoia hosts:

  • FPGetFileDirParms:test420: FPGetFileDirParms an open file is renamed with local fs - FAILED
  • FPMoveAndRename:test302: file renamed by someone else, cnid not updated - FAILED
  • FPResolveID:test331: Resolve ID file modified with local fs - FAILED
  • FPResolveID:test360: Resolve ID file modified with local fs and a file is opened - FAILED
  • FPResolveID:test412: Resolve ID file modified with local fs - FAILED
  • FPResolveID:test413: Resolve ID file modified with local fs - FAILED

One example of simplified logic from test360:

  1. Create a file for testing FPCreateFile(Conn, vol, 0, dir , name)
  2. Get parameters FPGetFileDirParams(Conn, vol, dir , name, bitmap,0)
  3. Unpack afp_filedir_unpack(&filedir, dsi->data +ofs, bitmap, 0)
  4. Store the ID fid = filedir.did
  5. Rename the file rename(temp, temp1)
  6. Unpack afp_filedir_unpack(&filedir, dsi->data +ofs, bitmap, 0)
  7. Assert that the ID remains the same if fid != filedir.did test_failed()

On Linux, the ID remains the same. on macOS, the ID changes.
@rdmark
Copy link
Member Author

rdmark commented Dec 25, 2024 

afp_spectest -7 -h 127.0.0.1 -p 548 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1 -v -f test420
Connecting to host 127.0.0.1:548
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk1"
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk2"
[FPOpenVolFull] Open Vol test1 bitmap 21
############## entering test420 ##############
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t420 dir>
directory ID 0x5f3
[FPCreateFile] Create File SOFT Vol 256 did : 0x5f3 <t420 file>
[FPOpenFork] Open Fork resource Vol 256 did : 0x5f3 <t420 file> access 33
fork refnum ID 1
[FPByteLock] Byte lock,fork 1, start, lock, offset: 0, size: 100
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5f3 <t420 file>
[FPResolveID] Resolve ID Vol: 256 id: 0x5f4 120
rename /tmp/afptest1/t420 dir/t420 file /tmp/afptest1/t420 dir/t420 file new name
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5f3 <t420 file new name>
	FAILED FPGetFileDirParams id differ f4050000 f5050000
	FAILED
[FPCloseFork] Close Fork 1
[FPDelete] FPDelete conn 47809800 Vol 256 did : 0x5f3 <t420 file new name>
[FPDelete] FPDelete conn 47809800 Vol 256 did : 0x5f3 <t420 file>
header.dsi_code       -5018	AFPERR_NOOBJ   
[FPDelete] FPDelete conn 47809800 Vol 256 did : 0x5f3 <>
FPGetFileDirParms:test420: FPGetFileDirParms an open file is renamed with local fs - FAILED
[FPCloseVol] Close Vol 256
[FPLogOut] Logout
[FPLogOut] Logout

@rdmark
Copy link
Member Author

rdmark commented Dec 25, 2024 

afp_spectest -7 -h 127.0.0.1 -p 548 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1 -v -f test302
Connecting to host 127.0.0.1:548
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk1"
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk2"
[FPOpenVolFull] Open Vol test1 bitmap 21
############## entering test302 ##############
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t302 dir>
directory ID 0x5f6
[FPCreateFile] Create File SOFT Vol 256 did : 0x5f6 <t302 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5f6 <t302 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5f6 <t302 file1>
	FAILED id are not the same 1527 1528
	FAILED
[FPDelete] FPDelete conn 34809800 Vol 256 did : 0x5f6 <t302 file1>
[FPDelete] FPDelete conn 34809800 Vol 256 did : 0x2 <t302 dir>
FPMoveAndRename:test302: file renamed by someone else, cnid not updated - FAILED
[FPCloseVol] Close Vol 256
[FPLogOut] Logout
[FPLogOut] Logout

@rdmark
Copy link
Member Author

rdmark commented Dec 25, 2024 

afp_spectest -7 -h 127.0.0.1 -p 548 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1 -v -f test331
Connecting to host 127.0.0.1:548
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk1"
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk2"
[FPOpenVolFull] Open Vol test1 bitmap 21
############## entering test331 ##############
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t331 dir>
directory ID 0x5f9
[FPCreateFile] Create File SOFT Vol 256 did : 0x5f9 <t331 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5f9 <t331 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5f9 <t331 file>
[FPResolveID] Resolve ID Vol: 256 id: 0x5fa 120
rename /tmp/afptest1/t331 dir/t331 file /tmp/afptest1/t331 dir/t331 file new name
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5f9 <t331 file new name>
	FAILED FPGetFileDirParams id differ fa050000 fb050000
	FAILED
[FPDelete] FPDelete conn 4e809800 Vol 256 did : 0x5f9 <t331 file new name>
[FPDelete] FPDelete conn 4e809800 Vol 256 did : 0x5f9 <t331 file>
header.dsi_code       -5018	AFPERR_NOOBJ   
[FPDelete] FPDelete conn 4e809800 Vol 256 did : 0x5f9 <>
FPResolveID:test331: Resolve ID file modified with local fs - FAILED
[FPCloseVol] Close Vol 256
[FPLogOut] Logout
[FPLogOut] Logout

@rdmark
Copy link
Member Author

rdmark commented Dec 25, 2024 

afp_spectest -7 -h 127.0.0.1 -p 548 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1 -v -f test360
Connecting to host 127.0.0.1:548
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk1"
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk2"
[FPOpenVolFull] Open Vol test1 bitmap 21
############## entering test360 ##############
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t360 dir>
directory ID 0x5fc
[FPCreateFile] Create File SOFT Vol 256 did : 0x5fc <t360 file>
[FPCreateFile] Create File SOFT Vol 256 did : 0x5fc <t360 open file>
[FPOpenFork] Open Fork resource Vol 256 did : 0x5fc <t360 open file> access 33
fork refnum ID 1
[FPByteLock] Byte lock,fork 1, start, lock, offset: 0, size: 100
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5fc <t360 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5fc <t360 file>
[FPResolveID] Resolve ID Vol: 256 id: 0x5fd 120
rename /tmp/afptest1/t360 dir/t360 file /tmp/afptest1/t360 dir/t360 file new name
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x5fc <t360 file new name>
	FAILED FPGetFileDirParams id differ fd050000 ff050000
	FAILED
 
---------------------
open("/tmp/afptest1/t360 dir/t360 open file", O_RDWR)
fcntl(1024)
[FPCloseFork] Close Fork 1
[FPDelete] FPDelete conn 4d009200 Vol 256 did : 0x5fc <t360 open file>
[FPDelete] FPDelete conn 4d009200 Vol 256 did : 0x5fc <t360 file new name>
[FPDelete] FPDelete conn 4d009200 Vol 256 did : 0x5fc <t360 file>
header.dsi_code       -5018	AFPERR_NOOBJ   
[FPDelete] FPDelete conn 4d009200 Vol 256 did : 0x5fc <>
FPResolveID:test360: Resolve ID file modified with local fs and a file is opened - FAILED
[FPCloseVol] Close Vol 256
[FPLogOut] Logout
[FPLogOut] Logout

@rdmark
Copy link
Member Author

rdmark commented Dec 25, 2024 

afp_spectest -7 -h 127.0.0.1 -p 548 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1 -v -f test412
Connecting to host 127.0.0.1:548
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk1"
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk2"
[FPOpenVolFull] Open Vol test1 bitmap 21
############## entering test412 ##############
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t412 dir>
directory ID 0x601
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t412 dir dest>
directory ID 0x602
[FPCreateFile] Create File SOFT Vol 256 did : 0x601 <t412 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x601 <t412 file>
[FPResolveID] Resolve ID Vol: 256 id: 0x603 120
rename /tmp/afptest1/t412 dir /tmp/afptest1/t412 dir dest/t412 dir
[FPResolveID] Resolve ID Vol: 256 id: 0x603 120
header.dsi_code       -5034	AFPERR_NOID    
[FPEnumerate] Enumerate Vol 256 did : 0x602 <>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x601 <t412 file>
header.dsi_code       -5018	AFPERR_NOOBJ   
	FAILED
[FPDelete] FPDelete conn 4c809800 Vol 256 did : 0x601 <t412 file>
header.dsi_code       -5018	AFPERR_NOOBJ   
[../test/testsuite/T2_FPResolveID.c:630] FPDelete(Conn, vol, dir1, name)
	FAILED
[FPDelete] FPDelete conn 4c809800 Vol 256 did : 0x601 <>
header.dsi_code       -5018	AFPERR_NOOBJ   
[../test/testsuite/T2_FPResolveID.c:631] FPDelete(Conn, vol, dir1, "")
	FAILED
[FPDelete] FPDelete conn 4c809800 Vol 256 did : 0x602 <>
header.dsi_code       -5007	AFPERR_DIRNEMPT
[../test/testsuite/T2_FPResolveID.c:632] FPDelete(Conn, vol, dir2, "")
	FAILED
FPResolveID:test412: Resolve ID file modified with local fs - FAILED
[FPCloseVol] Close Vol 256
[FPLogOut] Logout
[FPLogOut] Logout

@rdmark
Copy link
Member Author

rdmark commented Dec 25, 2024 

afp_spectest -7 -h 127.0.0.1 -p 548 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1 -v -f test413
Connecting to host 127.0.0.1:548
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk1"
[FPopenLoginExt] LoginExt Version: "AFP3.4" uam: "Cleartxt Passwrd" user: "atalk2"
[FPOpenVolFull] Open Vol test1 bitmap 21
############## entering test413 ##############
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t413 dir>
directory ID 0x609
[FPCreateDir] Create Directory Vol 256 did : 0x2 <t413 dir dest>
directory ID 0x60a
[FPCreateFile] Create File SOFT Vol 256 did : 0x609 <t413 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x609 <t413 file>
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x609 <t413 file>
[FPResolveID] Resolve ID Vol: 256 id: 0x60b 120
rename /tmp/afptest1/t413 dir/t413 file /tmp/afptest1/t413 dir dest/t413 file
[FPGetFileDirParams] GetFileDirParams Vol 256 did : 0x60a <t413 file>
	FAILED FPGetFileDirParams id differ b060000 c060000
	FAILED
[FPDelete] FPDelete conn 56009800 Vol 256 did : 0x60a <t413 file>
[FPDelete] FPDelete conn 56009800 Vol 256 did : 0x609 <t413 file>
header.dsi_code       -5018	AFPERR_NOOBJ   
[FPDelete] FPDelete conn 56009800 Vol 256 did : 0x609 <>
[FPDelete] FPDelete conn 56009800 Vol 256 did : 0x60a <>
FPResolveID:test413: Resolve ID file modified with local fs - FAILED
[FPCloseVol] Close Vol 256
[FPLogOut] Logout
[FPLogOut] Logout

@rdmark
Copy link
Member Author

rdmark commented Dec 25, 2024

As an experiment, I modified test420 to use FPMoveAndRename() (AFP command) instead of rename_unix_file() (system call). The file retains the ID in this scenario, and the assertion passes.

@NJRoadfan In my surface level reading of the AFP3 spec, I couldn't find anything that said what the correct CNID behavior is when modifying the host's file system directly. At the same time, it does say that when using FPMoveAndRename only the parent dir ID and time stamps should change, and all other parameters remain the same. So the AFP command itself is operating as specified.

Do you have any insights here?

rdmark added a commit that referenced this issue Dec 25, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
3334dfe
@rdmark rdmark linked a pull request Dec 25, 2024 that will close this issue
testsuite: Cross-platform compatible file ID tests #1826
Open
rdmark added a commit that referenced this issue Dec 25, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
c6d7531
rdmark added a commit that referenced this issue Dec 25, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
16a4d35
rdmark added a commit that referenced this issue Dec 25, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
bfd3739
rdmark added a commit that referenced this issue Dec 25, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
a048c37
rdmark added a commit that referenced this issue Dec 25, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
e74f521
@NJRoadfan
Copy link
Contributor

Does the behavior change if cnid dev = no is set in afp.conf? Are the file inodes changing after the file move operations? Netatalk appears to track CNID-to-file mapping using the inode and optionally the dev (although enabled by default).

@rdmark
Copy link
Member Author

rdmark commented Dec 26, 2024

@NJRoadfan Off the top of my head, I'm not sure how to dynamically validate the inodes of the test files that are created and modified during test runtime. Additional logging might be required.

I tried the cnid dev option, but it didn't make a difference...

A quick note on progress: I have workarounds for most of the tests, except for test412. This test moves a dir containing a file into another dir, creating a new nested dir structure on the fly. This particular operation seems to upset the CNID of the former dir and file. As far as AFP is concerned, they don't exist anymore at that point, and operations like FPGetFileDirParams or FPDelete fail.

Under real usage (macOS client) this doesn't cause any actual harm. I assume the CNID database gets refreshed by another mechanism.

rdmark added a commit that referenced this issue Dec 26, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
7408644
rdmark added a commit that referenced this issue Dec 26, 2024
@rdmark
testsuite: Cross-platform compatible file ID tests, GitHub #1825
9a7b404
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

testsuite: Cross-platform compatible file ID tests Netatalk/netatalk
2 participants
@rdmark @NJRoadfan